update: 8/28/2017:
Finished amethyst bot, Added support for finite state machine, (generate pretty graphs dynamically):
Update 8/05/2017
Improved rock detection, needs a little bit more work (more data) to improve detecting the different types of rocks.
Update 8/01/2017
working minimap
Updated: 6/07/2017
-Been working on a way to recognize coordinates on a map. I been testing a convolutional network that outputs just an x coordinate. I started out with a map picture of the varrock area, the scale is approximately the same as the real minimap on a client. I designated a custom coordinate system for each pixel.
varrock_map.jpg
After briefly training the network, I was able to get some impressive results.
test_1.jpgtest_2.jpgtest_3.jpgtest_4.jpg
Just from training observations, I notice I can get the loss very low, it's very possible its overfitting. The future plan is to include an output node for y coordinate, and an output node for angle.[302.53319, 358.72432, 420.84034, 372.00929]
What is pybot:
Pybot is an api that does pure color detection using deep learning, and the opencv library. The technique for object detection is SSD-multibox detector. It is implemented in tensorflow. The actual clicking is implemented through uinput, so it doesn't work on windows. I may make a windows version that controls an arduino acting as a hardware mouse. (depends on interest)
Here is a little preview:
Once this is more fleshed out, I plan to open source it. The tentative date for this is the end of June. I need to work out some logistic on how to release it.
It's not perfect yet, I trained it on very little data, alot of objects are missing. In the future I will expand the model to detect more objects.