PyBot [Deep learning to object detection]

[Darkscape]

interesting

[legitmail]

I used a sort of LSTM and just made predictions based on single time intervals until the correct x and y coordinate

Thanks for your reply Will try to replicate it

Location tracking based on minimap looks solid

[legitmail]

Currently testing power mining iron using pybot? 24/7 running no breaks and script have no randoms whatsoever. Now if the theory about using hardware mouse is right?I should not get banned

For how many days have you been doing that? Still no bans?

[weequ]

I checked your rock training data and it seems you are using bounding boxes as your output. Have you tried using a same size bitmap as output where each rock pixel is colored. This way the neural network would output for each pixel a probability of being part of a rock. In the script you could then choose the pixel with highest probability.

It would be great if a reflection/injection bot could be used to create those images but otherwise you could just draw a circle or square on top of each rock and convert those images to black and white bitmaps.

[Grunt]

I checked your rock training data and it seems you are using bounding boxes as your output. Have you tried using a same size bitmap as output where each rock pixel is colored. This way the neural network would output for each pixel a probability of being part of a rock. In the script you could then choose the pixel with highest probability.

If I understand correctly, you are talking about image segmentation. I haven't thought about trying it out because of 2 reasons, the first is the difficulty in collecting training data (maybe your second suggestions can fix that).

It would be great if a reflection/injection bot could be used to create those images but otherwise you could just draw a circle or square on top of each rock and convert those images to black and white bitmaps.

Using a bot to create training data is an excellent idea. It will be a very valuable contribution.

[weequ]

If I understand correctly, you are talking about image segmentation. I haven't thought about trying it out because of 2 reasons, the first is the difficulty in collecting training data (maybe your second suggestions can fix that).


Using a bot to create training data is an excellent idea. It will be a very valuable contribution.

I did a google search on that term and it seems to be roughly what I'm talking about: https://leonardoaraujosantos.gitbook...mentation.html.

I think it might be a more natural task for a neural network but I have never actually tried doing it.

[Sin]

Logged in for the first time in a few months to respond to this.
This is really great stuff. I was working on a PvM Aid with TensorFlow/OpenCV before I started my job, but this blows it out of the water. Super excited to see where this goes.

[Grunt]

update: finished amethyst bot. Added example script using finite state machine. Generate nice pretty graphs.
Screenshot from 2017-08-28 03-19-09.jpg

[One Kid]

What are some resources you've used if others want to do something similar but with different activities?

[Grunt]

What are some resources you've used if others want to do something similar but with different activities?

opencv
tensorflow (SSD for detection)
pytransitions (fsm)
mss (fast way to capture screen)

[SubiN]

Awesome work, would love to see your code .

[One Kid]

Awesome work, would love to see your code .

Agreed! I've been watching this thread closely and would love to see it open-sourced, or just the functioning code.

[KeepBotting]

Very cool. Would love to see it on GitHub so I can hack at it.

Side note: if I was looking to do something like this, where should I start? Learning Python seems like a good jumping-off point.

[Shout]

For how many days have you been doing that? Still no bans?

I'm also curious if you can provide any insight into the hardware vs. virtual mouse discussion.

[ItsTimmyMan]

Any progress? PoC code? Would this be implemented into Simba, a new framework or a combination of programs? I really look forward to any future updates!

[client_dude]

Here, I've made some training data for image segmentation

villavu.com/forum/showthread.php?t=118169

[hotyute]

Any updates on this project?

[edzilla]

Would love to work on this! Ive been thinking about designing a RL pk bot for a while...

[argothes]

Very cool project! I was actually going to embark on something very similar to this using Lazarus/Delphi so that it could be easily integrated with Simba. I'm curious, how taxing is this on your machine/what are the specs on the machines you've tested this on? I'd imagine a decent modern computer would have no problems running a single instance of this, but I'm thinking it might not scale too easily.

[onilika]

Got me quite interested on this since it’s Machine Learning related!

[folks]

Im assuming this has been abandoned but it gave me motivation to learn about ML myself. I am taking a 4** level ML class next semester so a head start wouldn't be too bad

[SubiN]

reviving this thread , found this neat source if anyone else wants to learn
https://github.com/slyautomation/osrs_yolov5