Results 1 to 5 of 5

Thread: [GUIDE]A simple tutorial to making a Color Recognition and Tracking Bot in c++/c

  1. #1
    Join Date
    Aug 2013
    Posts
    8
    Mentioned
    0 Post(s)
    Quoted
    0 Post(s)

    Default [GUIDE]A simple tutorial to making a Color Recognition and Tracking Bot in c++/c

    Hello, this is a simple and easy to understand tutorial on this subject. This program is rather simple, all it does it Track and Click once. I prefer to think of this as a Proof of Concept code and can be expanded upon to include more things. I'll be using the RuneScape's client in Runespan to farm runes in order to demostrate this program, however this can be applied to anything. With Color Bots you trade off the information for simplicity When I first got into it, the information was spread and sparse. I decided to compile it all and hopefully give insight and help you make your own. So without further adieu, let's start.


    ##########################
    Requirements for the compiling and running your program is simple.
    • Requirements.
    • 1: A compiler. ***I had trouble using Code:Blocks, and others. So I just used Visual C++ 2010 off of the Microsoft website.
    • 2: OpenCv ***This was the deal breaker with the trouble I had setting up in compilers. I'll write a mini tut for this later. At the time writing this, I used the version 2.4.6
    • 3: (Optional) The RuneScape client ***This does NOT MEAN you need to do this with RuneScape, this can we put to anything.

    ##########################
    The source code I will be using be fairly short, less than 150 lines. I'll do my best to explain it function by function without overwhelming you.

    Declarations.
    Code:
    //Color Tracking Runescape
    //Goals:############## I outline my Goals to breakdown everything and remember what I am doing.
    
    //01:Recognize color of creature and feed into a picture of live in game a window.
    //02:Move Cursor to Postion of creature 
    //03:Click
    //04:Repeat
    //Reasoning
    
    
    #include <stdafx.h> Precompiled header to speed up compiling
    #include <opencv2/core/core.hpp> File used to run OpenCv functions
    #include <opencv2/imgproc/imgproc.hpp> See above
    #include <opencv2/highgui/highgui.hpp> See above
    #include <opencv/cv.h> See above
    #include <stdio.h> File used to run C functions
    #include <Windows.h> File used to run functions in hwnd2mat(), its so I can operate and grab Window's Window
    #include <time.h>    File used to stop the program in order to stop it from progressing 
    
    using namespace cv; Used for running OpenCv specific commands.
    
    
    //Global declarations
    HWND hwnd = NULL; //Variable for Window's Window to be used Globally I declare is globally to be used in any function.
    Function used to control the mouse
    Code:
    void MouseEvents(float x,float y){
    		SetCursorPos(x*2.31,y*2.2); //Send Mouse to Position of detected object NEEDS TO BE CHANGED DEPENDING ON YOUR SITUATION. In a later version I'll be improving it because I need something that will work without user input. But I've ran this for 24 hours without fouling, but still need not to be lazy and help others out.
    		mouse_event(MOUSEEVENTF_LEFTDOWN, NULL, NULL, NULL, NULL); Left Click Down
    		Sleep(10);
    		mouse_event(MOUSEEVENTF_LEFTUP, NULL, NULL, NULL, NULL);   Left Click Up
    }
    Function used to track the Color

    Code:
    void trackObject(Mat gray_image1){
        // Calculate the moments of 'gray_image1'
        CvMat ipl_img = gray_image1;                         //Convert your filtered image to a CvMat
        CvMoments *moments = (CvMoments*)malloc(sizeof(CvMoments)); //Allocates memory for your predicted movements 
        cvMoments(&ipl_img, moments, 1);                     //Gets the which way it is currently 
        double moment10 = cvGetSpatialMoment(moments, 1, 0); //Math
        double moment01 = cvGetSpatialMoment(moments, 0, 1); //More Math
        double area = cvGetCentralMoment(moments, 0, 0);	 //Gets the greatest grouping of stuff
        int posX = moment10/area;                            //Gets the X Position of the detected grouping
        int posY = moment01/area;                            //Gets the Y Position of group
        //printf("Area:%f\n", area); //Area is 1-3 when he is dead, You can uncomment this to discover the area of the creature and NOISE of the place around you. Noise are false positives you will see
        // if the area<30, I consider that the there are no object in the image and it's because of the noise, the area is not zero 
        if(area>11){ For this example 11 depends on how BIG YOU WANT it to be. Uncomment above Area to find you want
    		timer++; I wanted time enough between each click, but I wanted it to be able to correct itself in timely manner
    		printf("\nTimer:%d", timer);
    		if(timer==550){ //(Variable/20 = how much time between each click. I prefer to wait till it dies.
    			timer = 0; Start over timer between clicks
    			// calculate the position of the ball       
    			//printf("X:%d", posX);
    			//printf("Y:%d\n", posY); Uncomment these for Debug purposes
    			MouseEvents(posX,posY); Use the Arguments given posX,posY by this function.
    			
    		}
    	}
    	if(area<10){ //He is dead, takes about 5-7 seconds to resawn
    		printf("\nWaiting, for respawn....\n");
    		Sleep(7000); Sleep for 7000 milliseconds, it will stop any progress in the program. I will probably will remove this because it seems lazy to me. 
    		timer = 549; Sets time 549 so it can equal to above if statement.
    
    	}
         free(moments); This frees the memory that was allocated to the variable to prevent a crash because of running out of memory
    }
    Important, to many, this will be a confusing function. You do not need to understand it completely, if you want to you can skip learning this.

    This takes a HWND (Handle, think of it as a Unique Identifier to a Window) and converts it to BitMap (image), and then into a Mat (Matrix) that OpenCv library functions can use.

    Code:
    Mat hwnd2mat(HWND hwnd){
    
        HDC hwindowDC,hwindowCompatibleDC;
    
        int height,width,srcheight,srcwidth;
        HBITMAP hbwindow;
        Mat src;
        BITMAPINFOHEADER  bi;
    
        hwindowDC=GetDC(hwnd);
        hwindowCompatibleDC=CreateCompatibleDC(hwindowDC);
        SetStretchBltMode(hwindowCompatibleDC,COLORONCOLOR);  
    
        RECT windowsize;    // get the height and width of the screen
        GetClientRect(hwnd, &windowsize);
    
        srcheight = windowsize.bottom;
        srcwidth = windowsize.right;
        height = windowsize.bottom/2;  //Change this to whatever size you want to resize to
        width = windowsize.right/2;    The bigger this is, the more resources it will take to render this, not much but it is noticeable 
    
        src.create(height,width,CV_8UC4);
    
        // create a bitmap
        hbwindow = CreateCompatibleBitmap( hwindowDC, width, height);
        bi.biSize = sizeof(BITMAPINFOHEADER);   
        bi.biWidth = width;    
        bi.biHeight = -height;  //this is the line that makes it draw upside down or not
        bi.biPlanes = 1;    
        bi.biBitCount = 32;    
        bi.biCompression = BI_RGB;    
        bi.biSizeImage = 0;  
        bi.biXPelsPerMeter = 0;    
        bi.biYPelsPerMeter = 0;    
        bi.biClrUsed = 0;    
        bi.biClrImportant = 0;
    
        // use the previously created device context with the bitmap
        SelectObject(hwindowCompatibleDC, hbwindow);
        // copy from the window device context to the bitmap device context
        StretchBlt( hwindowCompatibleDC, 0,0, width, height, hwindowDC, 0, 0,srcwidth,srcheight, SRCCOPY); 
        GetDIBits(hwindowCompatibleDC,hbwindow,0,height,src.data,(BITMAPINFO *)&bi,DIB_RGB_COLORS);  //copy from hwindowCompatibleDC to hbwindow
    	
        DeleteObject (hbwindow); 
        DeleteDC(hwindowCompatibleDC);
        ReleaseDC(hwnd, hwindowDC);
        return src;
    	
    	
    }
    This is the starting point, and the end of the source code.
    Code:
    int main()
    {
     hwnd = FindWindowA(NULL, "RuneScape"); //Grab Runescape's HWND with this
     while(1){ Never ending loop, press Esc Key to exit of
     Mat src = hwnd2mat(hwnd); Take output from funtion hwnd2mat with HWND you supplied with above line and store into a variable
     Mat gray_image;           
     Mat gray_image1;
     cvtColor( src, gray_image, CV_BGR2HSV ); Converts the Matrix from the above variable into a Binary image, it everything not in the Scalar() function black and everything within it white (more on that below)
     inRange(gray_image, Scalar(145,0,0), Scalar(148,150,150), gray_image1); This scale right here is set to pick up 
     
    You can get the Values of Scalar with MsPaint, simply just take a screenshot with "PrtSc" and paste it into Paint, then use the color picker (the pen with a blue top in mspaint) to get the color. Click "Edit colors" to get the HSV values (HUE, Sat, and Lum). Multiply the HUE by .75 (because mspaint's max is 240 while opencv's is 180) and round down. I keep the next to variables at 0 and adjust depending on the amount of Noise I get from the "area"  
    
     trackObject(gray_image1); Tracks the color of the object using the arguements supplied from Scalar() above
     
     namedWindow( "Runescape", CV_WINDOW_AUTOSIZE );    //Debuging purposes Creates a window with a titled called "RuneScape"
     namedWindow( "Gray image", CV_WINDOW_AUTOSIZE );   //Debuging purposes Creates a second window with a title
     imshow( "Runescape", src );			    //Debuging purposes Feeds the image of Runescape into the Window called "Runescape"
     imshow( "Gray image", gray_image1 );		    //Debuging purposes Feeds the image of the Filtered Runescape into the window called Gray Image
     char esc = cvWaitKey(33); Wait 33 Milisecond before starting the While loop over
     if(esc == 27){break;} Esc on the Asci table is 27, press Esc to stop the program
     }                      IMPORTANT: YOU MUST CLICK THE WINDOW AND THEN PRESS ESC
     return 0;
    }
    This shows me draining blood runes from a creature in runespan. When the timer reaches 550 is will click again, when it dies it will wait 7 seconds and click again. I give it more second to respawn encase of lag. I prefer to keep the entire island in view so you can click where ever you see the creature, Direction doesnt really matter as long as you can see where it walks.
    Color bot.jpg

    Setting up OpenCv
    I used this set up OpenCv, he provides a simple guide in which I learned from.


    This is the end of the guide, feel free to correct me where I am fault and post ideas.

    Villavu.com, vist and bot on.

  2. #2
    Join Date
    Oct 2011
    Posts
    422
    Mentioned
    15 Post(s)
    Quoted
    116 Post(s)

  3. #3
    Join Date
    Aug 2013
    Posts
    8
    Mentioned
    0 Post(s)
    Quoted
    0 Post(s)

    Default

    AS a quick replay to my own thread, I have improved the function for finding Tracking and finding the creature. I noticed that it had a HORRIBLE and Inefficient way of waiting for the re-spawn of the creature. I'd say replace the above code with the respective below code.

    trackObject() Function. UPGRADES: Rather than waiting 7 seconds. It will wait until he is dead rather than clicking on him again while he is already being killed. So if you want to, wait until he is dead, but it might be preference.

    Also added a counter to see how many monsters you have killed. This will go for 6 hours, because that is when the Runescape client will kick you off. However, I have had it multiple times and they will NOT flag your account. You will need a work around for the 6 hour limit, if you didn't know already how too.

    IMPORTANT: There is no reason for me to have a Sleep() function in this. I had it because it was required in older version of this and it just kinda fell in with the other code. Up next I'll be covering a tut on Adding in Priorities to targets (IE: If they both are here, click on this Target first.
    Code:
    		
    
    
                if(area>30){
    			timer++;
    			//printf("Timer:%d\n", timer);
    			if(timer==999){ //(Variable/20 = how much time between each click. I prefer to wait till it dies.
    				timer = 0;
    				// calculate the position of the ball       
    				//printf("X:%d", posX);
    				//printf("Y:%d\n", posY);
    				kill++;
    				MouseEvents(posX,posY);
    			
    			}
    		}
    		if(area<25){ //He is dead, takes about 5-7 seconds to respawn
    			printf("Total Kills:%d\n", kill);
                            
    			timer = 997;
    		}
    	}
    Results of a 6 hour shift:

    2929 tokens at 23:11
    10250 tockens at 04:59
    6 Hours: 7321 Tokens
    6 Hours: 2440 Blood Runes
    6 Hours: 178144 Exp
    Last edited by Algain; 09-01-2013 at 07:00 AM.

  4. #4
    Join Date
    Sep 2010
    Posts
    5,762
    Mentioned
    136 Post(s)
    Quoted
    2739 Post(s)

    Default

    Looks nice, you can put code into c++ blocks doing [highlight=c++[/highlight (add brackets)

  5. #5
    Join Date
    Apr 2016
    Posts
    1
    Mentioned
    0 Post(s)
    Quoted
    0 Post(s)

    Default

    Hi !, i try your code and minimalize all errors and last error i can't solve it: timer: undeclared identifier.. help please.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •