Results 1 to 11 of 11

Thread: [SRL] The core of colorfinding

  1. #1
    Join Date
    Feb 2012
    Location
    Norway
    Posts
    968
    Mentioned
    142 Post(s)
    Quoted
    582 Post(s)

    Default [SRL] The core of colorfinding

    Finding objects can be really tedious for people to learn, hope this helps.
    Be aware that this tutorial is written for SRL for OSRS, not any other library. Certain constructs are the same, but the naming-scheme is that of SRL's.

    Table of contents:






    TSRL.FindColors

    function TSRL.FindColors(out TPA: TPointArray; Color: TCTS2Color; Area: TBox): UInt32;

    This methods searches the screen in the area given for all pixels matching the given input color and returns a list of those colors, this is known as a TPA, `array of TPoint`, which simply means it's a list of points / coordinates.

    To access each individual point in that list you have to keep in mind that the list starts at 0, and ends at `High(MyList)` access it like `MyList[0]`, `MyList[1]`, `MyList[2]`, etc. The length (how many items it contains) is accessible by calling `Length(MyList)`. For normal simple usage accessing individual points like this isn't needed, but it's important to know and understand these basics.

    If you look to the right, and see how it gathers all the blue pixels in the image, now imagine what happens if there are two mario's in that animation, you'd have a single list containing the first Mario's pants and the second Mario's pants all jumbled up together. Later we are gonna talk about how we separate the two pants, so that they are in each their list.


    Related
    function CTS2(Color, Tolerance: Int32; HueMod, SatMod: Extended=0.2): TCTS2Color;
    function Box(X1,Y1, X2,Y2: Int32): TBox;
    TBox = record X1,Y1, X2,Y2: Int32 end;
    TPoint = record X,Y: Int32 end;


    Explaining the parameters:

    out TPA: TPointArray;

    out indicates that this is a Result parameter, which means that the function returns something here.
    This function returns the list of points that was gathered, see animation, in the bottom of it you see it builds a list of two dimensional points (coordinates). This list is what's returned here.

    Color: TCTS2Color;

    The parameter expects a "CTS2" color. There's a function in SRL that's named `CTS2` which you can use to build such a color. The function `CTS2` expects at minimum two parameter, `Color` and `Tolerance`, but accept two additional parameters which gives you better precision when searching for colors, these two extra parameters are `HueMod` and `SatMod`.

    myColor := CTS2($54323, 15);
    myAccurateColor := CTS2($54323, 15, 0.754, 0.094);

    Area: TBox;

    First you need to understand that a TBox consists of two coordinates, an upper-left coordinate, and a lower-right coordinate.
    This box explains where on the screen you wish to search.

    To create such a structure we use a function named Box (see related definitions above)
    myBox := Box(10,10, 100,100);

    The result

    The result of this function is the length of the list, same as Length(myList).


    Putting it together:
    Simba Code:
    {$I SRL/OSR.simba}

    var
      TPA: TPointArray; //the list in which we store the points
    begin
      if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
        WriteLn('I found ',Length(TPA),' points');
    end;
    Animation, where we search for blue colors (his pants).


    ClusterTPA & SplitTPA

    function TPointArray.Cluster(Dist: Int32): T2DPointArray;
    function TPointArray.Cluster(DistX, DistY: Int32): T2DPointArray;

    ClusterTPA (and SplitTPA) is a density-based clustering algorithm: given a set of points, it groups together points that are closely packed together.

    In SRL we wrap it under the name TPointArray.Cluster.

    ClusterTPA walks through the list of points, starting with the first point in the list, all points that are close to this point is merged into a group with this point, once this point connect to no other point, it moves to the next point in the group, and checks if it connects to any point not already grouped, and so it goes on until every point in the group has been used to check if they have any "neighbors" that should be added to the group. Once that group is completed, a new group will be created from the first next ungrouped point in the list, where all points near it will be merged into a group with this point, etc... And so it repeat until there are no more ungrouped points in the list.

    "Close" is defined by the `dist` parameter, ex `dist=3` would mean that points that are closer than or equal to 3 pixels away are considered close enough to merge into a singular group.

    ClusterTPA is particularly useful when it comes to separating objects in the game. Say you search for a rock to mine, as always you will first need to search for colors matching the color of a rock, however there are many rocks, so you will get a singular list containing all the coordinates that had the color of that rock. Now to separate each rock into each their list you need to somehow split them into each their list, this is where ClusterTPA & SplitTPA comes into the picture, it place each of the rocks in each their group, so long as there is nothing connecting the rocks (so they visually form a single object).

    For reference `ATPA` / `T2DPointArray` is defined as `array of TPointArray`.

    Putting it together:
    Simba Code:
    {$I SRL/OSR.simba}

    var
      TPA: TPointArray;    //the list in which we store the points
      ATPA: T2DPointArray; //the groups we found
    begin
      if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
      begin
        WriteLn('I found ',Length(TPA),' points');
        ATPA := TPA.Cluster(4);
        WriteLn('I split the points into ',Length(ATPA),' groups');
      end;
    end;
    Animation of ClusterTPA


    T2DPointArray.SortByMiddle

    procedure T2DPointArray.SortByMiddle(From: TPoint);

    Alright, so now we have a ton of groups, in order to click the group closest to our-self we change the order in which the groups are stored, so that the group closest to us is the first one in the list of groups. This is where `T2DPointArray.SortByMiddle` comes to play. It will reorder the groups, so that the first group in the list is closest to the given point, which in our case would be the center of the mainscreen, the second group is the second closest to us, and so on.

    However, this method works based of the mean / geometric middle of the group, so if you have a circle like in the animation the middle of the circle, well that's the center of the circle, so even tho the circle is far away from the center of the image it will be considered very close to the center, see animation.


    Putting it together:
    Simba Code:
    {$I SRL/OSR.simba}

    var
      TPA: TPointArray;    //the list in which we store the points
      ATPA: T2DPointArray; //the groups we found
    begin
      if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
      begin
        WriteLn('I found ',Length(TPA),' points');
        ATPA := TPA.Cluster(4);
        WriteLn('I split the points into ',Length(ATPA),' groups');
       
        // Make the first group in the list of groups `ATPA` closest to us.
        ATPA.SortByMiddle(mainscreen.GetMiddle);
      end;
    end;
    Animation of sorted by middle, and how the middle affects order



    T2DPointArray.SortByIndex

    procedure T2DPointArray.SortByIndex(From: TPoint; Index: Int32=0);

    So, to avoid the above problem the method SortByIndex solves the sorting by instead of sorting by the geometric mean, it instead sorts the groups by looking at the n-th coordinate in the group, and checking how far away that's from the given point (in our case center). By default it will use the first coordinate in every group `index: Int32 = 0`.


    Putting it together:
    Simba Code:
    {$I SRL/OSR.simba}

    var
      TPA: TPointArray;    //the list in which we store the points
      ATPA: T2DPointArray; //the groups we found
    begin
      if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
      begin
        WriteLn('I found ',Length(TPA),' points');
        ATPA := TPA.Cluster(4);
        WriteLn('I split the points into ',Length(ATPA),' groups');
       
        // Make the first group in the list of groups `ATPA` closest to us.
        ATPA.SortByIndex(mainscreen.GetMiddle);
      end;
    end;
    Animation of sorted by first index


    T2DPointArray.SortBySize

    procedure T2DPointArray.SortBySize(Size: Int32=0; ClosestFirst: Boolean=False);

    Another way of sorting the groups is to sort them by their individual group size / length, say you know you are searching for a really large object, you can sort the groups so that the largest groups are the first ones. This function can also be reversed, so that the smallest groups come out first in the order, however that's not nearly as often needed.


    Putting it together:
    Simba Code:
    {$I SRL/OSR.simba}

    var
      Group,TPA: TPointArray;    //the list in which we store the points
      ATPA: T2DPointArray; //the groups we found
    begin
      if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
      begin
        WriteLn('I found ',Length(TPA),' points');
        ATPA := TPA.Cluster(4);
        WriteLn('I split the points into ',Length(ATPA),' groups');
       
        // Make the first group in the list of groups `ATPA` largest one
        ATPA.SortBySize();
       
        // write the size of every group:
        for group in ATPA do
          WriteLn('This group has the size: ', Length(group));
      end;
    end;
    Animation of sorted by size


    T2DPointArray.FilterSize

    procedure T2DPointArray.FilterSize(MinLen, MaxLen: Int32); overload;

    In most cases you know that the size of the interesting groups roughly contains a certain number of points, so this is where `ATPA.FilterSize` helps out, it will remove outlier groups that does not have a length within the given range.

    So for example if your list of groups are the following lengths / sizes:
    [66, 45, 9, 1, 54, 4265]

    And you know that the object you are searching for roughly has 50 pixels.. more or less, you want to filter out everything notably less than that, and notably more than that.
    ATPA.FilterSize(30, 90)

    Now we have removed everything smaller than 30, and every group that's larger than 90 so what you are left with is
    [66, 45, 54]


    Putting it together:
    Simba Code:
    {$I SRL/OSR.simba}

    var
      TPA: TPointArray;    //the list in which we store the points
      ATPA: T2DPointArray; //the groups we found
    begin
      if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then
      begin
        WriteLn('I found ',Length(TPA),' points');

        ATPA := TPA.Cluster(4);
        WriteLn('I split the points into ',Length(ATPA),' groups');

        // filter out all groups that contains less than 200 points and more than 1000 points
        ATPA.FilterSize(200,1000);
        WriteLn('After filtering out outliers we have ',Length(ATPA),' groups left');

        // Make the first group in the list of groups `ATPA` closest to us.
        ATPA.SortByIndex(mainscreen.GetMiddle);
      end;
    end;
    Filter out groups less than 4 points
    `FilterSize(4,999999)`


    Filter out groups less than 4 and more than 60 points
    `FilterSize(4,60)`


    Putting together what we have learnt so far


    So by mainly using the methods we have learnt about so far we can write a small object finder. Keep in mind that color, tolerance and uptext-check, and the FilterSize parameters in the following code are all placeholders, you'd need to change those to fit whatever you wanna search for.

    Simba Code:
    var
      TPA,Group: TPointArray;  //the list in which we store the points
      ATPA: T2DPointArray;     //the groups we found
      i: Int32;
    begin
      if srl.FindColors(TPA, CTS2(1234567, 10), Mainscreen.GetBounds) > 0 then
      begin
        WriteLn('I found ',Length(TPA),' points');
        ATPA := TPA.Cluster(4);
        WriteLn('I split the points into ',Length(ATPA),' groups');

        // remove none-interesting outliers so we don't have to mouse over them
        ATPA.FilterSize(250, 1500);

        // Make the first group in the list of groups `ATPA` closest to us.
        ATPA.SortByIndex(mainscreen.GetMiddle);

        // loop through every group, nearest to furthest away
        for Group in ATPA do
        begin
          // move mouse to a random point in the group
          // We access a random individual point in that group by indexing
          //
          // Breaking it down:
          //   Length(Group) returns the number of points in the group
          //   Random( .. ) generates a random value in the range 0..Length(Group)-1
          //   Group[ .. ] accesses the item at that random index.
          Mouse.Move( Group[Random(Length(Group))] );

          // check if the uptext matches our expectations
          if MainScreen.IsUpText('Change me') then
          begin
             // click this object!!
             Mouse.Click(ctRed);

             // stop the loop, we found what we were looking for!
             Break;
          end;
          // we end up here if the group didn't match our expectations
          // so we will move to the next group and see if that's any better.
        end;
      end;
    end;



    Final notice:
    • As I find time I can probably document and create animations for more of the core methods.
    • Don't worry too much about it if you come across fucked up of horribly confusing sentences, that's probably just a result of me writing all of this at ones, without a break.
    • Animations are made in Simba, and then recorded with a screen recording tool.
    • For a smaller less detailed related tutorial see Find objects with SRL & RSWalker
    • For setting up Simba for SRL and RSWalker have a look at Setting up SRL with RSWalker or Complete Guide to Setting Up Simba and SRL
    Last edited by slacky; 05-02-2018 at 09:30 PM.
    !No priv. messages please

  2. #2
    Join Date
    Feb 2012
    Location
    Norway
    Posts
    968
    Mentioned
    142 Post(s)
    Quoted
    582 Post(s)

    Default

    Reserved
    !No priv. messages please

  3. #3
    Join Date
    Aug 2007
    Posts
    526
    Mentioned
    20 Post(s)
    Quoted
    262 Post(s)

    Default

    Nice work slacky! I've been more and more interested in color detection and this is a good tutorial to sharpen up on that.
    Please only contact me through SRL-Forums and through no other medium as that may not be me.

  4. #4
    Join Date
    Aug 2012
    Posts
    188
    Mentioned
    9 Post(s)
    Quoted
    71 Post(s)

    Default

    Great explanation on color finding! You've inspired me to actually read the code! I'm wondering, is the speed of these color finding algorithms ever an issue?

  5. #5
    Join Date
    Feb 2012
    Location
    Norway
    Posts
    968
    Mentioned
    142 Post(s)
    Quoted
    582 Post(s)

    Default

    Quote Originally Posted by Swag Bag View Post
    is the speed of these color finding algorithms ever an issue?
    nope, nothing really worth mentioning.
    Last edited by slacky; 04-23-2018 at 09:37 AM.
    !No priv. messages please

  6. #6
    Join Date
    Sep 2014
    Location
    C:\Simba\
    Posts
    559
    Mentioned
    9 Post(s)
    Quoted
    71 Post(s)

    Default

    Cool animations :^)
    Feel free to ask me any questions, I will do my best to answer them!

    Previously known as YouPee.

  7. #7
    Join Date
    Feb 2012
    Location
    Canada
    Posts
    1,153
    Mentioned
    25 Post(s)
    Quoted
    430 Post(s)

  8. #8
    Join Date
    Jan 2012
    Location
    Sydney, Australia
    Posts
    835
    Mentioned
    12 Post(s)
    Quoted
    345 Post(s)

  9. #9
    Join Date
    Dec 2006
    Location
    Program TEXAS home of AUTOERS
    Posts
    7,935
    Mentioned
    25 Post(s)
    Quoted
    235 Post(s)

    Default

    Wow, Great detail and explanations. Keep it up!


  10. #10
    Join Date
    Dec 2011
    Location
    East Coast, USA
    Posts
    4,197
    Mentioned
    112 Post(s)
    Quoted
    1857 Post(s)

    Default

    Very nice and detailed, thanks. What did you use to make the gifs?
    GitLab projects | Simba for Linux | Find me on IRC or Discord | OSRS scripts | Come play ScapeRune!

    <BenLand100> we're just in the transitional phase where society reclassifies guns as Bad™ before everyone gets laser pistols

  11. #11
    Join Date
    Feb 2012
    Location
    Norway
    Posts
    968
    Mentioned
    142 Post(s)
    Quoted
    582 Post(s)

    Default

    Quote Originally Posted by KeepBotting View Post
    Very nice and detailed, thanks. What did you use to make the gifs?
    Didn't spot this before now. I made them using Simba magic, drawing to the debug image and updating, works well alongside a GIF recorder.
    !No priv. messages please

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •