# Thread: [SRL] The core of colorfinding

1. ## [SRL] The core of colorfinding

Finding objects can be really tedious for people to learn, hope this helps.
Be aware that this tutorial is written for SRL for OSRS, not any other library. Certain constructs are the same, but the naming-scheme is that of SRL's.

TSRL.FindColors

 • function TSRL.FindColors(out TPA: TPointArray; Color: TCTS2Color; Area: TBox): UInt32; This methods searches the screen in the area given for all pixels matching the given input color and returns a list of those colors, this is known as a TPA, array of TPoint, which simply means it's a list of points / coordinates. To access each individual point in that list you have to keep in mind that the list starts at 0, and ends at High(MyList) access it like MyList[0], MyList[1], MyList[2], etc. The length (how many items it contains) is accessible by calling Length(MyList). For normal simple usage accessing individual points like this isn't needed, but it's important to know and understand these basics. If you look to the right, and see how it gathers all the blue pixels in the image, now imagine what happens if there are two mario's in that animation, you'd have a single list containing the first Mario's pants and the second Mario's pants all jumbled up together. Later we are gonna talk about how we separate the two pants, so that they are in each their list. Related • function CTS2(Color, Tolerance: Int32; HueMod, SatMod: Extended=0.2): TCTS2Color; • function Box(X1,Y1, X2,Y2: Int32): TBox; • TBox = record X1,Y1, X2,Y2: Int32 end; • TPoint = record X,Y: Int32 end; Explaining the parameters: out TPA: TPointArray; out indicates that this is a Result parameter, which means that the function returns something here. This function returns the list of points that was gathered, see animation, in the bottom of it you see it builds a list of two dimensional points (coordinates). This list is what's returned here. Color: TCTS2Color; The parameter expects a "CTS2" color. There's a function in SRL that's named CTS2 which you can use to build such a color. The function CTS2 expects at minimum two parameter, Color and Tolerance, but accept two additional parameters which gives you better precision when searching for colors, these two extra parameters are HueMod and SatMod. myColor := CTS2($54323, 15); myAccurateColor := CTS2($54323, 15, 0.754, 0.094); Area: TBox; First you need to understand that a TBox consists of two coordinates, an upper-left coordinate, and a lower-right coordinate. This box explains where on the screen you wish to search. To create such a structure we use a function named Box (see related definitions above) myBox := Box(10,10, 100,100); The result The result of this function is the length of the list, same as Length(myList). Putting it together: Simba Code: {$I SRL/OSR.simba}var TPA: TPointArray; //the list in which we store the pointsbegin if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then WriteLn('I found ',Length(TPA),' points');end; Animation, where we search for blue colors (his pants). ClusterTPA & SplitTPA  • function TPointArray.Cluster(Dist: Int32): T2DPointArray; • function TPointArray.Cluster(DistX, DistY: Int32): T2DPointArray; ClusterTPA (and SplitTPA) is a density-based clustering algorithm: given a set of points, it groups together points that are closely packed together. In SRL we wrap it under the name TPointArray.Cluster. ClusterTPA walks through the list of points, starting with the first point in the list, all points that are close to this point is merged into a group with this point, once this point connect to no other point, it moves to the next point in the group, and checks if it connects to any point not already grouped, and so it goes on until every point in the group has been used to check if they have any "neighbors" that should be added to the group. Once that group is completed, a new group will be created from the first next ungrouped point in the list, where all points near it will be merged into a group with this point, etc... And so it repeat until there are no more ungrouped points in the list. "Close" is defined by the dist parameter, ex dist=3 would mean that points that are closer than or equal to 3 pixels away are considered close enough to merge into a singular group. ClusterTPA is particularly useful when it comes to separating objects in the game. Say you search for a rock to mine, as always you will first need to search for colors matching the color of a rock, however there are many rocks, so you will get a singular list containing all the coordinates that had the color of that rock. Now to separate each rock into each their list you need to somehow split them into each their list, this is where ClusterTPA & SplitTPA comes into the picture, it place each of the rocks in each their group, so long as there is nothing connecting the rocks (so they visually form a single object). For reference ATPA / T2DPointArray is defined as array of TPointArray. Putting it together: Simba Code: {$I SRL/OSR.simba}var  TPA: TPointArray;    //the list in which we store the points  ATPA: T2DPointArray; //the groups we foundbegin  if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then  begin    WriteLn('I found ',Length(TPA),' points');    ATPA := TPA.Cluster(4);    WriteLn('I split the points into ',Length(ATPA),' groups');  end;end; Animation of ClusterTPA

T2DPointArray.SortByMiddle

 • procedure T2DPointArray.SortByMiddle(From: TPoint); Alright, so now we have a ton of groups, in order to click the group closest to our-self we change the order in which the groups are stored, so that the group closest to us is the first one in the list of groups. This is where T2DPointArray.SortByMiddle comes to play. It will reorder the groups, so that the first group in the list is closest to the given point, which in our case would be the center of the mainscreen, the second group is the second closest to us, and so on. However, this method works based of the mean / geometric middle of the group, so if you have a circle like in the animation the middle of the circle, well that's the center of the circle, so even tho the circle is far away from the center of the image it will be considered very close to the center, see animation. Putting it together: Simba Code: {$I SRL/OSR.simba}var TPA: TPointArray; //the list in which we store the points ATPA: T2DPointArray; //the groups we foundbegin if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then begin WriteLn('I found ',Length(TPA),' points'); ATPA := TPA.Cluster(4); WriteLn('I split the points into ',Length(ATPA),' groups'); // Make the first group in the list of groups ATPA closest to us. ATPA.SortByMiddle(mainscreen.GetMiddle); end;end; Animation of sorted by middle, and how the middle affects order T2DPointArray.SortByIndex  • procedure T2DPointArray.SortByIndex(From: TPoint; Index: Int32=0); So, to avoid the above problem the method SortByIndex solves the sorting by instead of sorting by the geometric mean, it instead sorts the groups by looking at the n-th coordinate in the group, and checking how far away that's from the given point (in our case center). By default it will use the first coordinate in every group index: Int32 = 0. Putting it together: Simba Code: {$I SRL/OSR.simba}var  TPA: TPointArray;    //the list in which we store the points  ATPA: T2DPointArray; //the groups we foundbegin  if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then  begin    WriteLn('I found ',Length(TPA),' points');    ATPA := TPA.Cluster(4);    WriteLn('I split the points into ',Length(ATPA),' groups');        // Make the first group in the list of groups ATPA closest to us.    ATPA.SortByIndex(mainscreen.GetMiddle);  end;end; Animation of sorted by first index

T2DPointArray.SortBySize

 • procedure T2DPointArray.SortBySize(Size: Int32=0; ClosestFirst: Boolean=False); Another way of sorting the groups is to sort them by their individual group size / length, say you know you are searching for a really large object, you can sort the groups so that the largest groups are the first ones. This function can also be reversed, so that the smallest groups come out first in the order, however that's not nearly as often needed. Putting it together: Simba Code: {$I SRL/OSR.simba}var Group,TPA: TPointArray; //the list in which we store the points ATPA: T2DPointArray; //the groups we foundbegin if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then begin WriteLn('I found ',Length(TPA),' points'); ATPA := TPA.Cluster(4); WriteLn('I split the points into ',Length(ATPA),' groups'); // Make the first group in the list of groups ATPA largest one ATPA.SortBySize(); // write the size of every group: for group in ATPA do WriteLn('This group has the size: ', Length(group)); end;end; Animation of sorted by size T2DPointArray.FilterSize  • procedure T2DPointArray.FilterSize(MinLen, MaxLen: Int32); overload; In most cases you know that the size of the interesting groups roughly contains a certain number of points, so this is where ATPA.FilterSize helps out, it will remove outlier groups that does not have a length within the given range. So for example if your list of groups are the following lengths / sizes: • [66, 45, 9, 1, 54, 4265] And you know that the object you are searching for roughly has 50 pixels.. more or less, you want to filter out everything notably less than that, and notably more than that. • ATPA.FilterSize(30, 90) Now we have removed everything smaller than 30, and every group that's larger than 90 so what you are left with is • [66, 45, 54] Putting it together: Simba Code: {$I SRL/OSR.simba}var  TPA: TPointArray;    //the list in which we store the points  ATPA: T2DPointArray; //the groups we foundbegin  if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then  begin    WriteLn('I found ',Length(TPA),' points');    ATPA := TPA.Cluster(4);    WriteLn('I split the points into ',Length(ATPA),' groups');    // filter out all groups that contains less than 200 points and more than 1000 points    ATPA.FilterSize(200,1000);     WriteLn('After filtering out outliers we have ',Length(ATPA),' groups left');    // Make the first group in the list of groups ATPA closest to us.    ATPA.SortByIndex(mainscreen.GetMiddle);  end;end; Filter out groups less than 4 points FilterSize(4,999999) Filter out groups less than 4 and more than 60 points FilterSize(4,60)

Putting together what we have learnt so far

So by mainly using the methods we have learnt about so far we can write a small object finder. Keep in mind that color, tolerance and uptext-check, and the FilterSize parameters in the following code are all placeholders, you'd need to change those to fit whatever you wanna search for.

Simba Code:
var  TPA,Group: TPointArray;  //the list in which we store the points  ATPA: T2DPointArray;     //the groups we found  i: Int32;begin  if srl.FindColors(TPA, CTS2(1234567, 10), Mainscreen.GetBounds) > 0 then  begin    WriteLn('I found ',Length(TPA),' points');    ATPA := TPA.Cluster(4);    WriteLn('I split the points into ',Length(ATPA),' groups');    // remove none-interesting outliers so we don't have to mouse over them    ATPA.FilterSize(250, 1500);    // Make the first group in the list of groups ATPA closest to us.    ATPA.SortByIndex(mainscreen.GetMiddle);    // loop through every group, nearest to furthest away    for Group in ATPA do    begin      // move mouse to a random point in the group      // We access a random individual point in that group by indexing      //      // Breaking it down:      //   Length(Group) returns the number of points in the group      //   Random( .. ) generates a random value in the range 0..Length(Group)-1      //   Group[ .. ] accesses the item at that random index.      Mouse.Move( Group[Random(Length(Group))] );      // check if the uptext matches our expectations      if MainScreen.IsUpText('Change me') then      begin         // click this object!!         Mouse.Click(ctRed);         // stop the loop, we found what we were looking for!         Break;      end;      // we end up here if the group didn't match our expectations      // so we will move to the next group and see if that's any better.    end;  end;end;

Final notice:
• As I find time I can probably document and create animations for more of the core methods.
• Don't worry too much about it if you come across fucked up of horribly confusing sentences, that's probably just a result of me writing all of this at ones, without a break.
• Animations are made in Simba, and then recorded with a screen recording tool.
• For a smaller less detailed related tutorial see Find objects with SRL & RSWalker
• For setting up Simba for SRL and RSWalker have a look at Setting up SRL with RSWalker or Complete Guide to Setting Up Simba and SRL
Last edited by slacky; 05-02-2018 at 09:30 PM.

2. Reserved

3. Nice work slacky! I've been more and more interested in color detection and this is a good tutorial to sharpen up on that.

4. SRL Junior Member
Join Date
Aug 2012
Posts
188
Mentioned
9 Post(s)
Quoted
71 Post(s)
Great explanation on color finding! You've inspired me to actually read the code! I'm wondering, is the speed of these color finding algorithms ever an issue?

5. Originally Posted by Swag Bag
is the speed of these color finding algorithms ever an issue?
nope, nothing really worth mentioning.
Last edited by slacky; 04-23-2018 at 09:37 AM.

6. Cool animations :^)

7. Very helpful tutorial as always. Will be building a script in SRL any second now!

8. Wow, Great detail and explanations. Keep it up!

9. Very nice and detailed, thanks. What did you use to make the gifs?

10. Originally Posted by KeepBotting
Very nice and detailed, thanks. What did you use to make the gifs?
Didn't spot this before now. I made them using Simba magic, drawing to the debug image and updating, works well alongside a GIF recorder.

11. Registered User
Join Date
Sep 2019
Posts
5
Mentioned
0 Post(s)
Quoted
0 Post(s)
i'm getting error in colors.simba file