Finding objects can be really tedious for people to learn, hope this helps.

Be aware that this tutorial is written for SRL for OSRS, not any other library. Certain constructs are the same, but the naming-scheme is that of SRL's.

Table of contents:

- srl.FindColors
- ClusterTPA & SplitTPA
- T2DPointArray -> SortByMiddle
- T2DPointArray -> SortByIndex
- T2DPointArray -> SortBySize
- T2DPointArray -> FilterSize
- Putting it together

TSRL.FindColors¶

• functionTSRL.FindColors(outTPA: TPointArray; Color: TCTS2Color; Area: TBox): UInt32;

This methods searches the screen in the area given for all pixels matching the given input color and returns a list of those colors, this is known as a TPA, `array ofTPoint`, which simply means it's a list of points / coordinates.

To access each individual point in that list you have to keep in mind that the list starts at 0, and ends at `High(MyList)` access it like `MyList[0]`, `MyList[1]`, `MyList[2]`, etc. The length (how many items it contains) is accessible by calling `Length(MyList)`. For normal simple usage accessing individual points like this isn't needed, but it's important to know and understand these basics.

If you look to the right, and see how it gathers all the blue pixels in the image, now imagine what happens if there aretwo mario's in that animation, you'd have a single list containing the first Mario's pants and the second Mario's pants all jumbled up together. Later we are gonna talk about how we separate the two pants, so that they are in each their list.

Related

•functionCTS2(Color, Tolerance: Int32; HueMod, SatMod: Extended=0.2): TCTS2Color;

•functionBox(X1,Y1, X2,Y2: Int32): TBox;

•TBox=recordX1,Y1, X2,Y2: Int32end;

•TPoint=recordX,Y: Int32end;

Explaining the parameters:

out TPA: TPointArray;

outindicates that this is a Result parameter, which means that the function returns something here.

This function returns the list of points that was gathered, see animation, in the bottom of it you see it builds a list of two dimensional points (coordinates). This list is what's returned here.

Color: TCTS2Color;

The parameter expects a "CTS2" color. There's a function in SRL that's named `CTS2` which you can use to build such a color. The function `CTS2` expects at minimum two parameter, `Color` and `Tolerance`, but accept two additional parameters which gives you better precision when searching for colors, these two extra parameters are `HueMod` and `SatMod`.

myColor := CTS2($54323, 15);

myAccurateColor := CTS2($54323, 15, 0.754, 0.094);

Area: TBox;

First you need to understand that a TBox consists of two coordinates, an upper-left coordinate, and a lower-right coordinate.

This box explains where on the screen you wish to search.

To create such a structure we use a function namedBox(see related definitions above)

myBox := Box(10,10, 100,100);

The result

The result of this function is the length of the list, same asLength(myList).

Putting it together:

Simba Code:{$I SRL/OSR.simba}

var

TPA: TPointArray; //the list in which we store the points

begin

if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then

WriteLn('I found ',Length(TPA),' points');

end; Animation, where we search for blue colors (his pants).

ClusterTPA & SplitTPA¶

• functionTPointArray.Cluster(Dist: Int32): T2DPointArray;

•functionTPointArray.Cluster(DistX, DistY: Int32): T2DPointArray;

ClusterTPA (and SplitTPA) is a density-based clustering algorithm: given a set of points, it groups together points that are closely packed together.

In SRL we wrap it under the nameTPointArray.Cluster.

ClusterTPA walks through the list of points, starting with the first point in the list, all points that are close to this point is merged into a group with this point, once this point connect to no other point, it moves to the next point in the group, and checks if it connects to any point not already grouped, and so it goes on until every point in the group has been used to check if they have any "neighbors" that should be added to the group. Once that group is completed, a new group will be created from the first next ungrouped point in the list, where all points near it will be merged into a group with this point, etc... And so it repeat until there are no more ungrouped points in the list.

"Close" is defined by the `dist` parameter, ex `dist=3` would mean that points that are closer than or equal to 3 pixels away are considered close enough to merge into a singular group.

ClusterTPA is particularly useful when it comes to separating objects in the game. Say you search for a rock to mine, as always you will first need to search for colors matching the color of a rock, however there are many rocks, so you will get a singular list containing all the coordinates that had the color of that rock. Now to separate each rock into each their list you need to somehow split them into each their list, this is where ClusterTPA & SplitTPA comes into the picture, it place each of the rocks in each their group, so long as there is nothing connecting the rocks (so they visually form a single object).

For reference `ATPA` / `T2DPointArray` is defined as `array ofTPointArray`.

Putting it together:

Simba Code:{$I SRL/OSR.simba}

var

TPA: TPointArray; //the list in which we store the points

ATPA: T2DPointArray; //the groups we found

begin

if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then

begin

WriteLn('I found ',Length(TPA),' points');

ATPA := TPA.Cluster(4);

WriteLn('I split the points into ',Length(ATPA),' groups');

end;

end; Animation of ClusterTPA

T2DPointArray.SortByMiddle¶

• procedureT2DPointArray.SortByMiddle(From: TPoint);

Alright, so now we have a ton of groups, in order to click the group closest to our-self we change the order in which the groups are stored, so that the group closest to us is the first one in the list of groups. This is where `T2DPointArray.SortByMiddle` comes to play. It will reorder the groups, so that the first group in the list is closest to the given point, which in our case would be the center of the mainscreen, the second group is the second closest to us, and so on.

However, this method works based of the mean / geometric middle of the group, so if you have a circle like in the animation the middle of the circle, well that's the center of the circle, so even tho the circle is far away from the center of the image it will be considered very close to the center, see animation.

Putting it together:

Simba Code:{$I SRL/OSR.simba}

var

TPA: TPointArray; //the list in which we store the points

ATPA: T2DPointArray; //the groups we found

begin

if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then

begin

WriteLn('I found ',Length(TPA),' points');

ATPA := TPA.Cluster(4);

WriteLn('I split the points into ',Length(ATPA),' groups');

// Make the first group in the list of groups `ATPA` closest to us.

ATPA.SortByMiddle(mainscreen.GetMiddle);

end;

end; Animation of sorted by middle, and how the middle affects order

T2DPointArray.SortByIndex¶

• procedureT2DPointArray.SortByIndex(From: TPoint; Index: Int32=0);

So, to avoid the above problem the method SortByIndex solves the sorting by instead of sorting by the geometric mean, it instead sorts the groups by looking at the n-th coordinate in the group, and checking how far away that's from the given point (in our case center). By default it will use the first coordinate in every group `index: Int32 = 0`.

Putting it together:

Simba Code:{$I SRL/OSR.simba}

var

TPA: TPointArray; //the list in which we store the points

ATPA: T2DPointArray; //the groups we found

begin

if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then

begin

WriteLn('I found ',Length(TPA),' points');

ATPA := TPA.Cluster(4);

WriteLn('I split the points into ',Length(ATPA),' groups');

// Make the first group in the list of groups `ATPA` closest to us.

ATPA.SortByIndex(mainscreen.GetMiddle);

end;

end; Animation of sorted by first index

T2DPointArray.SortBySize¶

• procedureT2DPointArray.SortBySize(Size: Int32=0; ClosestFirst: Boolean=False);

Another way of sorting the groups is to sort them by their individual group size / length, say you know you are searching for a really large object, you can sort the groups so that the largest groups are the first ones. This function can also be reversed, so that the smallest groups come out first in the order, however that's not nearly as often needed.

Putting it together:

Simba Code:{$I SRL/OSR.simba}

var

Group,TPA: TPointArray; //the list in which we store the points

ATPA: T2DPointArray; //the groups we found

begin

if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then

begin

WriteLn('I found ',Length(TPA),' points');

ATPA := TPA.Cluster(4);

WriteLn('I split the points into ',Length(ATPA),' groups');

// Make the first group in the list of groups `ATPA` largest one

ATPA.SortBySize();

// write the size of every group:

for group in ATPA do

WriteLn('This group has the size: ', Length(group));

end;

end; Animation of sorted by size

T2DPointArray.FilterSize¶

• procedureT2DPointArray.FilterSize(MinLen, MaxLen: Int32);overload;

In most cases you know that the size of the interesting groups roughly contains a certain number of points, so this is where `ATPA.FilterSize` helps out, it will remove outlier groups that does not have a length within the given range.

So for example if your list of groups are the following lengths / sizes:

•[66, 45, 9, 1, 54, 4265]

And you know that the object you are searching for roughly has 50 pixels.. more or less, you want to filter out everything notably less than that, and notably more than that.

•ATPA.FilterSize(30, 90)

Now we have removed everything smaller than 30, and every group that's larger than 90 so what you are left with is

•[66, 45, 54]

Putting it together:

Simba Code:{$I SRL/OSR.simba}

var

TPA: TPointArray; //the list in which we store the points

ATPA: T2DPointArray; //the groups we found

begin

if srl.FindColors(TPA, CTS2(1234567, 10), Box(4,4, 503, 336)) > 0 then

begin

WriteLn('I found ',Length(TPA),' points');

ATPA := TPA.Cluster(4);

WriteLn('I split the points into ',Length(ATPA),' groups');

// filter out all groups that contains less than 200 points and more than 1000 points

ATPA.FilterSize(200,1000);

WriteLn('After filtering out outliers we have ',Length(ATPA),' groups left');

// Make the first group in the list of groups `ATPA` closest to us.

ATPA.SortByIndex(mainscreen.GetMiddle);

end;

end; Filter out groups less than 4 points

`FilterSize(4,999999)`

Filter out groups less than 4 and more than 60 points

`FilterSize(4,60)`

Putting together what we have learnt so far¶

So by mainly using the methods we have learnt about so far we can write a small object finder. Keep in mind that color, tolerance and uptext-check, and the FilterSize parameters in the following code are all placeholders, you'd need to change those to fit whatever you wanna search for.

Simba Code:var

TPA,Group: TPointArray; //the list in which we store the points

ATPA: T2DPointArray; //the groups we found

i: Int32;

begin

if srl.FindColors(TPA, CTS2(1234567, 10), Mainscreen.GetBounds) > 0 then

begin

WriteLn('I found ',Length(TPA),' points');

ATPA := TPA.Cluster(4);

WriteLn('I split the points into ',Length(ATPA),' groups');

// remove none-interesting outliers so we don't have to mouse over them

ATPA.FilterSize(250, 1500);

// Make the first group in the list of groups `ATPA` closest to us.

ATPA.SortByIndex(mainscreen.GetMiddle);

// loop through every group, nearest to furthest away

for Group in ATPA do

begin

// move mouse to a random point in the group

// We access a random individual point in that group by indexing

//

// Breaking it down:

// Length(Group) returns the number of points in the group

// Random( .. ) generates a random value in the range 0..Length(Group)-1

// Group[ .. ] accesses the item at that random index.

Mouse.Move( Group[Random(Length(Group))] );

// check if the uptext matches our expectations

if MainScreen.IsUpText('Change me') then

begin

// click this object!!

Mouse.Click(ctRed);

// stop the loop, we found what we were looking for!

Break;

end;

// we end up here if the group didn't match our expectations

// so we will move to the next group and see if that's any better.

end;

end;

end;

Final notice:

- As I find time I can probably document and create animations for more of the core methods.
- Don't worry too much about it if you come across fucked up of horribly confusing sentences, that's probably just a result of me writing all of this at ones, without a break.
- Animations are made in Simba, and then recorded with a screen recording tool.
- For a smaller less detailed related tutorial see Find objects with SRL & RSWalker
- For setting up Simba for SRL and RSWalker have a look at Setting up SRL with RSWalker or Complete Guide to Setting Up Simba and SRL