PDA

View Full Version : Finding Objects Using Relative Objects and Filtering



Camel
10-08-2014, 11:44 PM
Finding Objects Using Relative Objects and Filtering
By Camel

Foreword

Welcome to my guide. I wrote this guide with the assumption that the reader understands and is comfortable with the concepts of TBoxs, TPoints, TPointArrays, and Arrays of TPointArrays. If that sounds like gibberish to you, I suggest you stop here and read the Mayor's tutorial here (https://villavu.com/forum/showthread.php?t=107757). This guide shows an approach to finding objects that share similar colors with a lot of the surrounding area, thus making them more difficult to find. The guide will work based of an example. My hope is that you can learn from the this example and use the concepts elsewhere. I will show step by step the thought process I go through when determining how to find objects that are more difficult to find than usual.

NOTE: When I made the images of ACA, I didn't pick too many colors from other worlds, the images are there to make a point. Because of this, the colors in the code differ a bit from the ones shown in ACA as I went through later and gathered more colors.


Contents


Scenario 1

Goal: What are we trying to do in the first place? (Open a door)
Usual Methods: Lets try to achieve this goal with simple methods first before going overboard.
Finding a Relative Object: We need to establish some a base for us to work from
Filtering: Ensure we only find what we want to find
Determining: How can the data we have be used?
Finalizing: Finishing the job
Bonus: Things that stray from what this guide was intended for. But are required to perfect the goal




Goal


So lets start by figuring out exactly what we want to do. In this scenario, we want a procedure that determines if the door to the tanner is open or closed. Then, if the door was closed, it will open it. We also want this procedure to work at any camera angle. To really visualize this goal, I took a picture and poorly edited it with Microsoft Paint.

http://i.imgur.com/VAOOWh3.png

Usual Methods


Before we start making this complex, we should simply try to get a color for the door. Then we can check if its open or not that way, right? Lets consult ACA.

http://i.imgur.com/oWCWZzF.png

Ouch, there is a lot of similar colors in this area. Lets not lose hope yet, lets plug these colors in and try to tackle this using a findColorsSpiralTolerance call. Convert the returned TPA to an ATPA, debug that ATPA, and look at the results.

To do this, I'll run this code.

// Scenario 1
procedure HandleDoor()
var
x,y : Integer;
TPA : TPointArray;
ATPA : T2DPointArray;
begin
findColorsSpiralTolerance(x, y, TPA, 3230561, mainScreen.getBounds(), 6, colorSetting(2, 0.12, 0.91));
if (Length(TPA) < 1) then
begin
Exit;
end;

// The door is about that size
ATPA := TPA.toATPA(20, 40);
ATPA.sortFromMidPoint(Mainscreen.playerPoint); // Sort found colors

SmartImage.debugATPA(ATPA);

end;

Here is the debug

http://i.imgur.com/1XduOY4.png

That is a huge mess of things. However, the first TPointArray in the Array of TPointArrays makes up the door we want to find. But what if, for example, the procedure is called when the player is closer to one of the many other TPointArrays found, then we might have to go through (mouse) several TPointArrays before finding what we want. We could do some filtering and maybe better bounds for the door, but I think this method is a no-go. Continuing with this method to find the door could work in some scenarios, but we want a reliable method to go about this.


Finding a Relative Object


As we have seen in the images above, this area of Varrock has a ton of colors that similar to the door. To be accurate, we need a unique color. So we need to find an object with unique colors to base our search for the door off of. Because if we know where one thing is, we can determine where something else is relative to that thing.

So lets look at the area again, you will notice that there is a bowl in the tanner's building. It seems like its a pretty unique color to me... Lets check the bowl out in ACA

http://i.imgur.com/g11exai.png

That looks great! The bowl has unique colors so we now have a base to work from.

NOTE: There might be another object that has unique colors we could have used. But, I saw the bowl first and it worked so I stuck with it.

Alright, lets put those colors into code and debug the TPA we get.

procedure HandleDoor()
var
x,y : Integer;
TPA : TPointArray;
begin

findColorsSpiralTolerance(x, y, bTPA, 876686, mainScreen.getBounds(), 4, colorSetting(2, 0.16, 0.36));
if (Length(TPA) < 1) then
begin
WriteLn('Failed to find bowl colors');
Exit;
end;

SmartImage.debugTPA(TPA);

end;

And now the debug from this code

http://i.imgur.com/HlWZI3g.png

That looks very clean. We now have a base to work of off. Now we could just filter based of off this bowl and find the door, but that would only work if the camera was facing north or south. How would we know if the door is open or closed if we look at several different angles? Well to know, we are going to need another relative object. The frame above the door. Now you may be thinking. "Why did we just find the bowl as a relative object when we could have just used the door frame?" The answer is simple. The door frame does not have unique colors. Lets look at it in ACA.

http://i.imgur.com/qgWeGoa.png

That looks pretty good. The color of the door frame is fairly unique. There is a bit of it residing in the edging of the building and on some windows, but the vast majority of that color is on the door frame. We will filter out those smaller groups of colors and other door frames in the next section.

NOTE: This object is very annoying to get get colors on. The entire frame is one solid color. To gather multiple colors I had to swap worlds several times.


Filtering


So we have our relative objects (the door frame and bowl) how do we find the door with them? First, we need to look at what we can find with no filtering. Let's try making an Array of TPointArrays from the colors of the door frame. Then sort it from the location of the bowl. To this we will run this code.

// Scenario 1
function IsDoorOpen() : Boolean
var
x,y : Integer;
bTPA,frameTPA : TPointArray; //bTPA is for the bowl, frameTPA is for the frame
frameATPA : T2DPointArray;
BowlPoint : TPoint;
begin
findColorsSpiralTolerance(x, y, bTPA, 876686, mainScreen.getBounds(), 4, colorSetting(2, 0.16, 0.36));
if (Length(bTPA) < 1) then
begin
WriteLn('Failed to find bowl colors');
Exit;
end;

BowlPoint := middleTPA(bTPA); //Find the central point of the bowl

findColorsSpiralTolerance(x,y, frameTPA, 3953514, mainScreen.getBounds(), 2, colorSetting(2, 0.27, 0.44));
if (Length(frameTPA) < 1) then
begin
WriteLn('Failed to find frame colors');
Exit;
end;

frameATPA := frameTPA.toATPA(46,4); // 46 by 4 are about the dimensions of the frame.

frameATPA.sortFromMidPoint(BowlPoint); // sort the TPAs of frame colors around the bowlPoint

SmartImage.debugATPA(frameATPA);

SmartImage.debugTPA(bTPA);

end

And the debug

http://i.imgur.com/2vNfSJI.png

Well I'd say we are a bit closer to our goal, but we need to do more to accurately find the correct door frame, and in turn determine the state of the door and open it if needed. Lets filter out some colors. To do this, we need to use a few different procedures. I am going to go over each of these briefly.


TPointArray.excludePointsDist(const minD, maxD : Extended; const mX, mY : Integer);
This procedure will remove points from a TPA based on the distance from a certain point. The minD and maxD are the minimum and maximum distances this procedure will filter out points by. For example, if a point is 10 pixels from the center point, and the minD is 20 it will be filtered out. In the following code, minD will be 0.


FilterTPAsBetween(var atpa: T2DPointArray; const minLength, maxLength: integer);
This procedure will remove TPointArrays from an array of TPointArrays that have lengths that fall between the two parameters minLength and MaxLength. In the code, I will set minLength to 1 and maxLength to 110. If a TPointArray in the array of TPointArrays has a length that falls between those two numbers ( length < 110 ) this procedure will remove them from the array of TPointArrays.

Now lets use these procedures in the code. I'll comment what each line does to help make it clear what I'm doing.

NOTE: To get the parameters for these procedures, I had to do a fair bit of guessing and checking.

// Scenario 1
procedure HandleDoor()
var
x,y : Integer;
bTPA,frameTPA : TPointArray;
frameATPA : T2DPointArray;
BowlPoint : TPoint;
begin
findColorsSpiralTolerance(x, y, bTPA, 876686, mainScreen.getBounds(), 4, colorSetting(2, 0.16, 0.36));
if (Length(bTPA) < 1) then
begin
WriteLn('Failed to find bowl colors');
Exit;
end;

BowlPoint := middleTPA(bTPA);

findColorsSpiralTolerance(x,y, frameTPA, 3953514, mainScreen.getBounds(), 2, colorSetting(2, 0.27, 0.44));
if (Length(frameTPA) < 1) then
begin
WriteLn('Failed to find frame colors');
Exit;
end;

{This line filter out points in the frameTPA that are farther than 180 pixels from the bowl point}
frameTPA.excludePointsDist(0, 150 , BowlPoint.x, BowlPoint.y);

{Converts frameTPA to an array of TPointArrays. You may have noticed I changed the bounds, I did this so this will work at any angle}
frameATPA := frameTPA.ToATPA(46,46);

{This will filter out TPointArrays in the frameATPA that contain fewer points that 110}
FilterTPAsBetween(frameATPA,1,110);

{Sort our findings from the bowl Point}
frameATPA.sortFromMidPoint(BowlPoint);

SmartImage.debugATPA(frameATPA);

SmartImage.debugTPA(bTPA);

end;

Lets look at the debug for this procedure at a few different camera angles.

http://i.imgur.com/Pim7kgJ.png

Now I would call that a success. It finds the frame just beautifully even at different angles. Now we just need to detect if the door is open or not and open it if needed.


Determining


Alright, so we have filtered our ATPA to only have one TPA highlighting the correct door frame now we need to determine if the door is open or not. To start, lets look at the above image, the TPA of door frame points doesn't really give any details about the doors state, just its location.

So lets find a way to determine if the door is open or closed. If you look at the picture long enough, you will notice that the top of the door has a fairly unique color. Lets take a look at it in ACA.

http://i.imgur.com/ZrD0hcR.png


NOTE: Like the door frame, this object is very annoying to get get colors on. The entire thing is one solid color. To gather multiple colors I had to swap worlds several times.

Alright that looks pretty good, you may notice that the color of the top of the door is also present in a few other places, but that does not matter as we can limit where we look for that color. Lets now create a box to limit where we look.

You may notice that the center point (the X in the middle of the debug) of the TPA is always in the middle of the door frame. You may also notice that there is only one TPA in the ATPA. We will use these facts to help limit where we look for the top of the door color.

Alright, I think we're ready to code all of this in. I am going to first merge the ATPA of door frame colors in the one TPA as the filtering should limit it to one TPA (if it doesn't for whatever reason, the center point of the merged TPA should still rest on the door frame). Once we have this merged TPA of door frame points, we will find the center point of it, just like we did with the bowl earlier. We will create a TBox that is centered around that frame point.

That seems like a lot of steps, but its a lot easier done than said. Lets code all this in and look at the debug.

// Scenario 1
procedure HandleDoor()
var
x,y : Integer;
bTPA,frameTPA : TPointArray;
frameATPA : T2DPointArray;
BowlPoint, DoorPoint : TPoint;

DoorCheckBox : TBox;
begin
findColorsSpiralTolerance(x, y, bTPA, 876686, mainScreen.getBounds(), 4, colorSetting(2, 0.16, 0.36));
if (Length(bTPA) < 1) then
begin
WriteLn('Failed to find bowl colors');
Exit;
end;

BowlPoint := middleTPA(bTPA);

findColorsSpiralTolerance(x,y, frameTPA, 3953514, mainScreen.getBounds(), 2, colorSetting(2, 0.27, 0.44));
if (Length(frameTPA) < 1) then
begin
WriteLn('Failed to find frame colors');
Exit;
end;

frameTPA.excludePointsDist(0, 150 , BowlPoint.x, BowlPoint.y);

frameATPA := frameTPA.ToATPA(46,46);

FilterTPAsBetween(frameATPA,1,110);

// if there is only one TPA in the ATPA no reason to sort it
//frameATPA.sortFromMidPoint(BowlPoint);


frameTPA := frameATPA.merge(); // merges the frameATPA to a TPA and sets frameTPA equal to it

DoorPoint := middleTPA(frameTPA); // find the middle point of the frame

DoorCheckBox := intToBox(DoorPoint.x-25, //this will create a 50x50 box around the middle point of the frame
DoorPoint.y-25, //notice how I just add 25 to the point in each direction to make it
DoorPoint.x+25,
DoorPoint.y+25);


SmartImage.drawBox(DoorCheckBox,False,ClBlue); // debug that box

SmartImage.debugTPA(bTPA);

end;

And here is the debug of this procedure

http://i.imgur.com/H8cFIkH.png

That looks good. Within that TBox we notice the top of the door color is present.

Now from here, we need to detect if the door is open or not. To do this, we are going to count the amount of pixels that are the color of the top of the door in that TBox, if there are more than a certain amount of those colors (we will have to guess and check a bit) then we can conclude that the door is open.

To do this we are going to use this function


function countColorTolerance(color: integer; searchBox: TBox; tol: Integer; settings: TColorSettings): integer;
To use this, we will just plug in the color info from ACA and it will count how many pixels of that color it finds.

So lets implement these ideas to our code.

Procedure HandleDoor();
var
x,y : Integer;
bTPA,frameTPA : TPointArray;
frameATPA : T2DPointArray;
BowlPoint, DoorPoint : TPoint;

DoorCheckBox : TBox;
begin
findColorsSpiralTolerance(x, y, bTPA, 876686, mainScreen.getBounds(), 4, colorSetting(2, 0.16, 0.36));
if (Length(bTPA) < 1) then
begin
WriteLn('Failed to find bowl colors');
Exit;
end;

BowlPoint := middleTPA(bTPA);

findColorsSpiralTolerance(x,y, frameTPA, 3953514, mainScreen.getBounds(), 2, colorSetting(2, 0.27, 0.44));
if (Length(frameTPA) < 1) then
begin
WriteLn('Failed to find frame colors');
Exit;
end;

frameTPA.excludePointsDist(0, 150 , BowlPoint.x, BowlPoint.y);

frameATPA := frameTPA.ToATPA(46,46);

FilterTPAsBetween(frameATPA,1,110);

frameTPA := frameATPA.merge();

DoorPoint := middleTPA(frameTPA);

DoorCheckBox := intToBox(DoorPoint.x-25,
DoorPoint.y-25,
DoorPoint.x+25,
DoorPoint.y+25);


SmartImage.drawBox(DoorCheckBox,False,ClBlue);

//SmartImage.debugTPA(bTPA);

if countColorTolerance(4675428, DoorCheckBox, 2, colorSetting(2, 0.29, 0.44)) > 50 then
smartImage.drawClippedText('Open',DoorPoint, 'StatChars',true, clWhite)
else
smartImage.drawClippedText('Closed',DoorPoint, 'StatChars',true, clWhite)

end;

And now the Debug

http://i.imgur.com/DjR9kR3.png

That looks pretty damn good to me. Now we can start finishing this up by opening the door if needed.


Finalizing


Ok, we can find the door and detect if it is open or not. Now we just have to open it. Thankfully, we have already passed the hard part of this whole process, now we can just search for the door colors (that we got at the very beginning) in the TBox that surrounds the door and click them. So, lets throw that in the code.

// Scenario 1
Procedure HandleDoor();
var
x,y : Integer;
bTPA,frameTPA, dTPA : TPointArray; //dTPA = TPA for the door
frameATPA : T2DPointArray;
BowlPoint, DoorPoint : TPoint;

DoorCheckBox : TBox;
begin
findColorsSpiralTolerance(x, y, bTPA, 876686, mainScreen.getBounds(), 4, colorSetting(2, 0.16, 0.36));
if (Length(bTPA) < 1) then
begin
WriteLn('Failed to find bowl colors');
Exit;
end;

BowlPoint := middleTPA(bTPA);

findColorsSpiralTolerance(x,y, frameTPA, 3953514, mainScreen.getBounds(), 2, colorSetting(2, 0.27, 0.44));
if (Length(frameTPA) < 1) then
begin
WriteLn('Failed to find frame colors');
Exit;
end;

frameTPA.excludePointsDist(0, 150 , BowlPoint.x, BowlPoint.y);

frameATPA := frameTPA.ToATPA(46,46);

FilterTPAsBetween(frameATPA,1,110);

frameTPA := frameATPA.merge();

DoorPoint := middleTPA(frameTPA);

DoorCheckBox := intToBox(DoorPoint.x-25,
DoorPoint.y-25,
DoorPoint.x+25,
DoorPoint.y+25);


SmartImage.drawBox(DoorCheckBox,False,ClBlue);

if countColorTolerance(4675428, DoorCheckBox, 2, colorSetting(2, 0.29, 0.44)) > 50 then
begin
smartImage.drawClippedText('Open',DoorPoint, 'StatChars',true, clWhite);
end else
begin
smartImage.drawClippedText('Closed',DoorPoint, 'StatChars',true, clWhite);

//Search DoorcheckBox for door colors, if the door is not found write a debug message
if findColorsSpiralTolerance(x,y, dTPA, 3230561, DoorCheckBox, 6, colorSetting(2, 0.12, 0.91)) then
begin
mouse(middleTPA(dTPA),MOUSE_MOVE);


if isMouseOverText(['Open','pen']) then
begin
fastClick(MOUSE_LEFT);
end;

end else
begin
writeLn('Failed to find door colors');
end;
end;
end;

Alright, so this should open the door in most cases. If you only called this procedure when the camera was facing north, this would work as is. You may notice that my if-else statements are a little strange as they differ a bit from the structure I have been using but there is reason for this. I am going to finish this procedure in the bonus section and go over some of its flaws. I encourage everyone to keep reading, but I will not be introducing any new ideas relating to this tutorial's topic. I will just be doing more finalizing and a bit of overall reviewing.


Bonus


Ok, so now we need to consider a few things. First, our original goal was for this procedure to work at any camera angle. If put your camera to facing the east or west, the door colors would not be visible on the screen. To combat this, we will need to rotate the camera if no colors are found. So all we have to do is rotate the camera if the door colors are not found or if we mouse the door TPointArray and we do not find the correct mouseover text. After we rotate, we will need to relocate the door as well, so we will be using a repeat-until loop. Lets add these ideas into the code.

// Scenario 1
Procedure HandleDoor();
var
x,y : Integer;
bTPA,frameTPA, dTPA : TPointArray;
frameATPA : T2DPointArray;
BowlPoint, DoorPoint : TPoint;

DoorCheckBox : TBox;

RotateCameraTrys : Integer; // Tracks how many camera rotations, used to avoid having a potentially infinite loop
begin
RotateCameraTrys := 0;
repeat
findColorsSpiralTolerance(x, y, bTPA, 876686, mainScreen.getBounds(), 4, colorSetting(2, 0.16, 0.36));
if (Length(bTPA) < 1) then
begin
WriteLn('Failed to find bowl colors');
Exit;
end;

BowlPoint := middleTPA(bTPA);

findColorsSpiralTolerance(x,y, frameTPA, 3953514, mainScreen.getBounds(), 2, colorSetting(2, 0.27, 0.44));
if (Length(frameTPA) < 1) then
begin
WriteLn('Failed to find frame colors');
Exit;
end;

frameTPA.excludePointsDist(0, 150 , BowlPoint.x, BowlPoint.y);

frameATPA := frameTPA.ToATPA(46,46);

FilterTPAsBetween(frameATPA,1,110);

frameTPA := frameATPA.merge();

DoorPoint := middleTPA(frameTPA);

DoorCheckBox := intToBox(DoorPoint.x-25,
DoorPoint.y-25,
DoorPoint.x+25,
DoorPoint.y+25);


SmartImage.drawBox(DoorCheckBox,False,ClBlue);

if countColorTolerance(4675428, DoorCheckBox, 2, colorSetting(2, 0.29, 0.44)) > 50 then
begin
smartImage.drawClippedText('Open',DoorPoint, 'StatChars',true, clWhite);
Exit; // exit the procedure if the door is already Open
end else
begin
smartImage.drawClippedText('Closed',DoorPoint, 'StatChars',true, clWhite);

if findColorsSpiralTolerance(x,y, dTPA, 3230561, DoorCheckBox, 6, colorSetting(2, 0.12, 0.91)) then
begin
mouse(middleTPA(dTPA),MOUSE_MOVE);

if isMouseOverText(['Open','pen']) then
begin
fastClick(MOUSE_LEFT);
Exit; // exit if the door is successfully found and clicked
end else
begin // mouseOver not found
randomCompass(0, 360, false);
smartImage.clear(); // clear debug so it doesn't get cluttered
end;

end else // no door colors found at all
begin
randomCompass(0, 360, false);
smartImage.clear();
end;
end;
RotateCameraTrys := RotateCameraTrys + 1;
until RotateCameraTrys > 3;
end;

Alright, that lets look at this in action.

http://i.imgur.com/HDVRlxc.gif

That looks great! I intended this gif to be bigger but I don't really know how to resize it. I'll try again someday to make this a bit better. For now just squint and try to see what its doing.

Potential Problems

I do want to mention some potential problems with this procedure.

First, this procedure assumes that the player is roughly in the right area (near the front of the tanner). This isn't really much of a problem, this procedure just requires the script to set it up right.

Second, if for whatever reason the door point is near the edge of smart, the tbox will go out of bounds. IF we try to debug a tbox out of bounds an execution error will follow causing the script to fail. It would not be very hard to prevent this, but I didn't as it strays from what this tutorial is about.

Third, if there is a another player spam opening and closing the door, I do not know how this procedure will react. This isn't really a problem either as the script could handle this as well.

I figured it would be important to bring up these potential issues just so one knows.




Conclusion


Well we have learned that we can locate difficult to find objects by using relative ones. Hopefully you this guide gave you some ideas and maybe apply these concepts elsewhere. Or at the very least puts you one method closer to a Varrock tanner. If you want to play with the code, I have attached it below. Well that is it, thank you for reading my guide.

This is my first guide, so if you feel I made jumps in my reasoning that were too big, let me know! I'll try to fill in the gaps I may have left out. Also if you notice any grammar/spelling errors please point those out. Thanks.

Frement
10-09-2014, 12:22 AM
Very nice and detailed tutorial, excellent work! I have awarded you some reputation cookies.

Ross
10-09-2014, 12:22 AM
Great job man, seriously! +repped

The Spark
10-09-2014, 12:24 AM
Really nice job, +rep

Kevin
10-09-2014, 12:25 AM
Wow, much respect. This is an amazing tutorial that I highly recommend people read. +Rep for sure.

Ian
10-09-2014, 02:01 AM
This is a very good tutorial, you explained everything well. I have a feeling this will be one of the tutorials I'll link people to when they have questions about finding things with more than the basic atpas :)

Great job!

The Mayor
10-09-2014, 05:30 AM
Nice tutorial :D it's very similar to an idea i've used before (https://villavu.com/forum/showthread.php?t=109637&p=1301518#post1301518).

Incurable
10-09-2014, 05:42 AM
Excellent tutorial, I learned quite a bit... +rep to you mine good Sir!

Hoodz
10-09-2014, 07:37 AM
gj, rep+

sipfer3
10-09-2014, 05:08 PM
wow. thats pretty cool idea. deffinetly im going to try this out! its perfect to solve some problems im suffering from. thanks for the guide!

Home
10-09-2014, 05:37 PM
Well done :) Rep++

~Home

Spaceblow
10-09-2014, 06:14 PM
Amazing work, I'll definitely check this out because I'm still kind of struggling with finding objects. Thank you for this! :)

Torol
10-27-2014, 03:34 PM
With this guide I was able to find a door. Thanks!

Swatarianess
10-31-2014, 09:25 AM
I will never look at a door the same way again in runescape.. Nice guide and thorough :)

keff
11-06-2014, 05:00 PM
I will never look at a door the same way again in runescape.. Nice guide and thorough :)

Agree fully with †his comment :) Great tut

Jayden C
02-01-2015, 03:50 PM
I read this guide at the perfect time. I hit a stop in my script earlier and I have been wondering how to use TPA and ATPA's this way. When you don't know something it seems almost impossible until you read a guide as good as this one. This will improve my scripting for sure, Thanks Camel. +rep

apple98
02-01-2015, 09:06 PM
wahh sweet rep++

jackieo500
02-12-2015, 11:53 AM
This is actually very smart, I will definitely be using this in my scripts! Thanks

dodo
06-22-2015, 01:17 AM
Very Useful! my cooking script was getting stuck every time someone closed a door -.-.

Shield
09-30-2015, 04:24 PM
Nice tutorial. This has come in handy!