PDA

View Full Version : NXT Dynamic Interfaces



Citrus
03-15-2018, 08:34 AM
These functions will find various interfaces within the resize-able NXT client. I will add more if they are requested and I have the time.

Backpack

Requirements: The entire backpack is uncovered, all 28 slots are visible (i.e. no scrollbar), total width less than 1135.

function FindBackpack(out Bounds: TBox): Boolean;
Finds the backpack by searching for bitmaps, outputs the bounds as a TBox, and returns a Boolean.
Takes ~20 ms.

function LoadSlotBoxes(const Bounds: TBox; const FileName: String): TBoxArray;
Given the previously found bounds, returns a TBoxArray of the 28 slots by loading a preset from a text file, found here: 28393
Note: these boxes are the correct visual bounds. If you want the correct clickable bounds, then shrink by 3 pixels.
Takes ~2 ms.

(you can ignore the Grid functions if using SRL-6)


//https://github.com/SRL/SRL-6/blob/d5817b11a391940d12180740a914a4cf5d72b204/lib/utilities/math.simba#L59
function GridBox(Slot, Columns, Rows, w, h, DiffX, DiffY: Int32; StartPoint: TPoint): TBox;
begin
Result.x1 := (StartPoint.x + ((Slot mod Columns) * DiffX) - (w div 2));
Result.y1 := (StartPoint.y + ((Slot div Columns) * DiffY) - (h div 2));
Result.x2 := (Result.x1 + w);
Result.y2 := (Result.y1 + h);
end;

function Grid(Columns, Rows, w, h, DiffX, DiffY: Int32; StarTPoint: TPoint): TBoxArray;
var
i: integer;
begin
SetLength(Result, (Columns * Rows));

for i := 0 to High(Result) do
Result[i] := GridBox(i, Columns, Rows, w, h, DiffX, DiffY, StarTPoint);
end;

function FindBackpack(out Bounds: TBox): Boolean;
const
LINE_COLOR = 6247745;
HEADER_COLOR = 332333;
HEADER_TOL = 9;
var
i, j, x, y, w, h, CoinBMP, PackBMP, DownBMP, CloseBMP: Int32;
TPA, TPA2: TPointArray;
ATPA, ATPA2: T2DPointArray;
begin
GetClientDimensions(w, h);
CoinBMP := BitmapFromString(9, 9, 'meJzrufGz58bPSbd/A1HNvgcO5fOU' +
'0jvNeteaTdjSgyQ14+6/RY//S0TUaVcvBqLOq1+ACCIFZEy9/lEqr' +
'gMortu8zqF3mdOElUZNqx2m7M/bead48yUOs3TxoEagiFHVdKPKOc' +
'oFc4AqoxYc7T7/mtMyhc8pVzF7pm3fartJ64FqLHt3AqWAulgNooC' +
'ycskTgVqMaxZAdAENrNx5Ay6lVz0PKAg3sOHgPbiBQPdbTtkG dA/Q' +
'QIgzILokI9sAFPp7uw==');

PackBMP := BitmapFromString(10, 10, 'meJyTt26Vt261cGoBIgkVcyACcl' +
'VssyEImQ1EQFlRdV+gIJAEclkFlZDVAAUh4rxigUCGmIg0Ews vRAQ' +
'iBRQBsoNrVwDNgWhh5RcGGgLkArUAZeUyuor33oiacwziDIhF QDbQ' +
'HCBpWLpYVLXQOmQlxBnILgRygXpB9qoWQmThjgEioHa1hCKT7 pUQq' +
'5EVgJBqIVANUNbn0E02FVW4H4EBArQLiIDGAh0JFAQAJ5hHLw ==');

DownBMP := BitmapFromString(10, 8, 'meJzLuP2r49ZnBgyQvuKaUc9Z51N' +
'fM6/+rL35qweMWs99Kdn1OHbtXbupF9kKDljvewNU4H/6R+QJEArc' +
'9cF8znXlnnNcVUdYs3caLz1vtvkxUI3p7m9ApLbitVDneaAUR/YOj' +
'piFQCu0Z5zWX/nQeMtbo+1fgbKCjSd4inezxy2Fu0Fr+imgAu1Vr5' +
'Vn3UOTggDl7iNAKanuCzzJSzC9AAQKjXulyrcjiwAAiq5bTg= =');

CloseBMP := BitmapFromString(10, 10, 'meJzj4PFh5U6GIDauLAiCcDnAUm' +
'y8NRyCzZgIqoW3Ju3YF5Os3eyCnUAEZAC5EFkWzmigrHbKzoy L/4D' +
'iQARhsPM1wWUhCkKP/QEiIAMoBRSBynKXANlyMdvsd/4GIiADoh4o' +
'xcIWBHSkaOB6jdW/xEI2AxGEAdQClxWe/J7HdwPIEO4SIAPIBQoCp' +
'VhYfBGeZYf6F8hgZU8HSnFymANJBAKrhyCgFAAg/Uys');

try
if FindColors(TPA, LINE_COLOR, 0, 0, w-1, h-1) then
begin
ATPA := ClusterTPA(TPA, 1);
for i := 0 to High(ATPA) do
begin
Bounds := GetTPABounds(ATPA[i]);
if (Bounds.Y2 = Bounds.Y1) then
begin
Bounds.Y2 := Min(h-1, Bounds.Y2 + 35);
if FindBitmapToleranceIn(CoinBMP, x, y, Bounds.X1, Bounds.Y1, Bounds.X2, Bounds.Y2, 0) then
begin
Bounds.X1 := Max(0, Bounds.X1 - 4);
Bounds.X2 := Min(w-1, Bounds.X2 + 4);

if FindBitmapIn(PackBMP, x, y, 0, 0, Bounds.X2, Bounds.Y2) then
begin
Bounds.Y1 := y + 15;
end else
if FindBitmapIn(CloseBMP, x, y, Bounds.X2 - 15, 0, Bounds.X2 + 5, Bounds.Y2 - 85) then
begin
Bounds.Y1 := y + 11;
end else
begin
WriteLn('Backpack error: didn''t find Red X or Backpack bitmap.');
Exit;
end;

if FindBitmapToleranceIn(DownBMP, x, y, Bounds.X1, Bounds.Y1, Bounds.X2, Bounds.Y2, 5) then
begin
WriteLn('Backpack error: not all slots visible.');
Exit;
end;

Exit(True);
end else
begin
WriteLn('Backpack error: didn''t find Coin bitmap.');
end;
end;
end;
end;
finally
FreeBitmap(CoinBMP);
FreeBitmap(PackBMP);
FreeBitmap(DownBMP);
FreeBitmap(CloseBMP);
end;
end;

function LoadSlotBoxes(const Bounds: TBox; const FileName: String): TBoxArray;
var
i, Width, Cols, Rows, SpaceX: Int32;
GridPoint: TPoint;
SS: TStringArray;
List: TStringList;
begin
Width := Bounds.X2 - Bounds.X1 + 1;
Cols := 2 + (Width - 94) div 40;
Rows := Ceil(28 / Cols);

if (not FileExists(FileName)) then
begin
WriteLn('Backpack slots error: didn''t find Slots.txt');
Exit;
end;
List.Init();
try
List.LoadFromFile(FileName);
for i := 0 to (List.GetCount() - 1) do
begin
SS := Explode(#9, List.GetStrings(i));
if (Length(SS) <> 3) then
begin
WriteLn('Backpack slots error reading Slots.txt');
Exit;
end;
if SameText(IntToStr(Width), SS[0]) then
begin
GridPoint.X := Bounds.X1 + StrToInt(SS[1]) + 19;
SpaceX := StrToInt(SS[2]);
Break;
end;
end;
finally
List.Free();
end;
if (SpaceX < 1) then
begin
WriteLn('Backpack slots error: didn''t Width in find Slots.txt');
Exit;
end;

GridPoint.Y := Bounds.Y1 + 6 + 16;
Result := Grid(Cols, Rows, 37, 33, SpaceX, 36, GridPoint);
SetLength(Result, 28);
end;

Results (the exact bounds and slots found by the functions):
https://i.imgur.com/HMW4gPg.png https://i.imgur.com/q0vVzXk.png https://i.imgur.com/Fp0pRWG.png https://i.imgur.com/KwXHrlU.png https://i.imgur.com/6Tnublh.png
https://i.imgur.com/fetywOg.png


Minimap

Requirements: Don't cover the corners of the minimap, other than that it should work anywhere at any size.

function FindMinimap(out Bounds: TBox): Boolean;
Finds the minimap by searching for bitmaps, outputs the bounds as a TBox, and returns a Boolean.
Takes ~20 ms.

function FindMinimap(out Bounds: TBox): Boolean;
var
x, y, w, h, TP_BMP, LG_BMP: Int32;
begin
GetClientDimensions(w, h);

TP_BMP := BitmapFromString(9, 9, 'meJw7u2JapmXM2ekzPt3dD0RHdq4E' +
'siEIKP5kx7yvl7YA2e2eBWX2qXAEFH9zbNX8kkygGjQEVD8rL j3e1' +
'B8ZQaT2T6yGcP21XH9//wpEyAog4nApCBuI4OJwKSclKwiCiwPZEC' +
'kraUNMqa+PLiMjAPyxc2A=');

LG_BMP := BitmapFromString(10, 10, 'meJzza1uWtflq8c6baAgo6Ne2zG' +
'HuiczLP5rv/EJDQEGglGLvAesDr0QiJyEjk6kH3fb+UJl4WLhls+z' +
'aGxAkPvs4T8ZsIIPdu1Zlx1OgFFfdWsG1t4HiXGHTGOwLgAjI hZBA' +
'KYHCpWKzLwOVAdUzmMQCEZALIYFSsilL9LvPW02/BkSqVdt47fKAX' +
'AYtbyAJlJKPXqZYd4JBwRIZAc0BCgJlgS6UKNollr1aMHMpnA SKAB' +
'FQCmgU0D1YEVAKAJUng5Y=');

try
if FindBitmap(TP_BMP, x, y) then
begin
Bounds.X1 := Max(0, x - 16);
Bounds.Y2 := Min(h-1, y + 34);
if FindBitmap(LG_BMP, x, y) then
begin
Bounds.X2 := Min(w-1, x + 10);
Bounds.Y1 := Max(0, y + 11);
Result := True;
end;
end;
finally
FreeBitmap(TP_BMP);
FreeBitmap(LG_BMP);
end;
end;
Results (the exact bounds found by the function):
https://i.imgur.com/c2JM17N.png https://i.imgur.com/7pH8EoY.png https://i.imgur.com/YsjOnYy.png
https://i.imgur.com/KRYY81y.png


Chatbox

Requirements: have "All Chat" as your first tab, don't hide the corners of the chatbox

function FindChatbox(out Bounds: TBox): Boolean;
Finds the chatbox by searching for bitmaps, outputs the bounds as a TBox, and returns a Boolean.
Takes ~60 ms.

function FindChatbox(out Bounds: TBox): Boolean;
var
x, y, w, h, ChatBMP, DownBMP, GrayBMP, YellowBMP: Int32;
begin
GetClientDimensions(w, h);

GrayBMP := BitmapFromString(8, 8, 'meJxjYICClt4JC5avgiOIYEVT6+S5' +
'CyCod9pMIAIKZpWWAxlA9cgoKjkNqBgT2bl6hMTEYyKgOUAp3 9BIT' +
'HGIFIQLNBaCYM5ElwIAmjFFZg==');

YellowBMP := BitmapFromString(8, 8, 'meJxjYIAC46XH9fa9gCOIoPySvRCu' +
'/NbHEAQUlJ68AcIWWXcPjtjrVkIYHKvuIyPZxBmaHRcwEdAcoJR 89' +
'WGl+jNo4nApiCwEwZzJIBQ7VaZoL1wKAHR0Tuo=');

ChatBMP := BitmapFromString(9, 9, 'meJz7c6Xzz5XO7oZ4ICoqjrEpj1Wb' +
'WgNE0sle17cEAqXsJuewxhUBxYEMiBSbmz5QCqgeKAIUZ2gIV ayKA' +
'up1cvdk0TS7lK1kVGeeGWsyq0gcSALNUUsoAiJmG3cgF0jyhC faO5' +
'hCZIHauZoXsjiHARHQFubCLiACMoC6gGqAKoEGcjllAEXkMrq AKiE' +
'K5Gz8pRSEmdJcgbJAjUBlECuADKD5jGsrgSjZgN9QXwdoCFCx mJup' +
'jJIa0NlAcaBrAbHUTng=');

DownBMP := BitmapFromString(10, 8, 'meJzLuP2r49ZnBgyQvuKaUc9Z51N' +
'fM6/+rL35qweMWs99Kdn1OHbtXbupF9kKDljvewNU4H/6R+QJEArc' +
'9cF8znXlnnNcVUdYs3caLz1vtvkxUI3p7m9ApLbitVDneaAUR/YOj' +
'piFQCu0Z5zWX/nQeMtbo+1fgbKCjSd4inezxy2Fu0Fr+imgAu1Vr5' +
'Vn3UOTggDl7iNAKanuCzzJSzC9AAQKjXulyrcjiwAAiq5bTg= =');

try
if (FindBitmap(GrayBMP, x, y) or FindBitmap(YellowBMP, x, y)) then
begin
Bounds.X1 := Max(0, x - 6);
Bounds.Y1 := y + 14;
end else
if FindBitmap(ChatBMP, x, y) then
begin
Bounds.X1 := Max(0, x - 20);
Bounds.Y1 := y + 15;
end else
begin
WriteLn('Chatbox error: didn''t find All Chat or Left Arrow');
Exit;
end;
if FindBitmapToleranceIn(DownBMP, x, y, Bounds.X1, Bounds.Y1, w-1, h-1, 9) then
begin
Bounds.X2 := Min(w-1, x + 16);
Bounds.Y2 := Min(h-1, y + 35);
Result := True;
end else
begin
WriteLn('Chatbox error: didn''t find Down Arrow');
Exit;
end;
finally
FreeBitmap(GrayBMP);
FreeBitmap(YellowBMP);
FreeBitmap(ChatBMP);
FreeBitmap(DownBMP);
end;
end;

Results (the exact bounds found by the function):
https://i.imgur.com/8dJS4T3.png https://i.imgur.com/U0DOY1v.png https://i.imgur.com/f9gu6n4.png
https://i.imgur.com/1S7gLXB.png

Citrus
03-15-2018, 08:44 AM
reserved

Lucidity
03-15-2018, 09:47 AM
Nice work!

klamor
03-16-2018, 03:26 AM
This is some great stuff Citrus! How long have you been working with NXT?

Citrus
03-16-2018, 09:37 AM
This is some great stuff Citrus! How long have you been working with NXT?

Just the few hours that it took to make these.

Bixby Sayz
04-18-2018, 02:54 AM
Pretty impressive. Backpack does fail under a couple conditions. Namely if backpack is the only tab (so tab header is not visible) and interfaces are locked (hides close button). But how often are you going to run into that situation actually.

klamor
07-30-2019, 09:38 PM
Citrus have you made any further developments with dynamic interfaces?

Citrus
07-31-2019, 01:01 AM
Citrus have you made any further developments with dynamic interfaces?

Nope. Haven't given it a single thought since 03-15-2018 at 08:13 AM.
All of these functions are super easy to make, if a bit tedious. It's just basic bitmaps and static offsets.

klamor
07-31-2019, 02:06 AM
Nope. Haven't given it a single thought since 03-15-2018 at 08:13 AM.
All of these functions are super easy to make, if a bit tedious. It's just basic bitmaps and static offsets.

I was speaking in terms of getting these functions put into an official SRL release, but I guess I neglected to mention that.
Anyways, are you doing any botting in RS3? I'm getting back into things and trying to get a feel for who's working on what.

Citrus
07-31-2019, 03:22 AM
I was speaking in terms of getting these functions put into an official SRL release, but I guess I neglected to mention that.
Anyways, are you doing any botting in RS3? I'm getting back into things and trying to get a feel for who's working on what.

Nah, not really botting or gaming anymore. I've never had anything to do with the official includes, but anyone is free to use this stuff anywhere they want.

klamor
07-31-2019, 08:09 PM
Nah, not really botting or gaming anymore. I've never had anything to do with the official includes, but anyone is free to use this stuff anywhere they want.

Alright, I did have to change the CoinsBMP, but after that everything with backpack seems to work wonderfully. I'm in the process of merging your code into SRL1.2/tabBackpack now.

For anyone interested in downloading a copy of SRL-6 that includes these functions, they will be available on github soon. I will see about having these merged into the official SRL-6 once I get everything working nicely.

nkd2009
04-18-2020, 07:05 PM
@klamor did you ever get this integrated?

Thomas
04-21-2020, 01:12 PM
@klamor did you ever get this integrated?

I did, it is public on my github and in our members section of discord.
Just not ready for a proper release.

nkd2009
04-30-2020, 07:27 PM
@thomas nice sweet i found it. Struggling to get SPS to work (duplicate declaration error) and backslots is giving error didnt find the slots.txt file but the file is in the srl6/lib/interfaces folder...still trying.

Thomas
05-01-2020, 11:45 AM
@thomas nice sweet i found it. Struggling to get SPS to work (duplicate declaration error) and backslots is giving error didnt find the slots.txt file but the file is in the srl6/lib/interfaces folder...still trying.

PM me on discord