PDA

View Full Version : OCR-engine for OSR (Uptext).



slacky
12-03-2013, 04:28 AM
Deprecated use https://villavu.com/forum/showthread.php?t=111156

Someone told me that the current OCR-engine for UpText is OSR was somewhat unreliable, so I figured I could make a little something. So I made a plugin wrapping up my RS OCR-engine, hoping that somebody would find use for it. It should be noted that this is for RS shadow fonts only.

New build: Support for PS!

I did not do many tests where I compared it to the current GetUpText found in SRL-OSR, but a few. Performance varies a bit (could still use some tweaking), but should generally be more robust the what is now. Here are some BAD CASE SCENARIO examples of performance:



[NEW] Talk-to Duke Horacio / 7 more options
[OLD] TNlk-t# - @uke Horaci

[NEW] Walk here / 1 more options
[OLD] Walk here r '

[NEW] Walk here chaos combos (level-63) / 3 more options .
[OLD] Walk here c

[NEW] Open Large door / 2 more options
[OLD] @' 0@-'-r Larg e door

[NEW] Attack Man (le..el-2) / 4 more options
[OLD] @BE[ llan

[NEW] Walk here lady vx (level-54) / 3 more options
[OLD] Walk here l

[NEW] Adjust Screen Brightness
[OLD] Ad'lust Scr

[NEW] Talk-to Osman / 2 more options
[OLD] Talk-t@ Dsman7r

[NEW] Walk here A A R 0 N (level-95) / 3 more options '
[OLD] Walk here F

[NEW] Talk-to Al the Camel / 2 more options
[OLD] Talk-to Al lh' i'l

Quite decent, but not perfect. Usually it does read the text quite flawlessly.

It's speed is very optimal, running in just a few ms usually around 2-10ms.

The functions which are exposed to the user:


//Import / Load the uptext font (must have shadows).
procedure ocr_LoadFont(FontPath:String; var Font:TCharsN);

//Release the fonts, should be called at end of script.
procedure ocr_FreeFont(var Font:TCharsN);

//Important, must be used before everytime you try to read the uptext (see example)
procedure ocr_SetClient(ATIA:T2DIntegerArray);

//Makes it simple.. Everything is hardcoded in the engine
function ocr_ReadText(Font:TCharsN): AnsiString;

//Read text in client, allows to modify spacers.
function ocr_ReadTextEx(Font:TCharsN, SpaceMod,MaxSpace:Integer): AnsiString;

//Reads the text in the client and directly stops if "EndAt"-string is matched, it makes it a bit faster, and removes some "end noise".
function ocr_ReadTextEnd(Font:TCharsN, SpaceMod,MaxSpace:Integer; EndAt:String): AnsiString; overload;

//Simple regular IsUpText
function ocr_IsText(Text:String; Chars:TCharsN): Boolean;

//Same/comparable to IsUpTextMulti
function ocr_IsTextMulti(TextArr:TStringArray; Chars:TCharsN): Boolean;

//IsTextEx matches the string by also allowing you to use the "*"-sign to make it "skip".
//IE: TextPtrn := 'Attack * (level*)'; which would result True if a text like: 'Attack Giant Rat (level-19)' was found.
function ocr_IsTextEx(TextPtrn:String; Sensitive:Boolean; Chars:TCharsN): Boolean;



Example usage:

program new;
{$loadlib rstext.dll}

var UpFont:TCharsN;

function ClientATIA(x1,y1,width,height: Integer): T2DIntegerArray;
var bmp:Integer;
begin
bmp := BitmapFromClient(x1,y1,x1+width,y1+height);
Result := GetBitmapAreaColors(bmp, 0,0,width,height);
FreeBitmap(bmp);
end;

function RS_GetUpText2: String;
begin
ocr_SetClient(ClientATIA(7,7,500,16));
Result := ocr_ReadTextEnd(UpFont,0,200,'more options');
//Result := ocr_ReadText(UpFont);
end;

function RS_IsUpText(Text:String): Boolean;
begin
ocr_SetClient(ClientATIA(7,7,500,16));
Result := ocr_IsTextEx(Text, True, UpFont);
end;

var
t,i: Integer;
text: String;
begin
ocr_LoadFont(AppPath + 'Fonts\UpChars07_s\', UpFont);
WriteLn(RS_GetUpText2);
ocr_FreeFont(UpFont);
end.


If there should be any problems, please give me some feedback with relative info (EG: Where you whare when it happened).

A thanks to Janilabo for making am string functions used internally, and Olly for doing some extra testing! :)

Kyle
12-03-2013, 05:20 AM
Looks awesome man! I don't have a use for it in my current scripts i'm working on, but i'm sure others will!

Olly
12-03-2013, 05:34 AM
This is very good! a massive... huge.. improvement!

caused
12-03-2013, 07:06 AM
Awesome :). I posted in the bugtracker about the current OCR Functions.
The problem is that they can't read letters where the pixels are connected.

Please make support for Pascal script too. Lape compiles somewhat slow.

Flight
12-03-2013, 07:55 AM
Very good indeed. I'm going to plug this into AL.

Hoodz
12-03-2013, 08:08 AM
gj! to bad it wont work with pascalscript

Dgby714
12-03-2013, 09:43 AM
Why wont it work in PascalScript? Just wondering...

Kevin
12-03-2013, 06:11 PM
Very nice! I almost support not including PascalScript on purpose just to encourage upgrading...

slacky
12-03-2013, 06:58 PM
Why wont it work in PascalScript? Just wondering...
Got in to problems when I tried to use the exported font-record in PS. It instantly crashed Simba when I tried to load a font, I did not look long for a workaround to this issue, as I didn't really see it as that big a problem, and as for exporting and all, there should not be any problems, so don't see that I can do much about it.

Dgby714
12-03-2013, 09:00 PM
Got in to problems when I tried to use the exported font-record in PS. It instantly crashed Simba when I tried to load a font, I did not look long for a workaround to this issue, as I didn't really see it as that big a problem, and as for exporting and all, there should not be any problems, so don't see that I can do much about it.

What does the record look like?

slacky
12-03-2013, 09:03 PM
What does the record look like?

type
TCharN = record
Pts,Shadows: TPointArray;
Height,Width: Integer;
end;
TCharsN = array of TCharN;

I've gone over the code qute a few times, there is no reason for it to not work in PS. Yet it doesn't. So I don't believe the mistake is on my end.

Export file: http://pastebin.com/raw.php?i=PtLJH6SR

Dgby714
12-03-2013, 09:06 PM
type
TCharN = record
Pts,Shadows: TPointArray;
Height,Width: Integer;
end;
TCharsN = array of TCharN;

I've gone over the code qute a few times, there is no reason for it to not work in PS. Yet it doesn't. So I don't believe the mistake is on my end.

Is the source of the plugin public?

rj
12-03-2013, 09:08 PM
Nice work! OSR has been using that gettext function for a whole now hasn't it?

This may be a little off topic but can I view a source somewhere? :d didn't see it on your github

slacky
12-03-2013, 09:13 PM
Nice work! OSR has been using that gettext function for a whole now hasn't it?

This may be a little off topic but can I view a source somewhere? :d didn't see it on your github
No it's not available anywhere. I do not see that it's worth uploading anywhere, it's nothing out of the ordinary.
It's still quite "dirty" as it uses my SCARExt to fulfill it's needs, might look at upping it somewhere if I get it cleaned out a bit.

slacky
12-04-2013, 12:30 AM
Here is the requested source (exports are without overload as in the previous upped files):
http://slacky.info/browser/?fold=files/RS/RSText#files/RS/RSText

Updated with latest compiled version (old one was removed from speedy.sh for some reason).

Janilabo
12-04-2013, 12:33 AM
Here is the requested source (exports are without overload as in the previous upped files):
http://www.speedyshare.com/aP5XG/RSText.rar
Brilliant mate.
Glad to see you decided to share the source for this awesome project - appreciated move! :)

Flight
12-04-2013, 03:27 PM
Well I've fit this into my Lape SRL-OSR and it's working exceptionally well. The accuracy is improved and the speed is so much faster than before. You did very well Mr. WarPie. :thumbsup:

Should I come across any issues I'll be sure to report them here.

Zyt3x
12-04-2013, 05:13 PM
Very, very nice!
Great job :)

slacky
12-05-2013, 07:02 AM
New build! Supports PS, font-loading is now a procedure loadfont(fontpath:string, var Font:TCharsN); .
Also since it supports PS, overloads are also removed.

Se first post to grab the latest build. As always, if any errors show up, please post it here.

The Mayor
12-05-2013, 09:54 PM
Great work here warpie! Keep it up

Gavril
12-06-2013, 06:25 AM
Would be nice to support system fonts. If size must be specified, double would be nicer than integer (so you could say 10.5 for ex).
Would also be nice to be able to specify where it should look for text to OCR. That way it can be used for more than just RS.

Flight
12-06-2013, 07:03 AM
Would be nice to support system fonts. If size must be specified, double would be nicer than integer (so you could say 10.5 for ex).
Would also be nice to be able to specify where it should look for text to OCR. That way it can be used for more than just RS.

You can look for text in a specific region, look at his example functions; they're default set for the UpText area for RS, you can change these coordinates if you want.

Hoodz
12-06-2013, 10:22 AM
New build! Supports PS, font-loading is now a procedure loadfont(fontpath:string, var Font:TCharsN); .
Also since it supports PS, overloads are also removed.

Se first post to grab the latest build. As always, if any errors show up, please post it here.

thank you so much!


edit: is this going to be posted on the srl-osr github?

slacky
12-06-2013, 06:16 PM
Would be nice to support system fonts. If size must be specified, double would be nicer than integer (so you could say 10.5 for ex).
Would also be nice to be able to specify where it should look for text to OCR. That way it can be used for more than just RS.
I believe that can be done with the current OCR-engine in Simba, this one is made simply to better the reading of RS UpText (might be able to read other shadowfonts), as the color change, and background change a lot.

Olly
12-17-2013, 01:58 AM
I got Wizzup to add upchars07_s to the Simba fonts package. :)

Wizzup?
12-21-2013, 12:22 AM
So how does this work in mines where there is no shadow available?

PS: Didn't know the OCR for OSR was broken that much? It used to work quite well, I believe... Guess Jagex keeps updating the engine... Previously it was mostly 'rt' that were connected...

Olly
12-21-2013, 12:28 AM
So how does this work in mines where there is no shadow available?

PS: Didn't know the OCR for OSR was broken that much? It used to work quite well, I believe... Guess Jagex keeps updating the engine... Previously it was mostly 'rt' that were connected...

Well, the filters were never done for OSR and even bigger I only just got you to add shadowed fonts :p

slacky
12-21-2013, 01:55 AM
So how does this work in mines where there is no shadow available?
It works fine in mines, even works over pure black. The shadows are available even there, they might not be visible due to blending in to background, but that does not matter.
The engine first matches the chars color, if they match then it goes on an check if shadow also matches (if the background/shadow-points is darker then a given number).

Millenium
12-21-2013, 04:56 PM
function ocr_IsTextMulti(TextArr:TStringArray; Chars:TCharsN): Boolean;


What goes in the Chars parameter?

Millenium
12-21-2013, 04:57 PM
So how does this work in mines where there is no shadow available?

PS: Didn't know the OCR for OSR was broken that much? It used to work quite well, I believe... Guess Jagex keeps updating the engine... Previously it was mostly 'rt' that were connected...

I've been using WaitUpTextMulti and since yesterday it hasn't worked at all. Not even for detecting a single character. :/

slacky
12-21-2013, 05:41 PM
function ocr_IsTextMulti(TextArr:TStringArray; Chars:TCharsN): Boolean;


What goes in the Chars parameter?
The example in the main post shows usage, read it. The Font which you have loaded goes in Chars parameter.

Millenium
12-21-2013, 06:15 PM
The example in the main post shows usage, read it. The Font which you have loaded goes in Chars parameter.

Already read it, it still remains unclear.


begin
ocr_LoadFont(AppPath + 'Fonts\UpChars07_s\', UpFont);

ocr_FreeFont(UpFont);
end.

Is that all I need to load the font, but there is no explanation, is UpFont in ocr_LoadFont where the font discovered is stored? Or is it something different?

EDIT: I added it in like this and it posts an access violation, so I guess that's the wrong way, with UpFont as a global variable and placed in that parameter. Also defined and freed in the exact same way you do.

slacky
12-22-2013, 05:28 AM
Already read it, it still remains unclear.


begin
ocr_LoadFont(AppPath + 'Fonts\UpChars07_s\', UpFont);

ocr_FreeFont(UpFont);
end.

Is that all I need to load the font, but there is no explanation, is UpFont in ocr_LoadFont where the font discovered is stored? Or is it something different?

EDIT: I added it in like this and it posts an access violation, so I guess that's the wrong way, with UpFont as a global variable and placed in that parameter. Also defined and freed in the exact same way you do.

My example contains everything you must have. The font must also be in place, it is in the rar you downloaded), it should be placed in Fonts folder.

IsUpTextMulti, IsUpText, GetUpTextEx:

program new;
{$loadlib rstext.dll}

var UpFont:TCharsN;

function ClientATIA(x1,y1,width,height: Integer): T2DIntegerArray;
var bmp:Integer;
begin
bmp := BitmapFromClient(x1,y1,x1+width,y1+height);
Result := GetBitmapAreaColors(bmp, 0,0,width,height);
FreeBitmap(bmp);
end;

function RS_GetUpText2: String;
begin
ocr_SetClient(ClientATIA(7,7,500,16));
Result := ocr_ReadTextEnd(UpFont,0,200,'more options');
end;

function RS_IsUpTextMulti(Text: TStringArray): Boolean;
begin
//update client
ocr_SetClient(ClientATIA(7,7,500,16));
//check for text
Result := ocr_IsTextMulti(Text, UpFont);
end;

function RS_IsUpText(Text:String): Boolean;
begin
ocr_SetClient(ClientATIA(7,7,500,16));
Result := ocr_IsTextEx(Text, True, UpFont);
end;

var
t,i: Integer;
text: String;
begin
//At beginning of script call to load the font from path in to UpFont:
ocr_LoadFont(AppPath + 'Fonts\UpChars07_s\', UpFont);

//usage in scipt:
WriteLn(RS_IsUpTextMulti(['hello', 'world']);

//At end of script call:
ocr_FreeFont(UpFont);
end.

Access violation most likely due to the font not being loaded. That might be caused if you don't load the correct path: AppPath + 'Fonts\UpChars07_s\' - you need to have everything which is in the RAR extracted, this is as simple as extracting the rar directly in to Simba-folder.

Wizzup?
12-22-2013, 10:46 AM
I've been using WaitUpTextMulti and since yesterday it hasn't worked at all. Not even for detecting a single character. :/

Well, that is possible. I never tested the OCR filters and engine against the ``new'' old runescape. It did work quite well before they split, though... Has anyone even looked at making filters? Beyond Olly?
What I see warpie describe is basically what my engine in Simba does. So I'd be surprised if it wouldn't just work with the right colours and some minor tweaks. :)

Brandon
12-22-2013, 04:20 PM
Well, that is possible. I never tested the OCR filters and engine against the ``new'' old runescape. It did work quite well before they split, though... Has anyone even looked at making filters? Beyond Olly?
What I see warpie describe is basically what my engine in Simba does. So I'd be surprised if it wouldn't just work with the right colours and some minor tweaks. :)


I tried it. Doesn't work:


{$DEFINE SMART}
{$i SRL/SRL.Simba}

const
ocr_Limit_High = 190;
ocr_Limit_Med = 130;
ocr_Limit_Low = 65;

ocr_White = 16777215; //correct.
ocr_Green = 65280; //correct.
ocr_Red = 131800; //correct.
ocr_Yellow = 65535; //correct.
ocr_Blue = 13819407; //correct.
ocr_ItemC = 3572697; //correct.
ocr_ItemC2 = ocr_Red or ocr_Green; //dunno what this is.
ocr_Purple = 8388736; //wrong.. shadow is black.. Fails completely if this changes even by +- 1.

OF_LN = 256;
OF_HN = -1;

function load0(rl, rh, gl, gh, bl, bh, set_col: integer; is_text_color: boolean): TOCRFILTERDATA;
begin
result.r_low := rl;
result.r_high := rh;
result.g_low := gl;
result.g_high := gh;
result.b_low := bl;
result.b_high := bh;
result.set_col := set_col;
result._type := 0;
result.is_text_color:= is_text_color;
end;

procedure filter;
var filterdata: TOCRFilterDataArray;
begin
setlength(filterdata, 9);

filterdata[0] := load0(OCR_LIMIT_LOW, OF_HN, OF_LN, OCR_LIMIT_HIGH, OF_LN, OCR_LIMIT_HIGH, ocr_Blue, True);
filterdata[1] := load0(OCR_LIMIT_LOW, OF_HN, OF_LN, OCR_LIMIT_HIGH, OCR_LIMIT_LOW, OF_HN, ocr_Green, True);
filterdata[2] := load0(OCR_LIMIT_LOW, OF_HN, OF_LN, OCR_LIMIT_HIGH, OCR_LIMIT_LOW, OF_HN, ocr_Green, True);
filterdata[3] := load0(OF_LN, OCR_LIMIT_HIGH, OF_LN, OCR_LIMIT_HIGH, OCR_LIMIT_LOW, OF_HN, ocr_Yellow, True);
filterdata[4] := load0(OF_LN, OCR_LIMIT_HIGH, OCR_LIMIT_LOW, OF_HN, OCR_LIMIT_LOW, OF_HN, ocr_Red, True);
filterdata[5] := load0(OF_LN, OCR_LIMIT_HIGH, OF_LN, OCR_LIMIT_LOW, OCR_LIMIT_LOW, OF_HN, ocr_Red, True);
filterdata[6] := load0(OCR_LIMIT_HIGH + 10, OCR_LIMIT_MED, OF_LN, OCR_LIMIT_LOW - 10, 20, OF_HN, ocr_Green, True);
filterdata[7] := load0(OCR_LIMIT_LOW, OF_HN, OF_LN, OCR_LIMIT_HIGH, OCR_LIMIT_LOW, OF_HN, ocr_Green, True);
filterdata[8] := load0(OCR_LIMIT_LOW, OF_HN, OCR_LIMIT_LOW, OF_HN, OCR_LIMIT_LOW, OF_HN, ocr_Purple, False);

rs_SetUpTextFilter(filterdata);
end;

var
Smart: TSmart;
BMP: Integer;
begin
Smart.Create(765, 503);
MoveMouse(620, 233);
filter;

BMP := BitmapFromCLient(6, 6, 76, 24);
FastReplaceColor(BMP, 1068104, 0);
FastReplaceColor(BMP, 1268309, 0);
FastReplaceColor(BMP, 1265742, 0);
FastReplaceColor(BMP, 1267025, 0);
FastReplaceColor(BMP, 1267020, 0);
FastReplaceColor(BMP, 1268304, 0);
DisplayDebugImgWindow(70, 18);
DrawBitmapDebugImg(Bmp);

SetTargetBitmap(bmp);
writeln(rs_GetUpTextAt(0, 0));

rs_ResetUpTextFilter;
end.

slacky
12-22-2013, 06:56 PM
Well, that is possible. I never tested the OCR filters and engine against the ``new'' old runescape. It did work quite well before they split, though... Has anyone even looked at making filters? Beyond Olly?
What I see warpie describe is basically what my engine in Simba does. So I'd be surprised if it wouldn't just work with the right colours and some minor tweaks. :)
I however do not do any filtering... it's not really needed.. I do no character splitting either.. I look trough the image as it is, from left to right. Don't match to specific colors either.
I have not read completely trough what you do in yours, but there are some notable differences if I remember correctly.

Olly
12-22-2013, 08:00 PM
I tried it. Doesn't work:



That shouldn't work, rs_getUpTextAt is hardcoded to use the old upchars font (what rs3 is now), for osr you would have to do rs_GetUpTextAtEx with upchars07_s (i think) :p

Brandon
12-22-2013, 09:09 PM
That shouldn't work, rs_getUpTextAt is hardcoded to use the old upchars font (what rs3 is now), for osr you would have to do rs_GetUpTextAtEx with upchars07_s (i think) :p

Well now that really sucks. Should fix the hardcoding :S


Anyway, in that case, the code works then:

http://i.imgur.com/ZQ6nrh5.png
http://i.imgur.com/8KFixcT.png
http://i.imgur.com/AvnEf0f.png

And it's flawless.. Only thing is that it shows duplicates but that's nothing a regex can't fix. IIRC, GetUptext using a regex for matching anyway. Well it used to.

Millenium
12-23-2013, 01:29 AM
My example contains everything you must have. The font must also be in place, it is in the rar you downloaded), it should be placed in Fonts folder.

IsUpTextMulti, IsUpText, GetUpTextEx:

program new;
{$loadlib rstext.dll}

var UpFont:TCharsN;

function ClientATIA(x1,y1,width,height: Integer): T2DIntegerArray;
var bmp:Integer;
begin
bmp := BitmapFromClient(x1,y1,x1+width,y1+height);
Result := GetBitmapAreaColors(bmp, 0,0,width,height);
FreeBitmap(bmp);
end;

function RS_GetUpText2: String;
begin
ocr_SetClient(ClientATIA(7,7,500,16));
Result := ocr_ReadTextEnd(UpFont,0,200,'more options');
end;

function RS_IsUpTextMulti(Text: TStringArray): Boolean;
begin
//update client
ocr_SetClient(ClientATIA(7,7,500,16));
//check for text
Result := ocr_IsTextMulti(Text, UpFont);
end;

function RS_IsUpText(Text:String): Boolean;
begin
ocr_SetClient(ClientATIA(7,7,500,16));
Result := ocr_IsTextEx(Text, True, UpFont);
end;

var
t,i: Integer;
text: String;
begin
//At beginning of script call to load the font from path in to UpFont:
ocr_LoadFont(AppPath + 'Fonts\UpChars07_s\', UpFont);

//usage in scipt:
WriteLn(RS_IsUpTextMulti(['hello', 'world']);

//At end of script call:
ocr_FreeFont(UpFont);
end.

Access violation most likely due to the font not being loaded. That might be caused if you don't load the correct path: AppPath + 'Fonts\UpChars07_s\' - you need to have everything which is in the RAR extracted, this is as simple as extracting the rar directly in to Simba-folder.

EDIT: It now runs without an access violation, thanks. But it still won't detect the text at all, the client bounds are included and it is loading the right text bmps, could it possibly be becuase of things in the background of the uptext area? like trees, fences ect.

slacky
01-27-2014, 03:12 AM
EDIT: It now runs without an access violation, thanks. But it still won't detect the text at all, the client bounds are included and it is loading the right text bmps, could it possibly be becuase of things in the background of the uptext area? like trees, fences ect.
Sorry, it's a late replay.. But honestly I don't see why it's not working. I have tested it in the worst possible places, and it works fine.
Some regular noise should not be enough to upset the process.

If I am going to guess where the problem lies (as you don't gave me enough info):
> Wrong font (You are not using the font I added in the RAR).
> Wrong bounds: the Y-axis need to be perfect.
> Increase the height for the search area: ocr_SetClient(ClientATIA(7,7,500,25))..


One of the above is the cause.. Bounds is correct for OSRS. Font is also correct for OSR. Height 16 also works for OSR..
Honestly if you are trying to use this for OSR, my example should work as is. If you are trying it for some other game, or RSPS, please say so.

fulldragons
08-26-2015, 05:24 PM
can you please tell me where i should extract this file? to which folder, thanks