OCR with Simba + Anti Aliasing.

[caused]

I'm trying to read fonts on the Screen.

Using GetTextAt and my own font I created.

Since the text has anti-aliasing and is rather small, it often detects leters wrong. Is there some way to optimize the ocr to highten the detection rate ?

Heres an example of text that I was trying to read.

http://666kb.com/i/c4i1fw7c4fdw8s0t6.png

It detects the Numbers fine, but when it comes to the letters it fails really badly..

If you want i can also upload the font I created.

~caused

[masterBB]

The bottle neck of OCR is the amount of colors you find. Make sure you got the color and tolerance right. Look into srl text.simba for a blue print to create your own color finding method like with different tolerances and maybe even a different CTS. Simba receives a TPA and will match point from that TPA on the font you created. Make sure it found all the points.

[Main]

This might not be exactly what your looking for. but

If the phrase that you are trying to detect are quite different, ie some are 3 characters and some are 15 character.

I find that using length of the color (of the text) can do quite a decent job and I used this during the period where rs is constantly updating their font.


Heres an example in rs:
So instead of looking for "Enter" you do

If FindColorsTolerance( X,Y, TPA, TextCol, Xs, Ys, Xe, Ye) Then
Begin
If Length( TPA) = 3580 Then
Blah;
End;


The login procedure for SRL also use this, because its a hassel to create ocr just for the login text.

[Sex]

This might not be exactly what your looking for. but

If the phrase that you are trying to detect are quite different, ie some are 3 characters and some are 15 character.

I find that using length of the color (of the text) can do quite a decent job and I used this during the period where rs is constantly updating their font.


Heres an example in rs:
So instead of looking for "Enter" you do

If FindColorsTolerance( X,Y, TPA, TextCol, Xs, Ys, Xe, Ye) Then
Begin
If Length( TPA) = 3580 Then
Blah;
End;


The login procedure for SRL also use this, because its a hassel to create ocr just for the login text.

I'm pretty sure he is trying to actually read text.

[Main]

Yes indeed, but if he have a fix set of words then length of the text tpa should be fine :)

[caused]

The bottle neck of OCR is the amount of colors you find. Make sure you got the color and tolerance right. Look into srl text.simba for a blue print to create your own color finding method like with different tolerances and maybe even a different CTS. Simba receives a TPA and will match point from that TPA on the font you created. Make sure it found all the points.

Yup thats what I thought.

The problem I think there is is that.. for example:

It scans the letter e, and the letter e "with anti aliasing and color tolerance" contains an r. So r will be in e, and i can never detect e. So the text is messed up.. Just as an example. I hope that you can follow.

I think i need to add fewer points to my font and less tolerance then.

But how can i know how much tolerance is needed to find the points I want ?
and to not find other points so that it misreads the text.

I'm using Photoshop which allows me to create a color selection with tolerance(Select Color Range), is that kind of tolerance similar to simbas tolerance ?

Actually if you're able to make simba read that text, you're awesome xD... I'm working on it for hours now ;/ ...

@Main: I need to read the exact text and put it to a string for further filtering.

[masterBB]

Yup thats what I thought.

The problem I think there is is that.. for example:

It scans the letter e, and the letter e "with anti aliasing and color tolerance" contains an r. So r will be in e, and i can never detect e. So the text is messed up.. Just as an example. I hope that you can follow.

I think i need to add fewer points to my font and less tolerance then.

But how can i know how much tolerance is needed to find the points I want ?
and to not find other points so that it misreads the text.

I'm using Photoshop which allows me to create a color selection with tolerance(Select Color Range), is that kind of tolerance similar to simbas tolerance ?

Actually if you're able to make simba read that text, you're awesome xD... I'm working on it for hours now ;/ ...

@Main: I need to read the exact text and put it to a string for further filtering.

Sorry I delayed all my school work, so I practically have to do my entire school year this week. Might have some time to look into it sunday, but probably not.

If you want to know what simba uses as tolerance you can read my cts tutorial.
Summary:
cts 0 -> result := reddif < tol and greendif < tol and bluedif < tol;
cts 1(standard) -> result := reddif * reddif + greendif * greendif + bluedif * bluedif < tol * tol;

[caused]

Than better get back to work ;) ...

If you find time to look into it, that'd be much appreciated though :)

Or if someone else got an idea how to read this kind of text.

[caused]

Bump

I'm still struggling with this.
I now did the following:
a) using findcolors with tolerance to find the points gettextat is looking for
b) making a font of exactly these points
c) using gettextat with the same tolerance and color from a)

it's still messing up the text though :/ .. numbers seem to somewhat work.

~caused

[caused]

Okay. I figured the gettextat function is just not working with small anti aliased text.

No matter what I did it failed.

So I thought about writing a more advanced OCR for simba.

Actually I thought like doing it like this:

a) Create Bitmaps like the current ones

b) Function turns bitmaps to an tpoint[][]

c) Script gets specified region

d) Script searches for pixel in defined color and returns an array of tpoint.

e) Theres a max letter size, and in this size the script looks for the charachter where most pixels line up..

f) if theres more than one image, where like 10 pixel line up it looks for the image, where there are the least pixel "remaining"(on the source not used)

g) returns letter and proceeds x pixel to next charachter depending on the size of the current character.


What do you think ?

Is that a good way to read chars from the screen ?

Dang.. I wish the current functions would work :( ...

~caused

[Brandon]

Hmm.. I just tried creating the font.. It's really hard.. works somewhat:


{$I SRL/SRL.Simba}
{$I SRL/SRL/Misc/Debug.Simba}


Procedure TPAToBitmap;
var
TPA: TPointArray;
ATPA: T2DPointArray;
CTS, I, L: Integer;
begin
CTS:= GetColorToleranceSpeed;
ColorToleranceSpeed(2);
SetColorSpeed2Modifiers(0.00, 1.14);
FindColorsTolerance(TPA, 14506843, 611, 304, 752, 317, 50);
writeln(GetTextAtExWrap(611, 304, 752, 317, 0, 5, 1, 14506843, 30, 'Caused'));
ColorToleranceSpeed(CTS);
SetColorSpeed2Modifiers(0.2, 0.2);
ATPA:= TPAToATPAEx(TPA, 5, 10);
DebugATPA(ATPA, '');
L:= High(ATPA);
for I:=0 To L Do
begin
DebugTPA(ATPA[I], '');
//Wait(2000);
ClearDebugImg;
end;
end;

begin
SetupSRL;
TPAToBitmap;
end.



I dont know how to make an ATPA into a bitmap or match it with a font.. BUT the idea I had was simple. Basically grab a TPA of all the colors on the text. Split it to the size of the font which i found to be 5-7. Regardless of height.

Somehow convert Each ATPA[I] To a bitmap and match that to every character in my font folder. Unfortunately i don't know how to make the bitmap from the ATPA so it's only a theory for now.

My fonts print almost 1/3 then it starts printing the music symbol lol.. I put the GetText between speed modifiers because it picks up more characters than without it.

[caused]

Hey Brandon.

Thanks ALOT for the effort.
I get it to print like 1/3 too it goes like "+282-657 Ferrh++en" or something like that was the best I got. Notice that it doesnt seem to have much problems with numbers.

But it has with letters like r or u.

Oh and I will try the speed modifiers thing :)

And I'm still open for suggestions : D

[Velvet Glam]

Well I started work on this yesterday because it seems like fun, however, I was having problems getting each char/digit separated from one another completely..

Post your current code so we can help you properly? and improve it

[LordJashin]

Here's some links.

Creating your own font set for simba (http://villavu.com/forum/showthread.php?t=60451)

Simba Docs (http://docs.villavu.com/simba/scriptref/ocr.html#loadfont)

Maybe try changing the font for your browser. or w/e you are trying to read from. Font style, size, family, anything. If its on a web site the source code will most likely contain this information. If not it will be the browser default.

[Runaway]

Hey caused,

I really don't know how useful this would be and if it's even an efficient way to identify text, but check out the FilterTPASegment link in my signature. I've been thinking about building an OCR around that concept since it can use gap tolerance, but I haven't really delved into it much yet.

[caused]

@Runaway: This looks very interesting. I'll look into it once i find the time.

I'll also post my OCR code that i have so far.

[caused]

I'm still struggling witht this.

I do even believe, that the gettextat function may have a bug, where it messes up everything when the minvspacing is zero. It has to be zero for such a small font though.

[Velvet Glam]

I'm still struggling witht this.

I do even believe, that the gettextat function may have a bug, where it messes up everything when the minvspacing is zero. It has to be zero for such a small font though.

So your code has not gotten each individual char by itself? I say we try to make our own make-shift OCR

[caused]

So your code has not gotten each individual char by itself? I say we try to make our own make-shift OCR

It actually gets the chars "by itself", but not a full string. It seems to have problems detecting where a character ends :/