WANTED: Huge SPS map

[Incurable]

Hi,

I want to release a script that will take a new character to and unlock every F2P lodestone by walking/teleporting (eg, do one's around Burthorpe, lodestone to Lumbridge to grab Al Kharid, etc) but I'm going to need SPS maps of the entire route in order to do it... so, does anyone have an updated map of the entire F2P surface that they'd be willing to share, or am I the first to want it/asking too much? I don't mind making it myself, but if I can save a few hours of time I'm not going to say no.

Cheers.

[Frement]

I don't have the map, but I want the script.

[Justin]

I don't have the map, but I want the script.

Leecher!

I might make one, I'll start now :P

[Clarity]

Howabout a script that takes the screenshots, photoshops them together, and saves them as one big map...

Hmm...

[NKN]

Howabout a script that takes the screenshots, photoshops them together, and saves them as one big map...

Hmm...
You could probably revive the old tool that did that.

[Spaceblow]

But wouldn't it take a long time to calculate the position according to The Mayor's tutorial (https://villavu.com/forum/showthread.php?t=107757)?


However, the bigger you make your map, the longer it takes Simba to calculate where you are, so make your map big enough, but not outrageously big!

[Olly]

Yes, anything over 500x500 gets too slow imo.

[Camel]

To get around 500ms calls of get position, you could just make several SPSAreas as opposed to one big map. Doing that might also allow you to not map some areas you would have had to.

[Justin]

Alright, I've mapped out most of the F2P map,

http://puu.sh/bVdQc/129bede513.jpghttp://puu.sh/bVdZR/4cc41bd208.jpghttp://puu.sh/bVe4f/66a3fab7bc.jpg

-- worldmap.getPlayerPos(): result = {X = 762, Y = 1438}, took 4812 ms

Only issue is, when loading a huge map, the time to calculate your position becomes long

This is the map zoomed out 25%
http://puu.sh/bVdTC/ac9fa5c325.jpg

I think your best bet would be to create several smaller maps, load them, walk the path then load the next set of maps up and repeat

I think i'm going to stop my project of mapping the entire f2p place as walking with the map would be horrible (loading time)

Here is the map if anyone wanted to keep working on it/finish it.

http://puu.sh/bVec0/3a893be210.png

[Ross]

But wouldn't it take a long time to calculate the position according to The Mayor's tutorial (https://villavu.com/forum/showthread.php?t=107757)?


Yes, anything over 500x500 gets too slow imo.


...]


For a regular script this could be an issue, but IMO a script that will only be run once to unlock lodes I wouldn't care if it took 60 seconds to calc my position :p

Also nice work Justin!

[Spaceblow]

Alright, I've mapped out most of the F2P map,Good job, I think this can be helpful for a lot of people that are working on a script and need a map for it, now they just have to slice out the part they need. Maybe you need to make a thread or something because this is really handy.

[cosmasjdz]

Also you can scale the map interface to nearly whole screen so it covers huge area. SO you can get less screenshots for the same area. for example in my scripts walking to rune ores tooked like 6 or 7 screenshots only to make a map.

[Justin]

Good job, I think this can be helpful for a lot of people that are working on a script and need a map for it, now they just have to slice out the part they need. Maybe you need to make a thread or something because this is really handy.

I'd like to post the completed map, not a half completed map. Maybe one day during the week I might finish it off but no promises to be honest.

[Incurable]

Alright, I've mapped out most of the F2P map

Wow, that's awesome, thank you so much! You really should post a thread, so many people could make use of that. :D

So, could someone please explain two things to me?

1. Does SPS have to load the map every time it wants to calculate position, hence why a large map is a bad idea?
2. How would I load a small map, walk it, free it (does it need freeing from memory?), and then load another before continuing?

[Justin]

...

Loading the map I posted takes around 400ms, the reason why (I believe) it takes so long to calculate your position is because it needs to scan every 70x70 box on the map


var worldmap: TSPSArea;
worldmap.setup('SPS_worldmap', RUNESCAPE_OTHER); //loads the "SPS_worldmap" image located in C:\Simba\Includes\SPS\img\runescape_other folder
worldmap.walkPath([Point(100,100), Point(105,105), Point(130,120)]); //Walks the path using the worldmap SPS map

[Ian]

I made this around 1 year ago, so some of it is outdated (clan camp/lumnbridge world event area) but it might be helpful for someone.

It includes the entire F2P map excluding the wilderness.

Also there are no symbols or minimap dots so you don't need to remove them.

Preview:
http://i.imgur.com/fpcGxGO.png

Full image:
http://i.imgur.com/5G7fY7B.png

[Lipcot]

1. Does SPS have to load the map every time it wants to calculate position, hence why a large map is a bad idea?
2. How would I load a small map, walk it, free it (does it need freeing from memory?), and then load another before continuing?

1. Yes
2. i dont think you have to free the map, but i have no idea how to do this, although i´ve tried unsuccesfully in the past but was too shy to ask how to do it.

[Spaceblow]

I made this around 1 year ago, so some of it is outdated (clan camp/lumnbridge world event area) but it might be helpful for someone.
This is also really impressive, good work! I see that you've removed all the symbols, does this make such a difference?

[Lipcot]

...


...

i was wondering the same thing, since all my SPS maps include symbols and work pretty well!

[Justin]

Back when the minimap icons were dynamic (would shift positions every time the map chuck was loaded) it did help/make a difference, but since the map icons are now static there really isn't a need to remove them IMO

[Incurable]

Back when the minimap icons were dynamic (would shift positions every time the map chuck was loaded) it did help/make a difference, but since the map icons are now static there really isn't a need to remove them IMO

Are you sure they're static now? I could swear they're dynamic.

[Ian]

This is also really impressive, good work! I see that you've removed all the symbols, does this make such a difference?

It's not that I removed the symbols, it's that they were never captured. This was done with Brandon's OGL plugin that allowed capturing the minimap without symbols or dots.

As for if it makes a difference or not: It does, but not very much of one. For example, if you capture the minimap (with symbols) when your camera is facing north, the symbols will be facing north too. But then when you're using SPS if you're using it at an angle that is not north the symbols will be different relative to the ground.

I do not think it makes very much of a difference though because of the tolerance that SPS allows, because when I was using SPS on legacy (That has the old symbols) I used an SPS map from EOC with the new symbols and it still worked.

[Turpinator]

1. Yes
2. i dont think you have to free the map, but i have no idea how to do this, although i´ve tried unsuccesfully in the past but was too shy to ask how to do it.

no you dont need to free it really because the map is simply stored as a matrix of values of color.

To improve time performance of justin's map, you could simply decrease the accuracy (make the number larger), as there would be less 'blocks' to compare. this would make it faster, but less reliable. if one were to make a lodestone unlocker, id suggest maps from A->B then B->C, etc. and find the distance between each and what the fastest route would be (but it doesnt have be be continuous as you can simply teleport back to a lodestone you have already unlocked.

[Spaceblow]

It's not that I removed the symbols, it's that they were never captured. This was done with Brandon's OGL plugin that allowed capturing the minimap without symbols or dots.

As for if it makes a difference or not: It does, but not very much of one. For example, if you capture the minimap (with symbols) when your camera is facing north, the symbols will be facing north too. But then when you're using SPS if you're using it at an angle that is not north the symbols will be different relative to the ground.

I do not think it makes very much of a difference though because of the tolerance that SPS allows, because when I was using SPS on legacy (That has the old symbols) I used an SPS map from EOC with the new symbols and it still worked.That makes sense, thanks for the explanation!

[slacky]

Loading the map I posted takes around 400ms, the reason why (I believe) it takes so long to calculate your position is because it needs to scan every 70x70 box on the map
...
Elaborating a bit further: It has to compare the patch against the source image by sliding the patch one pixel at a time (left to right, top to bottom), and at each pixel the whole patch (sub image) has to be compared to each pixel insight based on the current patch position.

Example, imagine that each light-gray square is a pixel in the world map, and the green-ish square is the patch (aka snapshot of your minimap on RS):

http://slackworld.net/downloads/SPS2.gif
The dark-gray points which are left holds the value of the similarity at that corresponding point.

So given a patch that is 70x70, and a world map that is 1000x1000 we are left with about { 931*931 * 70*70 = } 4,250,000,000 pixel-comparisons if SPS accuracy is set to 1

[Clarity]

Back when the minimap icons were dynamic (would shift positions every time the map chuck was loaded) it did help/make a difference, but since the map icons are now static there really isn't a need to remove them IMO

Isn't it more complex now that they are not circular, they have an arrow at the bottom, and rotate that arrow/symbol based on minimap compass orientation?
Maybe SPS doesn't care though.

[The Mayor]

http://slackworld.net/downloads/SPS2.gif


That gif though :p

[Justin]

Isn't it more complex now that they are not circular, they have an arrow at the bottom, and rotate that arrow/symbol based on minimap compass orientation?
Maybe SPS doesn't care though.

From my testing, SPS has been almost spot on every time with the default settings, I'm not sure what you mean by arrows at the bottom but as for the symbols changing on compass orientation, SPS is always set to north (unless anyAngle is set to true) so they minimap symbols match up with the map

Thanks for the information slacky

[Incurable]

1. Yes
2. i dont think you have to free the map, but i have no idea how to do this, although i´ve tried unsuccesfully in the past but was too shy to ask how to do it.

Thank you. :)

You should never be too shy, unless you're asking how to make letters appear on the screen using the plastic button thingy on your desk... :p

One of my favourite phrases from an unknown author is: "The only stupid questions are those left unasked."

It means that you should never be too afraid to ask questions. Asking questions is how we learn, and the greatest minds are great in part because they never stop asking questions. ;)


It's not that I removed the symbols, it's that they were never captured. This was done with Brandon's OGL plugin that allowed capturing the minimap without symbols or dots.

Is there a thread where I can learn more about this? I'm really interested in using Brandon's OGL plugin, but I don't have a clue where to start.


no you dont need to free it really because the map is simply stored as a matrix of values of color.

To improve time performance of justin's map, you could simply decrease the accuracy (make the number larger), as there would be less 'blocks' to compare. this would make it faster, but less reliable. if one were to make a lodestone unlocker, id suggest maps from A->B then B->C, etc. and find the distance between each and what the fastest route would be (but it doesnt have be be continuous as you can simply teleport back to a lodestone you have already unlocked.

Thank you, I think I know where to start with this script now. :D


Elaborating a bit further: It has to compare the patch against the source image by sliding the patch one pixel at a time (left to right, top to bottom), and at each pixel the whole patch (sub image) has to be compared to each pixel insight based on the current patch position.

Example, imagine that each light-gray square is a pixel in the world map, and the green-ish square is the patch (aka snapshot of your minimap on RS):

http://slackworld.net/downloads/SPS2.gif
The dark-gray points which are left holds the value of the similarity at that corresponding point.

So given a patch that is 70x70, and a world map that is 1000x1000 we are left with about { 931*931 * 70*70 = } 4,250,000,000 pixel-comparisons if SPS accuracy is set to 1

That gif is incredibly helpful, thank you. I had no idea that that's how SPS worked, and I'm really quite impressed at the genius of whoever proposed the idea in the first place.

[The Mayor]

I'm really quite impressed at the genius of whoever proposed the idea in the first place.

That would be marpis;

[slacky]

I had no idea that that's how SPS worked, and I'm really quite impressed at the genius of whoever proposed the idea in the first place.
It works, but it's not the best way of going about it in this case, as it will result in a crazy amount of comparisons. It can be made faster by doing the correlation in frequency domain (using fourier transform) rather then spatial domain, while keeping the same accuracy. The way SPS does it, is considered the "naive way", it's simple it works, but it's far from optimal when the area of the patch and image is large.
SPS pretty much uses a replica of the algorithm posted in this article (http://en.wikipedia.org/wiki/Template_matching#Implementation), except for some changes, like matching colored values, rather then just gray, and pre-processing to downsample the image and the patch.

[Incurable]

It works, but it's not the best way of going about it in this case, as it will result in a crazy amount of comparisons. It can be made faster by doing the correlation in frequency domain (using fourier transform) rather then spatial domain, while keeping the same accuracy. The way SPS does it, is considered the "naive way", it's simple it works, but it's far from optimal when the area of the patch and image is large.
SPS pretty much uses a replica of the algorithm posted in this article (http://en.wikipedia.org/wiki/Template_matching#Implementation), except for some changes, like matching colored values, rather then just gray, and pre-processing to downsample the image and the patch.

I understood none of that, but it sounds cool anyway. Does that mean though that SPS could actually be rewritten to use the "better" way and enable the use of large maps without the massive calculation times?

[slacky]

I understood none of that, but it sounds cool anyway. Does that mean though that SPS could actually be rewritten to use the "better" way and enable the use of large maps without the massive calculation times?
It will probably be quite a bit faster, tho depends on the implementation. This concept should/will be in the next (big) SPS release (https://github.com/SRL/SPS/tree/Next/plugin/source). Still what's in the SPS-"next" is not the most optimal solution. EG: It does not keep the image (read: world map) in the frequency domain, meaning we have to transform it every time you walk a step, and that is a waist of time, but it's a good start.

[Incurable]

It will probably be quite a bit faster, tho depends on the implementation. This concept should/will be in the next (big) SPS release (https://github.com/SRL/SPS/tree/Next/plugin/source). Still what's in the SPS-"next" is not the most optimal solution. EG: It does not keep the image (read: world map) in the frequency domain, meaning we have to transform it every time you walk a step, and that is a waist of time, but it's a good start.

So this is a purely ignorant question, but why is it not being written with the "most optimal" solution as opposed to one that's almost as good?

[slacky]

So this is a purely ignorant question, but why is it not being written with the "most optimal" solution as opposed to one that's almost as good?
Because it takes time and knowledge. And those with knowledge usually lack the time, and those with time, lack the knowledge. And then we have people like me, who just doesn't give a shit.

More on point: The current (read: next SPS) implementation relies on a third part library to do the real processing. A specialized implementation (only for SPS) of something like that will be pretty large, and quite advanced, and has to be put a lot of time into, that is if one wants to beat the current implementation (the current one is good). And the man who wrote it is probably one of the only ones who would understand it, which makes it quite hard to maintain.

[Incurable]

Because it takes time, and knowledge. And those with knowledge usually lacks the time, and those with time, lacks the knowledge. And then we have people like me, who just doesn't give a shit.

More on point: The current implementation relies on a third part library to do the real processing. A specialized implementation (only for SPS) of something like that will be pretty large, and quite advanced, and has to be put a lot of time into, that is if one wants to beat the current implementation (the current one is good). And the man who wrote it is probably one of the only ones who would understand it, which makes it quite hard to maintain.

I figured it was something like that, fair enough. Thanks for taking the time to explain it all to me. :)

[The Mayor]

..those with knowledge usually lack the time, and those with time, lack the knowledge. And then we have people like me, who just doesn't give a shit.


Best thing I've read in a while :D

[the bank]

A better solution would be to store the regions SPS maps in some sort of database (maybe JSON even?) and retrieve them region-by-region.

Ironically, thats exactly how RS actually defines and changes regions.

[marpis]

Elaborating a bit further: It has to compare the patch against the source image by sliding the patch one pixel at a time (left to right, top to bottom), and at each pixel the whole patch (sub image) has to be compared to each pixel insight based on the current patch position.

Example, imagine that each light-gray square is a pixel in the world map, and the green-ish square is the patch (aka snapshot of your minimap on RS):

http://slackworld.net/downloads/SPS2.gif
The dark-gray points which are left holds the value of the similarity at that corresponding point.

So given a patch that is 70x70, and a world map that is 1000x1000 we are left with about { 931*931 * 70*70 = } 4,250,000,000 pixel-comparisons if SPS accuracy is set to 1

I'm amazed that someone is still using or even looking at SPS, thanks guys!
Slacky I'm glad you spent time to understand how SPS works... almost ;)
Instead of going pixel-by-pixel, SPS first lowers the resolution of both minimap and the world map, so that each 4x4=16 pixels box turns into a big pixel. Kind of.
This works in a way that SPS_MakeColorBox() goes through individual pixels from an actual bitmap, and sums all red, green and blue values. It then gives you the totals in an integer array with length of 3.
So this TIntegerArray now represents a "SPS pixel", with tia[0]=red, tia[1]=green and tia[2]=blue.
Areas of these "SPS pixels" can be represented via 3-dimensional integer array, where the dimensions are as follows TIA[x][y][r/g/b].
After this "lowering of resolution" is done, the method depicted in the gif applies. Kudos to whoever made the gif, I could not explain the principle better myself.

This saves a good deal of iterations without actually lowering the accuracy of positioning. I doubt the actual method can get any more efficient in this use, only the used algorithms and code.

---

I still lurk around every now and then, by the way :) Pretty busy studying though, I just started doing stem cell research

[slacky]

I'm amazed that someone is still using or even looking at SPS, thanks guys!
Slacky I'm glad you spent time to understand how SPS works... almost ;)
Instead of going pixel-by-pixel, SPS first lowers the resolution of both minimap and the world map, so that each 4x4=16 pixels box turns into a big pixel. Kind of.
This works in a way that SPS_MakeColorBox() goes through individual pixels from an actual bitmap, and sums all red, green and blue values. It then gives you the totals in an integer array with length of 3.
So this TIntegerArray now represents a "SPS pixel", with tia[0]=red, tia[1]=green and tia[2]=blue.
Areas of these "SPS pixels" can be represented via 3-dimensional integer array, where the dimensions are as follows TIA[x][y][r/g/b].
After this "lowering of resolution" is done, the method depicted in the gif applies. Kudos to whoever made the gif, I could not explain the principle better myself.
I mentioned the down-sampling technique of the image in my 2. post in this thread (without going in to any detail).
As the down-sampling is just a prepossessing step, and not directly a part of the algorithm which cross-correlates the images, I did not feel like going in to detail on it. And adding that step to the gif would make the gif so over-complicated.. ;P
PS: I made that Gif, so thanks for the kudos.