Results 1 to 10 of 10

Thread: Review of Botting Technology (for NXT)?

  1. #1
    Join Date
    Dec 2006
    Location
    Banville
    Posts
    3,914
    Mentioned
    12 Post(s)
    Quoted
    98 Post(s)

    Default Review of Botting Technology (for NXT)?

    Going forward, what needs to be done to macro with the most recent techniques? What is the most recent method? OpenGL hooking for NXT, or? How does this compare to OSRS (it was my impression that OSRS "should work" with Simba as-is).

    I tried to ask this some time ago, but I didn't understand the answer, specifically how what needed to be done related to what had not been done already with either Simba, SRL, or an external library. Has NXT progressed to the point where cheating in it would use the same methods that one might use to create a program that fakes input to a first-person shooter?
    The jealous temper of mankind, ever more disposed to censure than
    to praise the work of others, has constantly made the pursuit of new
    methods and systems no less perilous than the search after unknown
    lands and seas.

  2. #2
    Join Date
    Mar 2012
    Posts
    107
    Mentioned
    2 Post(s)
    Quoted
    49 Post(s)

    Default

    Quote Originally Posted by R0b0t1 View Post
    Going forward, what needs to be done to macro with the most recent techniques? What is the most recent method? OpenGL hooking for NXT, or? How does this compare to OSRS (it was my impression that OSRS "should work" with Simba as-is).

    I tried to ask this some time ago, but I didn't understand the answer, specifically how what needed to be done related to what had not been done already with either Simba, SRL, or an external library. Has NXT progressed to the point where cheating in it would use the same methods that one might use to create a program that fakes input to a first-person shooter?
    I'm more interested in seeing how AI and Machine Learning is going to disrupt the botting scene.

  3. #3
    Join Date
    Dec 2006
    Location
    Banville
    Posts
    3,914
    Mentioned
    12 Post(s)
    Quoted
    98 Post(s)

    Default

    Quote Originally Posted by One Kid View Post
    I'm more interested in seeing how AI and Machine Learning is going to disrupt the botting scene.
    Some interface will be necessary to provide information to the machine learning algorithm. What will that interface be?
    The jealous temper of mankind, ever more disposed to censure than
    to praise the work of others, has constantly made the pursuit of new
    methods and systems no less perilous than the search after unknown
    lands and seas.

  4. #4
    Join Date
    Apr 2017
    Posts
    11
    Mentioned
    1 Post(s)
    Quoted
    5 Post(s)

    Default

    I've been working on a NXT bot using OpenGL on Linux.

    NXT's rendering is very strange, very much unlike the Java client or any game engine. Have you heard of a sprite batch? (If you haven't, it's a way to emit fewer draw calls for 2D things, e.g. sprites [hence the name], by batching together draws that use the same state/resources). Well, NXT uses a similar technique... for the 3D rendering. A single draw call could be rendering a dozen trees, some doors, an anvil, and some grass. Objects, like players and monsters, are rendered similar. The calls are split by textures and material properties performed by shaders (which perform operations like wrapping modes, since all textures are stored in a few giant atlases).

    I generate low-detail point clouds and perform fuzzy matches for animated objects, like monsters. To detect animations, I compare the current bone transforms (models are skinned on the GPU) with captured transforms. To match static objects, like anvils, I extract material properties and perform comparisons based on the materials. This is very slow; I manage 10-20 updates per second, and it has to be done in another thread. Some GPGPU stuff could speed it up, but you'll never reach parity with whatever FPS you have (except in simple, low populated areas).

    A few cool things: 1) Items are rendered on the GPU and then cached [using their drop models], so if you compare model data, you can detect items that are visually identical very efficiently. 2) The minimap is a 3D rendering of the immediate area, but without elevation data AFAIK. 3) The coordinates are persistent between areas, unlike Java (necessary due to draw distance/seamless loading). 4) The GUI is rendered in such a way that I devised a high-level "reverse sprite batch" to efficiently match GUI elements.

    I'm hesitant to release anything publicly, because there's a few simple things that Jagex could do that would hurt performance. I'm going to reverse engineer the client itself once I get my first bots full operational, then I suppose I would release the resources I've developed for what I call "game state extraction" via OpenGL.

    RE: Machine learning.

    My opinion is that machine learning would only be practical for things like object recognition. A behavior tree or GOAP are two of the most practical ways of executing incredibly adaptable, performant, and reliable AI that can overcome unexpected conditions, while allowing gradual improvements across the board by improving the individual nodes. AFAIK, that's what a lot of other MMO bots use (such a World of Warcraft).
    Last edited by Kompromaus; 11-01-2017 at 08:37 PM. Reason: machine learning

  5. #5
    Join Date
    Dec 2006
    Location
    Banville
    Posts
    3,914
    Mentioned
    12 Post(s)
    Quoted
    98 Post(s)

    Default

    Quote Originally Posted by Kompromaus View Post
    I've been working on a NXT bot using OpenGL on Linux.
    That is promising. Is there no longer an advantage to using Windows? Does the NXT client only target a few distributions well (e.g. Ubuntu)?

    Quote Originally Posted by Kompromaus View Post
    NXT's rendering is very strange, very much unlike the Java client or any game engine. Have you heard of a sprite batch? (If you haven't, it's a way to emit fewer draw calls for 2D things, e.g. sprites [hence the name], by batching together draws that use the same state/resources). Well, NXT uses a similar technique... for the 3D rendering. A single draw call could be rendering a dozen trees, some doors, an anvil, and some grass. Objects, like players and monsters, are rendered similar. The calls are split by textures and material properties performed by shaders (which perform operations like wrapping modes, since all textures are stored in a few giant atlases).

    I generate low-detail point clouds and perform fuzzy matches for animated objects, like monsters. To detect animations, I compare the current bone transforms (models are skinned on the GPU) with captured transforms. To match static objects, like anvils, I extract material properties and perform comparisons based on the materials. This is very slow; I manage 10-20 updates per second, and it has to be done in another thread. Some GPGPU stuff could speed it up, but you'll never reach parity with whatever FPS you have (except in simple, low populated areas).

    A few cool things: 1) Items are rendered on the GPU and then cached [using their drop models], so if you compare model data, you can detect items that are visually identical very efficiently. 2) The minimap is a 3D rendering of the immediate area, but without elevation data AFAIK. 3) The coordinates are persistent between areas, unlike Java (necessary due to draw distance/seamless loading). 4) The GUI is rendered in such a way that I devised a high-level "reverse sprite batch" to efficiently match GUI elements.
    Do you obtain all of this data by filtering OpenGL calls, or do you access it later by associating another program with the OpenGL context of the NXT process and reading the data back from the GPU?

    Conceptually I understand what is going on with OpenGL scraping (at least in the case of using trampoline functions). The main issues I have had are finding time to implement the library myself, even with help from Brandon. An outline of the process on Linux would be helpful.

    The OpenGL specification seems to have changed since this was first discussed on the forums (notably the addition of VAOs). I remember it being possible to recover a numeric ID for each grouping of vertices. Usually, the vertices were grouped by object. Even with VAOs there must be some information that uniquely identifies vertex clusters as a single textured object. You may already be doing this - if you are I misunderstood the first part of your post.

    Quote Originally Posted by Kompromaus View Post
    I'm hesitant to release anything publicly, because there's a few simple things that Jagex could do that would hurt performance. I'm going to reverse engineer the client itself once I get my first bots full operational, then I suppose I would release the resources I've developed for what I call "game state extraction" via OpenGL.
    It would help the forums a great deal if you released what you have. I think it unlikely that Jagex could seriously impact the performance of OpenGL scraping without also seriously impacting the performance of the game itself. Even if they could, more eyes trying to solve the same problems would result in better solutions.


    Quote Originally Posted by Kompromaus View Post
    RE: Machine learning.

    My opinion is that machine learning would only be practical for things like object recognition. A behavior tree or GOAP are two of the most practical ways of executing incredibly adaptable, performant, and reliable AI that can overcome unexpected conditions, while allowing gradual improvements across the board by improving the individual nodes. AFAIK, that's what a lot of other MMO bots use (such a World of Warcraft).
    I suspect it has some kind of utility with graphical detection methods. It may be possible to use it to detect vertex objects, but then the hard part is generating some measure of the vertex data that can be passed to an algorithm.
    The jealous temper of mankind, ever more disposed to censure than
    to praise the work of others, has constantly made the pursuit of new
    methods and systems no less perilous than the search after unknown
    lands and seas.

  6. #6
    Join Date
    Apr 2017
    Posts
    11
    Mentioned
    1 Post(s)
    Quoted
    5 Post(s)

    Default

    1) I just don't like Windows. It's also easier to hide my bot by compiling custom versions of MESA/Xorg if I need to (currently I don't). There's a reason VAC only works well on Windows.

    2) I wrote an in-process OpenGL tracing library, similar to apitrace, and I trace all calls in a frame. Then I process the frame trace in another thread.

    I create a copy of buffer and texture data since I need to process it on the CPU. Each model has a unique set of index buffer, vertex buffers, and a vertex array object, so you can identify them by any particular one.

    3) I'd be more than willing to release everything sans the bot itself (since that would be useless if it's going to be used with SIMBA) if I knew others would contribute in the event Jagex starts trying to break it. It's a giant task though--it's composed of a dozen different libraries, so it would be difficult for others to get up to speed. I'd also need to go back and document everything :P.

    There's actually a couple ways they could make things more difficult without affect performance or visual fidelity much, if at all.

  7. #7
    Join Date
    Jan 2012
    Location
    127.0.0.1
    Posts
    702
    Mentioned
    11 Post(s)
    Quoted
    76 Post(s)

    Default

    Quote Originally Posted by R0b0t1 View Post

    I suspect it has some kind of utility with graphical detection methods. It may be possible to use it to detect vertex objects, but then the hard part is generating some measure of the vertex data that can be passed to an algorithm.
    Edge detection is part of conv nets

  8. #8
    Join Date
    Dec 2006
    Location
    Banville
    Posts
    3,914
    Mentioned
    12 Post(s)
    Quoted
    98 Post(s)

    Default

    Quote Originally Posted by Enslaved View Post
    Edge detection is part of conv nets
    If a deterministic algorithm exists for a given problem, it is almost always faster to run and more reproducible. I might be tempted to feed edge data into a feature detector that is a neural network, but there are other methods as well.

    The above implies I'm not using something like OpenGL scraping.
    The jealous temper of mankind, ever more disposed to censure than
    to praise the work of others, has constantly made the pursuit of new
    methods and systems no less perilous than the search after unknown
    lands and seas.

  9. #9
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    Example: https://villavu.com/forum/showthread.php?t=117401

    Kompromaus was pretty much on it. I currently have a working bot that works in a similar way that he explains. However, I do manage to run it at 60 fps.
    And yes, Jagex can make it worse if they know exactly how Kompromaus (or I) did it.

    I am currently developing it with someone else from the forum. But if it was only me, I would have certainly released the code already (which is a bit of a mess though).

  10. #10
    Join Date
    Dec 2006
    Location
    Banville
    Posts
    3,914
    Mentioned
    12 Post(s)
    Quoted
    98 Post(s)

    Default

    Feels kind of silly to make an entire thread to post this, so I will put it here. After a great deal of reading I found that Microsoft does expose an API that is capable of creating Windows sessions. The landing page can be found here. The critical part is described below:



    Unfortunately despite these pages the interfaces are more or less entirely undocumented. It seems like WTSCreateListener is necessary but most of the function's purpose is left undescribed.

    This may not be worth following up on. I am currently looking at FreeRDP. Creating a session programmatically has a lot of other uses besides botting RS.
    The jealous temper of mankind, ever more disposed to censure than
    to praise the work of others, has constantly made the pursuit of new
    methods and systems no less perilous than the search after unknown
    lands and seas.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •