Results 1 to 22 of 22

Thread: OpenGL bot (RS3)

  1. #1
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Lightbulb OpenGL bot (RS3)

    Hello everyone

    Untitled.jpgUntitled2.jpg

    I'm currently developing a RS3 bot that only uses OpenGL hooking (written in C++). It identifies models (that you specify) as enemies or dropped items, including their location and name. It communicates this information to my core bot (this implements the actual actions that the player has to do).
    At first, I thought that it was kinda unique. But after searching for a while, I found out that 'ogLib' from Obscurity is almost the same, but then for Simba only.
    It does not use any reflection so (for now) it's only possible to run the game in the official RS3 client.

    So I'm not quite sure whether I would continue this bot.

    Here are some of the main features:
    - Identify 3D models in game by clicking on it and giving it a name.
    - Extracting information like MVP matrices and player position.
    - Checking for these identified 3D models in-game and communicating this to the core bot in real-time.
    - By using this technique, we can for example check whether the player is fighting or not.
    - Overlay for all this information.
    - Making sure that the bot is as sneaky as possible by hooking any functions that would give away the presence of the bot.

    Planned features:
    - Record your own actions in game. The bot will try to replicate it in a more random way.

    Implemented extra:
    - Extracting any 3D model in the scene and save in a simple 'OBJ' file format.
    Last edited by El Leyos; 04-14-2017 at 12:34 AM.

  2. #2
    Join Date
    Dec 2011
    Posts
    2,147
    Mentioned
    221 Post(s)
    Quoted
    1068 Post(s)

    Default

    Nice job! Is this for NXT or the Java client? Currently ogLib only functions with the Java client using a plugin created by @Brandon; - so you'd have a lot to bring to the table here if you have created hooking for NXT.

    Seems like you have solid core features, the final step you mention I actually think is the most important (remaining undetected).
    We've had endless debates around this but the most recent/relevant would be our discussion on using the hardware mouse versus injected input. Discussion here
    Last edited by Clarity; 04-13-2017 at 03:42 PM.

  3. #3
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    Thanks.
    This is for the NXT client and works out of the box, simply by running my bot. I don't want to give too much information away now, because I'm still working on some features that avoid detection.
    The big disadvantage that I can think of now is that it uses SetCursorPos. So you won't be able to use your computer while running the bot (for now).
    Another thing is that my overlay is running directly in the NXT client. I have to hook the GL functions to prevent them from requesting the resources (vao's, textures, ..) that my overlay uses.

    P.S: I have read some hardware vs injected mouse input that you linked to. I haven't given much thought about that, but that seems indeed challenging. You can probably just hook the desired function and clear that flag. I will take a good look at that later.

    EDIT: I have looked at Brandon's OpenGL hook tutorial, but I don't really understand why it couldn't be used for hooking the NXT client?
    Last edited by El Leyos; 04-13-2017 at 04:31 PM.

  4. #4
    Join Date
    Sep 2014
    Location
    C:\Simba\
    Posts
    565
    Mentioned
    9 Post(s)
    Quoted
    71 Post(s)

    Default

    NXT uses different calls than the java version so technically you just need to change up which calls to trampoline afaik.
    Feel free to ask me any questions, I will do my best to answer them!

    Previously known as YouPee.

  5. #5
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    At first, I wanted to make my own hooking plugin, but then I noticed that something very similar already existed (GLIntercept). So I used that and it's so easy that you only have to worry about giving the OpenGL calls a meaning.

    Link: github.com/dtrebilco/glintercept (Sorry, can't use direct links).

  6. #6
    Join Date
    Dec 2011
    Posts
    2,147
    Mentioned
    221 Post(s)
    Quoted
    1068 Post(s)

    Default

    Quote Originally Posted by El Leyos View Post
    EDIT: I have looked at Brandon's OpenGL hook tutorial, but I don't really understand why it couldn't be used for hooking the NXT client?
    I only meant that NXT hasn't been done here, not that it wasn't possible
    Somewhat related, people have also been very into memory reading lately, like @Ross;.

  7. #7
    Join Date
    Feb 2011
    Location
    The Future.
    Posts
    5,600
    Mentioned
    396 Post(s)
    Quoted
    1598 Post(s)

    Default

    The reason I personally won't bother with doing an NXT hook:

    NXT uses VAO for everything. You have to keep track of each and every VAO used and the buffers associated with it (by copying the buffers.. + every shader can be different).
    NXT uses VAO's for texturing as well. This means that the texturing is done through the shader now instead of the old pipeline. You'd have to check how the shader maps the texture onto a section on the screen (every shader can be different).

    Anyway, the difference between RS3 and NXT.. For example:

    NXT:
    C++ Code:
    //Single model

    glGenVertexArrays(1, &vao);  //create VAO (Vertex Array Object)
    glGenBuffers(1, &vbo);  //create VBO (Vertex Buffer Object)
    glGenBuffers(1, &nbo);  //create NBO (Normal Buffer Object)
    glGenBuffers(1, &ibo);  //create IBO (Index Buffer Object)

    glBindVertexArray(vao);  //bind the VAO so that it starts recording actions/calls.

    glEnableVertexAttribArray(0);  //Enable some shader Index (requires you to know that shader index 0 is for vertices)
    glBindBuffer(GL_ARRAY_BUFFER, vbo); //Bind the VBO and fill it with model vertices
    glBufferData(GL_ARRAY_BUFFER, vertices.size() * sizeof(vertices[0]), &vertices[0], GL_STATIC_DRAW);
    glVertexAttribPointer(0, 4, GL_FLOAT, GL_FALSE, 0, static_cast<void *>(0)); //Bind the vertices to shader index 0. 4 floats per vertex (X, Y, Z, W).
    glBindBuffer(GL_ARRAY_BUFFER, 0); //Unbind the VBO.

    glEnableVertexAttribArray(1); //Enable shader index 1 for normals.
    glBindBuffer(GL_ARRAY_BUFFER, nbo);  //Bind the NBO and fill it with normals.
    glBufferData(GL_ARRAY_BUFFER, normals.size() * sizeof(normals[0]), &normals[0], GL_STATIC_DRAW);
    glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, 0, static_cast<void *>(0)); //Bind the normals to shader index 1.. 3 floats per normal (X, Y, Z).
    glBindBuffer(GL_ARRAY_BUFFER, 0);

    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, ibo); //Bind the Index Buffer and fill it.
    glBufferData(GL_ELEMENT_ARRAY_BUFFER, indices.size() * sizeof(indices[0]), &indices[0], GL_STATIC_DRAW);
    glBindVertexArray(0); //Finished recording all calls..


    The above will only happen ONCE (the first time the model loads).. Then when RS is ready to draw the model above (every frame AFTER the first time).. it will just do:

    C++ Code:
    glBindVertexArray(vao);
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, ibo);

    glDrawElements(GL_TRIANGLES, indices.size(), GL_UNSIGNED_SHORT, static_cast<void *>(0));

    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 0);
    glBindVertexArray(0);

    The problem? One VAO can contain a lot of different models OR textures. If you don't copy the buffer upon initialization, you'd end up with a HUGE overhead of reading the buffer every time that model renders. It is harder to keep track of all of it AND keep track of each shader's IO.. You have to dis-assemble the shaders too. The whole thing is doable.. just tedious as hell.. In other words, nothing from the old plugins will work so it's a completely re-write (few weeks of work).


    So why did no one look at Direct-X? RS also uses that and it's much easier to hook..

    Anyway, GL to you OP (no pun intended). Seems you made quite the progress. Nice!
    Last edited by Brandon; 04-14-2017 at 12:22 AM.
    I am Ggzz..
    Hackintosher

  8. #8
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    That's exactly what I did. I have to keep track of all that information.
    Most OpenGL debuggers actually do the same. You can also use GL's 'get' functions to get some information instead of hooking it.

    But yeah, you are right. I agree it's tedious. Every modern game that uses OpenGL is using VAO's nowadays
    I wasn't aware that OSRS is still using the fixed pipeline though.

    And thanks.

    EDIT: Disassembling is surprisingly easy. I just hook 'glShaderSource'. They didn't tried to obfuscate it at all. All the original variable names (and even defines) are visible.

    For example:
    Code:
    attribute vec4 aVertexPosition_BoneLabel;
    attribute vec2 aTextureUV;
    attribute vec4 aVertexNormal_BatchFlags;
    attribute vec2 aSpecular_NormalScale;
    attribute vec4 aVertexColour;
    attribute vec4 aVertexColourUnwhitened;
    attribute vec4 aTextureMeta;
    attribute vec4 aSamplerWrap_TilePosition;
    Last edited by El Leyos; 04-14-2017 at 12:29 AM.

  9. #9
    Join Date
    Dec 2010
    Posts
    483
    Mentioned
    30 Post(s)
    Quoted
    328 Post(s)

    Default

    Quote Originally Posted by Brandon View Post
    So why did no one look at Direct-X? RS also uses that and it's much easier to hook..
    As far as I am aware, the DirectX implementation in NXT (aka "Compatibility Mode") is far worse than the DirectX implementation originally available in the Java client. I believe its just Microsoft's "ANGLE" implementation for converting OpenGL calls into DirectX calls. NXT is for all purposes, an OpenGL client.

    https://github.com/Microsoft/angle

    http://services.runescape.com/l=0/m=...r=Mod%A0Harker

  10. #10
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    It seems I can only answer in short posts or they get eternally queued for moderation.

  11. #11
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    @thebank: Can't respond to your PM's (too little posts). Tried to get it fixed, to no avail.

  12. #12
    Join Date
    Nov 2011
    Location
    England
    Posts
    3,072
    Mentioned
    296 Post(s)
    Quoted
    1094 Post(s)

    Default

    Quote Originally Posted by El Leyos View Post
    @thebank: Can't respond to your PM's (too little posts). Tried to get it fixed, to no avail.
    Reply this this three more times

  13. #13
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    The price good men pay for indifference to public affairs is to be ruled by evil men.

  14. #14
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    The heaviest penalty for declining to rule is to be ruled by someone inferior to yourself.

  15. #15
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    I am the wisest man alive, for I know one thing, and that is that I know nothing.

  16. #16
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    Quote Originally Posted by the bank View Post
    As far as I am aware, the DirectX implementation in NXT (aka "Compatibility Mode") is far worse than the DirectX implementation originally available in the Java client. I believe its just Microsoft's "ANGLE" implementation for converting OpenGL calls into DirectX calls. NXT is for all purposes, an OpenGL client.

    https://github.com/Microsoft/angle

    http://services.runescape.com/l=0/m=...r=Mod%A0Harker
    Yes, I read it somewhere that they convert the OpenGL calls to DirectX. I'm not sure what implementation they are using though.

  17. #17
    Join Date
    Dec 2010
    Posts
    483
    Mentioned
    30 Post(s)
    Quoted
    328 Post(s)

    Default

    It is true that liberty is precious; so precious that it must be carefully rationed.

  18. #18
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    Quote Originally Posted by the bank View Post
    It is true that liberty is precious; so precious that it must be carefully rationed.
    I'm not really into Lenin.

  19. #19
    Join Date
    Mar 2007
    Posts
    5,125
    Mentioned
    275 Post(s)
    Quoted
    901 Post(s)

    Default

    Quote Originally Posted by El Leyos View Post
    It seems I can only answer in short posts or they get eternally queued for moderation.
    Fixed.

    Quote Originally Posted by Olly View Post
    Reply this this three more times
    The public only saw 1 post, you're seeing the 'Moderated Post' from the spam filter because of your rank(s)

    Forum account issues? Please send me a PM

  20. #20
    Join Date
    Dec 2016
    Posts
    31
    Mentioned
    0 Post(s)
    Quoted
    8 Post(s)

    Default

    Looking forward to seeing this publicly released. Curious to see how well it works.

  21. #21
    Join Date
    Apr 2017
    Posts
    11
    Mentioned
    1 Post(s)
    Quoted
    5 Post(s)

    Default

    Good work. How long have you been working on it?

    I have a few questions. Models can require multiple draw calls when 1) the model uses multiple textures 2) the textures are on different atlases. The data in the underlying vertex buffers is not deterministic as a result. Does your model matching work when draw splits occur? Or does this not happen with textures disabled? (This may be different with the newer NXT updates, I can't be certain--my testing was done in December).

    Similarly, do you support batched models (e.g., cooking ranges, trees, scenery)? If so, how do you handle the non-deterministic transforms applied to these objects?

  22. #22
    Join Date
    Apr 2017
    Location
    Panama
    Posts
    14
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    Quote Originally Posted by Kompromaus View Post
    Good work. How long have you been working on it?

    I have a few questions. Models can require multiple draw calls when 1) the model uses multiple textures 2) the textures are on different atlases. The data in the underlying vertex buffers is not deterministic as a result. Does your model matching work when draw splits occur? Or does this not happen with textures disabled? (This may be different with the newer NXT updates, I can't be certain--my testing was done in December).

    Similarly, do you support batched models (e.g., cooking ranges, trees, scenery)? If so, how do you handle the non-deterministic transforms applied to these objects?
    Almost a month now.
    In general, models can be rendered with multiple seperate textures or as a single texture atlas using one drawcall.
    I am not sure what you mean with 'draw splits'.
    So far I haven't noticed anything that would reveal some sort of dynamic or static batching. Transformations on the models are done in the shader of each GL program (like any modern game).

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •