Results 1 to 10 of 10

Thread: Reading off a web page:

  1. #1
    Join Date
    Aug 2010
    Posts
    122
    Mentioned
    0 Post(s)
    Quoted
    7 Post(s)

    Default Reading off a web page:

    I would like to have the bot access something like a CSV on a webpage, read the strings, and use as vars in a script.

    The page could be any type, and the only data there could be the vars.

    Possible to do this? If so, how? Any chance someone can maybe link me to a guide on exploding CSV as well?

    Any help is appreciated, thank you

  2. #2
    Join Date
    Feb 2011
    Location
    The Future.
    Posts
    5,600
    Mentioned
    396 Post(s)
    Quoted
    1598 Post(s)

    Default

    At least give an example page. It's hard to imagine what you want here as it isn't clear to me how you want to use this data as vars :S
    I am Ggzz..
    Hackintosher

  3. #3
    Join Date
    Mar 2007
    Location
    USA
    Posts
    1,433
    Mentioned
    1 Post(s)
    Quoted
    0 Post(s)

    Default

    Quote Originally Posted by Mister Snow View Post
    I would like to have the bot access something like a CSV on a webpage, read the strings, and use as vars in a script.

    The page could be any type, and the only data there could be the vars.

    Possible to do this? If so, how? Any chance someone can maybe link me to a guide on exploding CSV as well?

    Any help is appreciated, thank you
    Have a look at this:

    http://docs.villavu.com/simba/scriptref/web.html
    I'm Silent SPY
    Secret project: 0%
    Need help? Send me a PM

  4. #4
    Join Date
    Nov 2007
    Location
    46696E6C616E64
    Posts
    3,069
    Mentioned
    44 Post(s)
    Quoted
    302 Post(s)

    Default

    Wrote this for you, hope it helps:
    Simba Code:
    program CSVSnippet;

    type TCSVObject = record
      Header: TStringArray;
      Items: Array of TStringArray;
    end;

    const Delimeter = ';';
          LF = #10;

    function CSVParse(Data: String; WithHeaders: Boolean): TCSVObject;
    var ExplodedData, ExplodedSubData, HeaderData: TStringArray;
        I, J, K: Integer;
    begin
      ExplodedData := Explode(LF, Data);
      if (WithHeaders) then begin
        HeaderData := Explode(Delimeter, ExplodedData[0]);
        for I := 0 to High(HeaderData) do begin
          HeaderData[I] := Replace(HeaderData[I], '"', '', [rfReplaceAll]);
        end;
        Result.Header := HeaderData;
        K := 1;
      end else begin
        K := 0;
      end;
      SetLength(Result.Items, High(ExplodedData));
      for I := K to High(ExplodedData) do begin
        ExplodedSubData := Explode(Delimeter, ExplodedData[I]);
        for J := 0 to High(ExplodedSubData) do begin
          ExplodedSubData[J] := Replace(ExplodedSubData[J], '"', '', [rfReplaceAll]);
        end;
        Result.Items[I - 1] := ExplodedSubData;
      end;
    end;

    function CSVGetHeaderIndex(CSVObject: TCSVObject; Needle: String): Integer;
    var I: Integer;
    begin
      for I := 0 to High(CSVObject.Header) do begin
        if (CSVObject.Header[I] = Needle) then begin
          Result := I;
          Exit;
        end;
      end;
    end;

    function CSVFind(CSVObject: TCSVObject; Needle: String): Integer;
    var I, J: Integer;
    begin
      for I := 0 to High(CSVObject.Items) do begin
        for J := 0 to High(CSVObject.Items[I]) do begin
          if (CSVObject.Items[I][J] = Needle) then
            Result := I;
            Exit;
        end;
      end;
    end;

    function CSVGet(CSVObject: TCSVObject; Header: String): TStringArray;
    var I, Index: Integer;
    begin
      Index := CSVGetHeaderIndex(CSVObject, Header);
      SetLength(Result, High(CSVObject.Items) + 1);
      for I := 0 to High(CSVObject.Items) do begin
        Result[I] := CSVObject.Items[I][Index];
      end;
    end;

    var HTTPClient: Integer;
        Data: String;
        CSVObject: TCSVObject;

    begin
      // Initialize our HTTP client.
      HTTPClient := InitializeHTTPClient(False);

      // Get the CSV file as string.
      Data := GetHTTPPage(HTTPClient, 'http://static.frement.net/srl/example.csv');

      // Pass the retrieved CSV data to our function.
      // (Note: the example has headers, so WithHeaders has to be True,
      //  else it will treat headers as actual item)
      CSVObject := CSVParse(Data, True);

      // Output the whole object.
      Writeln(ToStr(CSVObject));

      // Find a specific item with any value in the item. (Note: will return the first found)
      Writeln(ToStr(CSVObject.Items[CSVFind(CSVObject, '00001')]));

      // Get all values from the object by header.
      Writeln(ToStr(CSVGet(CSVObject, 'Addresse')));

      // Free our HTTP client.
      FreeHTTPClient(HTTPClient);
    end.
    There used to be something meaningful here.

  5. #5
    Join Date
    Aug 2010
    Posts
    122
    Mentioned
    0 Post(s)
    Quoted
    7 Post(s)

    Default

    Quote Originally Posted by Brandon View Post
    At least give an example page. It's hard to imagine what you want here as it isn't clear to me how you want to use this data as vars :S
    Hey, sorry I was not clear enough... it was kind of meant that way. There simply is no page yet. I would make the page custom, so the sky is the limit there. The page could be in whatever way to make it easiest and most compatible with Simba.



    Quote Originally Posted by Frement View Post
    Wrote this for you, hope it helps:
    Simba Code:
    program CSVSnippet;

    type TCSVObject = record
      Header: TStringArray;
      Items: Array of TStringArray;
    end;

    const Delimeter = ';';
          LF = #10;

    function CSVParse(Data: String; WithHeaders: Boolean): TCSVObject;
    var ExplodedData, ExplodedSubData, HeaderData: TStringArray;
        I, J, K: Integer;
    begin
      ExplodedData := Explode(LF, Data);
      if (WithHeaders) then begin
        HeaderData := Explode(Delimeter, ExplodedData[0]);
        for I := 0 to High(HeaderData) do begin
          HeaderData[I] := Replace(HeaderData[I], '"', '', [rfReplaceAll]);
        end;
        Result.Header := HeaderData;
        K := 1;
      end else begin
        K := 0;
      end;
      SetLength(Result.Items, High(ExplodedData));
      for I := K to High(ExplodedData) do begin
        ExplodedSubData := Explode(Delimeter, ExplodedData[I]);
        for J := 0 to High(ExplodedSubData) do begin
          ExplodedSubData[J] := Replace(ExplodedSubData[J], '"', '', [rfReplaceAll]);
        end;
        Result.Items[I - 1] := ExplodedSubData;
      end;
    end;

    function CSVGetHeaderIndex(CSVObject: TCSVObject; Needle: String): Integer;
    var I: Integer;
    begin
      for I := 0 to High(CSVObject.Header) do begin
        if (CSVObject.Header[I] = Needle) then begin
          Result := I;
          Exit;
        end;
      end;
    end;

    function CSVFind(CSVObject: TCSVObject; Needle: String): Integer;
    var I, J: Integer;
    begin
      for I := 0 to High(CSVObject.Items) do begin
        for J := 0 to High(CSVObject.Items[I]) do begin
          if (CSVObject.Items[I][J] = Needle) then
            Result := I;
            Exit;
        end;
      end;
    end;

    function CSVGet(CSVObject: TCSVObject; Header: String): TStringArray;
    var I, Index: Integer;
    begin
      Index := CSVGetHeaderIndex(CSVObject, Header);
      SetLength(Result, High(CSVObject.Items) + 1);
      for I := 0 to High(CSVObject.Items) do begin
        Result[I] := CSVObject.Items[I][Index];
      end;
    end;

    var HTTPClient: Integer;
        Data: String;
        CSVObject: TCSVObject;

    begin
      // Initialize our HTTP client.
      HTTPClient := InitializeHTTPClient(False);

      // Get the CSV file as string.
      Data := GetHTTPPage(HTTPClient, 'http://static.frement.net/srl/example.csv');

      // Pass the retrieved CSV data to our function.
      // (Note: the example has headers, so WithHeaders has to be True,
      //  else it will treat headers as actual item)
      CSVObject := CSVParse(Data, True);

      // Output the whole object.
      Writeln(ToStr(CSVObject));

      // Find a specific item with any value in the item. (Note: will return the first found)
      Writeln(ToStr(CSVObject.Items[CSVFind(CSVObject, '00001')]));

      // Get all values from the object by header.
      Writeln(ToStr(CSVGet(CSVObject, 'Addresse')));

      // Free our HTTP client.
      FreeHTTPClient(HTTPClient);
    end.
    Awesome! Thank you so much +rep.

    Looks to my eyes like it just might work. Time to mess around with it.

  6. #6
    Join Date
    Oct 2008
    Location
    C:\Simba\Includes\
    Posts
    7,566
    Mentioned
    19 Post(s)
    Quoted
    180 Post(s)

    Default

    Why use a CSV? They can get corrupted easily, use a database
    Away for awhile, life is keeping me busy. | Want to get my attention in a thread? @Kyle Undefined; me.
    { MSI Phoenix || SRL Stats Sigs || Paste || Scripts || Quotes || Graphics }

    When posting a bug, please post debug! Help us, help you!

    I would love to change the world, but they won't give me the source code. || To be the best, you've got to beat the rest. || Logic never changes, just the syntax.
    If you PM me with a stupid question or one listed in FAQ, or about a script that is not mine, I will NOT respond.


    SRL is a Library of routines made by the SRL community written for the Program Simba. We produce Scripts for the game Runescape.


  7. #7
    Join Date
    Mar 2011
    Location
    Broomfield
    Posts
    57
    Mentioned
    0 Post(s)
    Quoted
    0 Post(s)

    Default

    Get cygwin if you are on windows, or install a Linux VM.
    http://www.cygwin.com/

    If you are on linux or a mac do this in a terminal.

    -Learn basic awk and regex's (Regular Expressions)

    -Use lynx dump to get the page and then use awk to parse the code.

    #!/usr/bin/bash

    lynx -dump -width 200 http://www.villavu.com/ | sed 's/\[[0-9]*\]//g' | awk "/has been released/"'{print}'
    That code will fetch the friendly message of what version of Simba has been released.

    If you want to loop it so that it it fetches automaticaly use a loop like this...

    while true
    do

    lynx -dump -width 200 http://www.villavu.com/ | awk "/has been released/"'{print}'

    sleep 15; clear; done
    If you need regex help let me know. These tools are limitless and far cleaner than Pascal for what i think you are trying to do.
    Last edited by activecamo; 07-25-2012 at 12:31 PM.
    I cant have a fancy sig, so click for a laugh that I made. http://i55.photobucket.com/albums/g1...er/Punched.gif

  8. #8
    Join Date
    Mar 2006
    Location
    USA
    Posts
    948
    Mentioned
    0 Post(s)
    Quoted
    1 Post(s)

    Default

    Quote Originally Posted by activecamo View Post
    Get cygwin if you are on windows, or install a Linux VM.
    http://www.cygwin.com/

    If you are on linux or a mac do this in a terminal.

    -Learn basic awk and regex's (Regular Expressions)

    -Use lynx dump to get the page and then use awk to parse the code.



    That code will fetch the friendly message of what version of Simba has been released.

    If you want to loop it so that it it fetches automaticaly use a loop like this...



    If you need regex help let me know. These tools are limitless and far cleaner than Pascal for what i think you are trying to do.
    I'm a big fan of linux and open source myself, but this hardly seems to be the simplest solution available.

  9. #9
    Join Date
    Mar 2011
    Location
    Broomfield
    Posts
    57
    Mentioned
    0 Post(s)
    Quoted
    0 Post(s)

    Default

    Quote Originally Posted by Bam Bam View Post
    I'm a big fan of linux and open source myself, but this hardly seems to be the simplest solution available.
    Give me a database and I will sed and awk the hell out of it for you. Simplicity could also based on the familiarity of the language. Since I actually work mostly in BASH its very easy for me.
    I cant have a fancy sig, so click for a laugh that I made. http://i55.photobucket.com/albums/g1...er/Punched.gif

  10. #10
    Join Date
    Nov 2007
    Location
    46696E6C616E64
    Posts
    3,069
    Mentioned
    44 Post(s)
    Quoted
    302 Post(s)

    Default

    Quote Originally Posted by activecamo View Post
    Give me a database and I will sed and awk the hell out of it for you. Simplicity could also based on the familiarity of the language. Since I actually work mostly in BASH its very easy for me.
    Given that a user must request help on this matter suggests that he/she might not want that solution, it is not the best nor the easiest way to do it. And as we basically work around Simba, it's highly unlikely that using other languages/methods would be acceptable.

    While there are other solutions, I'm sure he'll be fine with what he has now.

    But don't take this the wrong way, helping is always appreciated
    There used to be something meaningful here.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •