Results 1 to 9 of 9

Thread: PHP bot/spider help!

  1. #1
    Join Date
    Oct 2014
    Location
    Europe
    Posts
    98
    Mentioned
    4 Post(s)
    Quoted
    55 Post(s)

    Default PHP bot/spider help!

    I need to write a basic bot in PHP, which collects random urls from the internet to a database. Basically I need a bunch of random webpages. Any help would be appreciated! (I need it for work in a few weeks :/)

  2. #2
    Join Date
    Jan 2012
    Posts
    190
    Mentioned
    0 Post(s)
    Quoted
    10 Post(s)

    Default

    Quote Originally Posted by Torol View Post
    I need to write a basic bot in PHP, which collects random urls from the internet to a database. Basically I need a bunch of random webpages. Any help would be appreciated! (I need it for work in a few weeks :/)

    http://stackoverflow.com/questions/8...eb-page-in-php

  3. #3
    Join Date
    Feb 2007
    Location
    Colorado, USA
    Posts
    3,716
    Mentioned
    51 Post(s)
    Quoted
    624 Post(s)

    Default

    so you want a web crawler to grab random URL's from random sites?
    or you want a PHP script to grab all links off of certain pages?
    The only true authority stems from knowledge, not from position.

    You can contact me via matrix protocol: @grats:grats.win or you can email me at the same domain, any user/email address.

  4. #4
    Join Date
    Oct 2014
    Location
    Europe
    Posts
    98
    Mentioned
    4 Post(s)
    Quoted
    55 Post(s)

    Default

    Quote Originally Posted by m3gaman3g3nd View Post
    thanks!

    Quote Originally Posted by grats View Post
    so you want a web crawler to grab random URL's from random sites?
    or you want a PHP script to grab all links off of certain pages?
    yes, I want a web crawler

  5. #5
    Join Date
    Feb 2007
    Location
    Colorado, USA
    Posts
    3,716
    Mentioned
    51 Post(s)
    Quoted
    624 Post(s)

    Default

    Quote Originally Posted by Torol View Post
    thanks!



    yes, I want a web crawler
    http://phpcrawl.cuab.de/
    The only true authority stems from knowledge, not from position.

    You can contact me via matrix protocol: @grats:grats.win or you can email me at the same domain, any user/email address.

  6. #6
    Join Date
    Jan 2012
    Posts
    190
    Mentioned
    0 Post(s)
    Quoted
    10 Post(s)

    Default

    You can also use VBSCRIPT with InternetExplorerApplication as well as SIMBA and or SCAR! they use gethttp and can easily read and grab links from text!

  7. #7
    Join Date
    Oct 2014
    Location
    Europe
    Posts
    98
    Mentioned
    4 Post(s)
    Quoted
    55 Post(s)

    Default

    Quote Originally Posted by grats View Post
    Thanks, I checked it out!

    Quote Originally Posted by m3gaman3g3nd View Post
    You can also use VBSCRIPT with InternetExplorerApplication as well as SIMBA and or SCAR! they use gethttp and can easily read and grab links from text!
    Interesting, I'm familiar with vb but not vbscript, but I already started it in php so I will stick with it I think.

  8. #8
    Join Date
    Jan 2012
    Posts
    190
    Mentioned
    0 Post(s)
    Quoted
    10 Post(s)

    Default

    Code:
    Set xmlhttp = CreateObject("MSXML2.ServerXMLHTTP") 
    Set oWS = CreateObject("Shell.Application") 
    xmlhttp.open "GET","http://CODEGURU.COM", false 
    xmlhttp.send "" 
    xmlhttp.waitForResponse() 
    if xmlhttp.status = 200 then
        oWS.ShellExecute "TEXTSEARCH.BAT",CHR(34)&xmlhttp.responseText&CHR(34)
    else 
        WScript.Echo("bad response") 
    end if 
    Set xmlhttp = nothing 
    Set oWS = nothing
    WHEN TEXTSEARCH.BAT WILL LOOK LIKE
    Code:
    @ECHO OFF
    SET "HTML="%1""
    ECHO "%HTML%"
    PAUSE
    THOUGH BATCH CAPSULATING THE HTML IS TRIVIAL - THIS WILL OBVIOUSLY NOT WORK BUT IF IT DOES THEN ITS THE IDEA

  9. #9
    Join Date
    Feb 2013
    Posts
    89
    Mentioned
    0 Post(s)
    Quoted
    48 Post(s)

    Default

    I've made a crawler in php for school tring to find the code atm.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •