Results 1 to 12 of 12

Thread: Kill the Spiders!!

  1. #1
    Join Date
    Dec 2006
    Location
    Sweden
    Posts
    10,812
    Mentioned
    3 Post(s)
    Quoted
    16 Post(s)

    Default Kill the Spiders!!

    There is often 14, or ever sometimes more, web spiders indexing SRL Forums constantly. Why does SRL not just set a robots.txt file in their main directory and block all webspiders except for the Adsence bot for the ads? Each one of those spiders make SRL use more bandwidth, making SRL cost more per month.

    Google Spider
    Viewing Thread
    What does 'tut' mean?

    03:05 PM Google AdSense Spider
    Viewing Thread
    FrugooScape Autoclicker

    02:53 PM Yahoo! Slurp Spider
    Viewing User Profile

    03:05 PM Google Spider
    Viewing Index
    SRL-Forums

    03:05 PM Google Spider
    Viewing Thread
    Dodian Script Pack

    03:05 PM Google Spider
    Viewing Thread
    VERY easy way to add smart

    02:57 PM Google Spider
    Viewing Index
    SRL-Forums

    03:04 PM Google Spider
    Viewing Index
    SRL-Forums

    03:05 PM Google Spider
    Viewing Index
    SRL-Forums

    02:56 PM Google AdSense Spider
    Viewing Thread
    How much GP have you made from autoing? (staking is allowed, too)

    03:01 PM Google Spider
    Viewing Index
    SRL-Forums

    03:04 PM Google AdSense Spider
    Viewing Index
    SRL-Forums
    That's a list I got of them. Isn't that a great waste of SRL's bandwidth??

    -Discuss


    Send SMS messages using Simba
    Please do not send me a PM asking for help; I will not be able to help you! Post in a relevant thread or make your own! And always remember to search first!

  2. #2
    Join Date
    Feb 2006
    Location
    Belgium
    Posts
    3,137
    Mentioned
    3 Post(s)
    Quoted
    5 Post(s)

    Default

    No waste...
    No spiders, no indexing...

  3. #3
    Join Date
    Dec 2006
    Location
    Sweden
    Posts
    10,812
    Mentioned
    3 Post(s)
    Quoted
    16 Post(s)

    Default

    Why must we be indexed?


    Send SMS messages using Simba
    Please do not send me a PM asking for help; I will not be able to help you! Post in a relevant thread or make your own! And always remember to search first!

  4. #4
    Join Date
    Jun 2007
    Location
    Wednesday
    Posts
    2,446
    Mentioned
    3 Post(s)
    Quoted
    1 Post(s)

    Default

    Indexing increases chance of someone coming to SRL from a search engine. Not only that, but they use less bandwidth of one user, which is near nothing as is, due to no downloading files and just pages, so they don't affect the site negatively at all.
    By reading this signature you agree that mixster is superior to you in each and every way except the bad ways but including the really bad ways.

  5. #5
    Join Date
    Jun 2006
    Posts
    3,861
    Mentioned
    3 Post(s)
    Quoted
    1 Post(s)

    Default

    Freddy, I thought I have heard the admins say that they don't want people finding SRL through search engines. Though this was more than a year ago, so their views may have changed (assuming I am right about them once opposing it).

  6. #6
    Join Date
    Mar 2008
    Location
    The Netherlands
    Posts
    1,395
    Mentioned
    1 Post(s)
    Quoted
    1 Post(s)

    Default

    I found a nice quote from 2 years ago

    Quote Originally Posted by Flyboy View Post
    SRL is about to hit the streets. I can already detect the sound of countless feet tromping this direction! Good times are ahead I would think!

    Welcome aboard everyone. Buckle up and hang on, cause we are going for a ride of a lifetime!!!
    That's for sure. But in a topic like that (can't find it anymore ) they want SRL to be kept private, excluded from Google. Although, I don't know what their statements are now


  7. #7
    Join Date
    Oct 2007
    Location
    The deep web
    Posts
    2,496
    Mentioned
    1 Post(s)
    Quoted
    3 Post(s)

    Default

    more people = more scripts, but more leechers and scammers.
    Did someone say GDK?

  8. #8
    Join Date
    Mar 2008
    Posts
    1,492
    Mentioned
    0 Post(s)
    Quoted
    0 Post(s)

    Default

    I think ya'll have gotten a little off topic... Where did you find this Hy?

  9. #9
    Join Date
    Nov 2006
    Location
    Wisconsin
    Posts
    1,629
    Mentioned
    0 Post(s)
    Quoted
    3 Post(s)

    Default

    Well without the indexing, wouldn't we not be cached in google?


    Quote Originally Posted by Rubix View Post
    Quote Originally Posted by Dan Cardin View Post
    you ought to listen to Mr. Klean...he's magical!
    this.

  10. #10
    Join Date
    Mar 2006
    Posts
    13,241
    Mentioned
    228 Post(s)
    Quoted
    267 Post(s)

    Default

    Quote Originally Posted by laurenssie0 View Post
    I found a nice quote from 2 years ago



    That's for sure. But in a topic like that (can't find it anymore ) they want SRL to be kept private, excluded from Google. Although, I don't know what their statements are now
    That is from when SRL was about to go public. We used to be a private community as you can tell. Then Fakawi's wifey got sick and he had to leave for awhile, thats when it went public. Before it was, there was no way to "Stumble on SRL" either you knew it or you didnt, or someone told you about it. Cause you had to use the actual IP address of the server to get here.
    STOP PM'ING ME

  11. #11
    Join Date
    May 2008
    Posts
    15
    Mentioned
    0 Post(s)
    Quoted
    0 Post(s)

    Default

    i dont notice any lag....now if the site were really inresponsive (to the point of taking average 20 sec. a page to load), that would be differant.

  12. #12
    Join Date
    Feb 2008
    Location
    Norway
    Posts
    278
    Mentioned
    0 Post(s)
    Quoted
    0 Post(s)

    Default

    I do only notice lag when I'm postin a post (right now ) It's worth a bit of lag that more people do come to SRL.
    99 woodcutting

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. How can I kill some plants?
    By Brain in forum News and General
    Replies: 114
    Last Post: 04-15-2009, 11:07 AM
  2. Halo Kill Bot
    By Aser in forum OSR Help
    Replies: 18
    Last Post: 03-05-2009, 05:03 PM
  3. I am going to kill Jagex.
    By Harry in forum The Bashing Club / BBQ Pit
    Replies: 59
    Last Post: 07-30-2007, 06:46 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •