Results 1 to 4 of 4

Thread: Python Proxy Tester

  1. #1
    Join Date
    Jan 2008
    Location
    NC, USA.
    Posts
    4,429
    Mentioned
    0 Post(s)
    Quoted
    4 Post(s)

    Default Python Proxy Tester

    To use, put the source in a .py and in the same folder include proxies_1.txt (an example). There can be multiple proxies_X.txt files (X being numbers).
    Also include a working_proxies.txt in the same directory. It should look something like this, for those of you that cannot read and or understand the directions.


    You can specify the amount of threads to use when calling the script, or after running the script.

    For example: `python proxy_tester.py 6', or `python proxy_tester.py' it will then ask for how many threads you would like to use.

    Example files are attached, source code is below.
    Edit: Yea well it looks like you can attach .py files, so I'll harass Wizzup about that.

    python Code:
    # To change this template, choose Tools | Templates
    # and open the template in the editor.

    __author__="lucas"
    __date__ ="$Nov 14, 2010 12:14:30 PM$"

    import math
    import os
    import os.path
    import urllib2
    import threading
    import sys

    global good_list
    good_list = []


    class Proxish(threading.Thread):
        def __init__(self, proxy_list, threadnum, timeout=5):
            threading.Thread.__init__(self)
            self.good_list = []
            self.thread_number = threadnum
            self.timeout = timeout
            self.good_list = []
            self.list = proxy_list

        def verify_list(self, proxy_list):
            working_list = []
            for prox in proxy_list:
                try:
                    proxy = urllib2.ProxyHandler({'http':'http://'+prox+'/'})
                    opener = urllib2.build_opener(proxy)
                    site_code = opener.open("http://ipchicken.com/", timeout=self.timeout).readlines()
                    ip = site_code[site_code.index('      <p align="center"><font face="Verdana, Arial, Helvetica, sans-serif" size="5" color="#0000FF"><b> \r\n')+1].strip(' <br>\r\n')
                    print '[Thread:', self.thread_number, '] Current IP:', ip
                    print '[Thread:', self.thread_number, '] Proxy works:', prox
                    print '[Thread:', self.thread_number, '] match:', True if ip == prox.split(':')[0] else False
                    working_list.append(prox)
                except Exception, e:#urllib2.URLError:
                    print '[Thread:', self.thread_number, '] Proxy failed', prox
                    print '[Thread:', self.thread_number, '] Proxy failed', e
            print '[Thread:', self.thread_number, '] Working Proxies:', working_list
            return working_list

        def run(self):
            global good_list
            good_list += self.verify_list(self.list)
            print '[All       ] Working Proxies:', good_list
            f = open('working_proxies.txt', 'r+')
            to_write = ''
            for i in good_list:
                to_write+= i+'\n'
            f.write(to_write)
            f.close()


    def get_proxy_list():
        #get the directory
        file = os.path.abspath(__file__)
        for i in sorted(range(len(file)), reverse=True):
            if '/' in file[i] or '\\' in file[i]:
                directory = file[:i+1]
                break
        file_list = os.listdir(directory)
        proxy_list = []
        for file in file_list:
            if len(file) > 12:
                if file[:8] == 'proxies_':
                    if not file[len(file)-1]== '~': # Gedit kept making temp files and it would read those
                        proxy_list.append(directory+file)
        return proxy_list

    def get_proxies(files):
        proxy_list = []
        for file in files:
            for prox in open(file, 'r+').readlines():
                proxy_list.append(prox.strip())
        return proxy_list

    def setup(number_threads):
        thread_amount = float(number_threads) # how many threads do you want the program to have?
        proxy_list = get_proxies(get_proxy_list()) # load the lists
        # get the amount to div the list into
        amount = int(math.ceil(len(proxy_list)/thread_amount))
        # split the large list into multiple smaller lists
        proxy_lists = [proxy_list[x:x+amount] for x in xrange(0, len(proxy_list), amount)]
        # if there is one left over, add ito the end of the last list
        if len(proxy_list) % thread_amount > 0.0:
            proxy_lists[len(proxy_lists)-1].append(proxy_list[len(proxy_list)-1])
        return proxy_lists

    def start(threads):
        lists = setup(threads)
        thread_list = []
        good_list = []
        count = 0
        for list in lists:
            thread_list.append(Proxish(list, count))
            thread_list[len(thread_list)-1].start() # the above creates, but this starts
            count+=1

    if __name__ == "__main__":
        if len(sys.argv) > 1:
            start(sys.argv[1])
        else:
            start(raw_input('How many threads you would like to test proxies with: '))
    This is what it looks like while working.


    This is what it looks like finished and after writing the good proxies to working_proxies.txt

    Last edited by noidea; 11-21-2010 at 03:18 PM.
    Quote Originally Posted by irc
    [00:55:29] < Guest3097> I lol at how BenLand100 has become noidea
    [01:07:40] <@BenLand100> i'm not noidea i'm
    [01:07:44] -!- BenLand100 is now known as BenLand42-
    [01:07:46] <@BenLand42-> shit
    [01:07:49] -!- BenLand42- is now known as BenLand420
    [01:07:50] <@BenLand420> YEA

  2. #2
    Join Date
    Mar 2009
    Location
    North
    Posts
    513
    Mentioned
    0 Post(s)
    Quoted
    0 Post(s)

    Default

    so this is the thing you told me you working on and i honestly had no idea wtf you were talking about...

    :P

    It always confuses me why people go into wars about how we got here, and where we go when we die.

  3. #3
    Join Date
    Nov 2007
    Location
    46696E6C616E64
    Posts
    3,069
    Mentioned
    44 Post(s)
    Quoted
    302 Post(s)

    Default

    You should take a look at NiX Proxy Checkers features, found at http://myproxylists.com/. Thought its coded with PHP + ZSH. Its pretty neat tool. I don't understand much python, but looks cool.
    There used to be something meaningful here.

  4. #4
    Join Date
    Jan 2008
    Location
    NC, USA.
    Posts
    4,429
    Mentioned
    0 Post(s)
    Quoted
    4 Post(s)

    Default

    At the top of that site it says (based on all proxy checks in the past 8 hours).
    So if you've got a large list of files and want to test them quickly, use the script
    Using many threads works best with a multicore cpu. I blasted through >3k proxies in less than an hour.

    Update soon: forgot to add in the option of changing the timeout parameter when starting when clearly the code supports it.
    Quote Originally Posted by irc
    [00:55:29] < Guest3097> I lol at how BenLand100 has become noidea
    [01:07:40] <@BenLand100> i'm not noidea i'm
    [01:07:44] -!- BenLand100 is now known as BenLand42-
    [01:07:46] <@BenLand42-> shit
    [01:07:49] -!- BenLand42- is now known as BenLand420
    [01:07:50] <@BenLand420> YEA

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •