Metadata-Version: 1.0
Name: collective.anonymousbrowser
Version: 0.5dev-r79038
Summary: A zope.testbrowser extension with useragent faking and proxy abilities
Home-page: https://svn.plone.org/svn/collective/collective.anonymousbrowser/trunk
Author: Mathieu Pasquet
Author-email: kiorky@cryptelium.net
License: GPL
Description: Introduction
        =============
        This Yet-Another-Mechanize implementation aims to give the developper those new
        features:
        
        - It can be proxified
        - It fakes user agent ``by default``
        - It does not handle robots ``by default``
        - There is a 'real" modification which uses an underlying moz repl server to
        control a distance firefox instance
        
        It uses sys.prefix/etc/config.ini with a part [collective.anonymousbrowser] for its settings::
        
        [collective.anonymousbrowser]
        proxies=
        ; for a mozrepl server
        host = localhost
        port = 4242
        
        This file is generated at the first run without proxies. It s your own to feed it with some open proxies.
        
        Of course, it can take another configuration file, please see the __init__ method.
        
        TODO
        =====
        - lxml integration, maybe steal z3c.etestbrowser
        
        
        Tests and Handbook
        ======================
        
        First, we need to instantiate the sources where we come from::
        
        >>> import BaseHTTPServer
        >>> from SimpleHTTPServer import SimpleHTTPRequestHandler
        >>> from collective.anonymousbrowser.browser import Browser, FF2_USERAGENT
        >>> from threading import Thread
        
        Run a basic request printers to check user agent and further requests::
        
        >>> class ReqHandler(SimpleHTTPRequestHandler):
        ...     def do_GET(self):
        ...         self.end_headers()
        ...         self.send_response(200, '\n\n<html>%s</html>' % self.headers)
        >>> httpd0 =  BaseHTTPServer.HTTPServer(('', 45675,) , ReqHandler)
        >>> httpd1 =  BaseHTTPServer.HTTPServer(('', 45676,) , ReqHandler)
        >>> httpd2 =  BaseHTTPServer.HTTPServer(('', 45677,) , ReqHandler)
        >>> httpd3 =  BaseHTTPServer.HTTPServer(('', 45678,) , ReqHandler)
        >>> httpd4 =  BaseHTTPServer.HTTPServer(('', 45679,) , ReqHandler)
        >>> for item in (httpd0, httpd1, httpd2, httpd3, httpd4):
        ...      t = Thread(target=item.serve_forever)
        ...      t.setDaemon(True)
        ...      t.start()
        
        User Agent
        ------------
        Oh, my god, we have a brand new user agent by default::
        
        >>> br = Browser()
        >>> br.open('http://localhost:45678')
        >>> FF2_USERAGENT in br.contents
        True
        >>> br2 = Browser('http://localhost:45678')
        >>> FF2_USERAGENT in br2.contents
        True
        
        
        Proxy mode
        ------------
        But, we want to be anonymous, and we ll set a proxy
        To define those proxies, just just a config.ini file like::
        
        [collective.anonymousbrowser]
        proxies =
        host1:port
        host2:port
        
        When the browser has many proxies defined, it will circly through those ones.
        But, it will not use the same host indefinitivly, just set the ``proxy_max_use`` argument::
        
        >>> from StringIO import StringIO
        >>> from tempfile import mkstemp
        >>> __, config = mkstemp()
        >>> open(config, 'w').write("""[collective.anonymousbrowser]
        ... proxies =
        ...     127.0.0.1:45675
        ...     127.0.0.1:45676
        ...     127.0.0.1:45677
        ...     127.0.0.1:45678
        ...     127.0.0.1:45679
        ...     """)
        >>> b = Browser(config=config)
        >>> b._config._sections
        {'collective.anonymousbrowser': {'__name__': 'collective.anonymousbrowser', 'proxies': '\n127.0.0.1:45675\n127.0.0.1:45676\n127.0.0.1:45677\n127.0.0.1:45678\n127.0.0.1:45679'}}
        >>> b.proxies
        ['127.0.0.1:45675', '127.0.0.1:45676', '127.0.0.1:45677', '127.0.0.1:45678', '127.0.0.1:45679']
        >>> b.proxified
        True
        >>> b.open('http://localhost:45678')
        >>> 'Host: localhost:45678' in b.contents
        True
        >>> b._lastproxy['count'] == 1 and b._lastproxy['proxy'] in [0,1,2,3,4]
        True
        
        We can have a normal unproxified browser too ::
        
        >>> b1 = Browser(proxify=False)
        >>> b1.proxified
        False
        
        Next thing to verify is that we have our pseudo-random loop running
        First thing is we will choose 2 times the 2nd proxy, then the third
        And of course, we will set the mocker to change the proxy at each row.::
        
        >>> import mocker
        >>> import random
        >>> mocked = mocker.Mocker()
        >>> custom_random_int = mocked.replace('random.randint')
        >>> custom_random_int(0, 4)
        <mocker.Mock ...
        >>> mocked.result(2)
        >>> custom_random_int(0,1)
        <mocker.Mock ...
        >>> mocked.result(0)
        >>> custom_random_int(0, 4)
        <mocker.Mock ...
        >>> mocked.result(2)
        >>> custom_random_int(0,1)
        <mocker.Mock ...
        >>> mocked.result(0)
        >>> custom_random_int(0, 4)
        <mocker.Mock ...
        >>> mocked.result(2)
        >>> custom_random_int(0,1)
        <mocker.Mock ...
        >>> mocked.result(0)
        >>> custom_random_int(0, 4)
        <mocker.Mock ...
        >>> mocked.result(3)
        >>> custom_random_int(0,1)
        <mocker.Mock ...
        >>> mocked.result(0)
        >>> custom_random_int(0, 4)
        <mocker.Mock ...
        >>> mocked.result(4)
        >>> custom_random_int(0,1)
        <mocker.Mock ...
        >>> mocked.result(0)
        >>> custom_random_int(0, 4)
        <mocker.Mock ...
        >>> mocked.result(2)
        >>> custom_random_int(0,1)
        <mocker.Mock ...
        >>> mocked.result(0)
        >>> custom_random_int(0, 4)
        <mocker.Mock ...
        >>> mocked.result(1)
        >>> custom_random_int(0,1)
        <mocker.Mock ...
        >>> mocked.result(0)
        >>> mocked.replay()
        >>> b = Browser('http://localhost:45678', config=config)
        >>> b.open('http://localhost:45678')
        >>> b._lastproxy
        {'count': 1, 'proxy': 2}
        >>> b.open('http://localhost:45678')
        >>> b._lastproxy
        {'count': 2, 'proxy': 2}
        >>> b.open('http://localhost:45678')
        >>> b._lastproxy
        {'count': 3, 'proxy': 2}
        >>> b.open('http://localhost:45678')
        >>> b._lastproxy
        {'count': 1, 'proxy': 0}
        >>> b.open('http://localhost:45678')
        >>> b._lastproxy
        {'count': 1, 'proxy': 3}
        >>> b.open('http://localhost:45678')
        >>> b._lastproxy
        {'count': 1, 'proxy': 4}
        >>> b.open('http://localhost:45678')
        >>> b._lastproxy
        {'count': 1, 'proxy': 2}
        >>> b.open('http://localhost:45678')
        >>> b._lastproxy
        {'count': 1, 'proxy': 1}
        >>> mocked.restore()
        
        If the proxies are dead, we remove them from the list::
        
        >>> __, config = mkstemp()
        >>> open(config, 'w').write("""[collective.anonymousbrowser]
        ... proxies =
        ...     127.0.0.1:35675
        ...     127.0.0.1:35676
        ...     127.0.0.1:35677
        ...     127.0.0.1:45678
        ...     """)
        >>> mybrowser = Browser(config=config)
        >>> mybrowser.proxies
        ['127.0.0.1:35675', '127.0.0.1:35676', '127.0.0.1:35677', '127.0.0.1:45678']
        >>> mybrowser.open('http://localhost:45678')
        >>> mybrowser.proxies
        ['127.0.0.1:45678']
        >>> mybrowser.proxies = ['127.0.0.1:34785']
        >>> mybrowser.open('http://localhost:45678')
        Traceback (most recent call last):
        ...
        Exception: There are no valid proxies left
        
        
        The loop is recursion protected. If we return always the same host, so the chooser cannot choose anything else.
        It will loop until it crashes or it handle the recursion::
        
        >>> def randomint(a,b):
        ...     return 2
        >>> import random; random.randint = randomint
        >>> b2 = Browser(config=config)
        >>> b2.proxy_max_use
        3
        >>> b2._lastproxy['count']
        0
        >>> b2.chooseProxy()
        '...
        >>> b2._lastproxy['count']
        1
        >>> b2.chooseProxy()
        '...
        >>> b2._lastproxy['count']
        2
        >>> b2.chooseProxy()
        '...
        >>> b2._lastproxy['count']
        3
        >>> b2.chooseProxy()
        '...
        >>> b2.chooseProxy()
        Ho, seems we got the max wills to choose, something has gone wrong
        '127.0.0.1:35675'
        
        
        ..  vim:set ft=doctest:
        
        
        HISTORY
        ========
        0.4
        -----
        -   doc + bugfixes
        
        0.3
        -----
        -   adding error message
        
        0.2
        -----
        -   Adding proxy fallback facility
        
        0.1
        -----
        -   Initial release
        
        
        
        
        
        
        
Platform: UNKNOWN
