How to make python Requests work via socks proxy
The modern way:
pip install -U requests[socks]
then
import requests
resp = requests.get('http://go.to',
proxies=dict(http='socks5://user:pass@host:port',
https='socks5://user:pass@host:port'))
As of requests
version 2.10.0, released on 2016-04-29, requests
supports SOCKS.
It requires PySocks, which can be installed with pip install pysocks
.
Example usage:
import requests
proxies = {'http': "socks5://myproxy:9191"}
requests.get('http://example.org', proxies=proxies)
In case someone has tried all of these older answers, and is still running into problems like:
requests.exceptions.ConnectionError:
SOCKSHTTPConnectionPool(host='myhost', port=80):
Max retries exceeded with url: /my/path
(Caused by NewConnectionError('<requests.packages.urllib3.contrib.socks.SOCKSConnection object at 0x106812bd0>:
Failed to establish a new connection:
[Errno 8] nodename nor servname provided, or not known',))
It may be because, by default, requests
is configured to resolve DNS queries on the local side of the connection.
Try changing your proxy URL from socks5://proxyhost:1234
to socks5h://proxyhost:1234
. Note the extra h
(it stands for hostname resolution).
The PySocks package module default is to do remote resolution, and I'm not sure why requests made their integration this obscurely divergent, but here we are.
You need install pysocks , my version is 1.0 and the code works for me:
import socket
import socks
import requests
ip='localhost' # change your proxy's ip
port = 0000 # change your proxy's port
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, ip, port)
socket.socket = socks.socksocket
url = u'http://ajax.googleapis.com/ajax/services/search/images?v=1.0&q=inurl%E8%A2%8B'
print(requests.get(url).text)
As soon as python requests
will be merged with SOCKS5
pull request it will do as simple as using proxies
dictionary:
#proxy
# SOCKS5 proxy for HTTP/HTTPS
proxies = {
'http' : "socks5://myproxy:9191",
'https' : "socks5://myproxy:9191"
}
#headers
headers = {
}
url='http://icanhazip.com/'
res = requests.get(url, headers=headers, proxies=proxies)
See SOCKS Proxy Support
Another options, in case that you cannot wait request
to be ready, when you cannot use requesocks
- like on GoogleAppEngine due to the lack of pwd
built-in module, is to use PySocks that was mentioned above:
- Grab the
socks.py
file from the repo and put a copy in your root folder; - Add
import socks
andimport socket
At this point configure and bind the socket before using with urllib2
- in the following example:
import urllib2
import socket
import socks
socks.set_default_proxy(socks.SOCKS5, "myprivateproxy.net",port=9050)
socket.socket = socks.socksocket
res=urllib2.urlopen(url).read()