Python FTP 应用程序中的代理

Proxies in Python FTP application

提问by

I'm developing an FTP client in Python ftplib. How do I add proxies support to it (most FTP apps I have seen seem to have it)? I'm especially thinking about SOCKS proxies, but also other types... FTP, HTTP (is it even possible to use HTTP proxies with FTP program?)

我正在用 Python ftplib 开发一个 FTP 客户端。我如何为它添加代理支持（我见过的大多数 FTP 应用程序似乎都有它）？我特别考虑 SOCKS 代理，但也考虑其他类型... FTP、HTTP（甚至可以将 HTTP 代理与 FTP 程序一起使用？）

Any ideas how to do it?

任何想法如何做到？

回答by Kevin Boyd

As per thissource.

根据这个来源。

Depends on the proxy, but a common method is to ftp to the proxy, then use the username and password for the destination server.

取决于代理，但常用的方法是 ftp 到代理，然后使用目标服务器的用户名和密码。

E.g. for ftp.example.com:

例如对于 ftp.example.com：

Server address: proxyserver (or open proxyserver from with ftp)
User:           [email protected]
Password:       password

In Python code:

在 Python 代码中：

from ftplib import FTP
site = FTP('my_proxy')
site.set_debuglevel(1)
msg = site.login('[email protected]', 'password')
site.cwd('/pub')

回答by S.Lott

You can use the ProxyHandlerin urllib2.

您可以使用ProxyHandler在urllib2。

ph = urllib2.ProxyHandler( { 'ftp' : proxy_server_url } )
server= urllib2.build_opener( ph )

回答by Raphael Jolivet

I had the same problem and needed to use the ftplibmodule (not to rewrite all my scripts with URLlib2).

我遇到了同样的问题，需要使用ftplib模块（不要用 URLlib2 重写我的所有脚本）。

I have managed to write a script that installs transparent HTTP tunnelingon the socket layer (used by ftplib).

我设法编写了一个在套接字层（由 ftplib 使用）上安装透明HTTP 隧道的脚本。

Now, I can do FTP over HTTPtransparently !

现在，我可以通过 HTTP透明地执行FTP 了！

You can get it there: http://code.activestate.com/recipes/577643-transparent-http-tunnel-for-python-sockets-to-be-u/

你可以在那里得到它：http: //code.activestate.com/recipes/577643-transparent-http-tunnel-for-python-sockets-to-be-u/

回答by Collin

Patching the builtin socket library definitely won't be an option for everyone, but my solution was to patch socket.create_connection()to use an HTTP proxy when the hostname matches a whitelist:

修补内置套接字库绝对不是每个人的选择，但我的解决方案是socket.create_connection()在主机名匹配白名单时修补以使用 HTTP 代理：

from base64 import b64encode
from functools import wraps
import socket

_real_create_connection = socket.create_connection
_proxied_hostnames = {}  # hostname: (proxy_host, proxy_port, proxy_auth)


def register_proxy (host, proxy_host, proxy_port, proxy_username=None, proxy_password=None):
    proxy_auth = None
    if proxy_username is not None or proxy_password is not None:
        proxy_auth = b64encode('{}:{}'.format(proxy_username or '', proxy_password or ''))
    _proxied_hostnames[host] = (proxy_host, proxy_port, proxy_auth)


@wraps(_real_create_connection)
def create_connection (address, *args, **kwds):
    host, port = address
    if host not in _proxied_hostnames:
        return _real_create_connection(address, *args, **kwds)

    proxy_host, proxy_port, proxy_auth = _proxied_hostnames[host]
    conn = _real_create_connection((proxy_host, proxy_port), *args, **kwds)
    try:
        conn.send('CONNECT {host}:{port} HTTP/1.1\r\nHost: {host}:{port}\r\n{auth_header}\r\n'.format(
            host=host, port=port,
            auth_header=('Proxy-Authorization: basic {}\r\n'.format(proxy_auth) if proxy_auth else '')
        ))
        response = ''
        while not response.endswith('\r\n\r\n'):
            response += conn.recv(4096)
        if response.split()[1] != '200':
            raise socket.error('CONNECT failed: {}'.format(response.strip()))
    except socket.error:
        conn.close()
        raise

    return conn


socket.create_connection = create_connection

I also had to create a subclass of ftplib.FTP that ignores the hostreturned by PASVand EPSVFTP commands. Example usage:

我还必须创建一个 ftplib.FTP 的子类，它忽略host返回的 byPASV和EPSVFTP 命令。用法示例：

from ftplib import FTP
import paramiko  # For SFTP
from proxied_socket import register_proxy

class FTPIgnoreHost (FTP):
    def makepasv (self):
        # Ignore the host returned by PASV or EPSV commands (only use the port).
        return self.host, FTP.makepasv(self)[1]

register_proxy('ftp.example.com', 'proxy.example.com', 3128, 'proxy_username', 'proxy_password')

ftp_connection = FTP('ftp.example.com', 'ftp_username', 'ftp_password')

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())  # If you don't care about security.
ssh.connect('ftp.example.com', username='sftp_username', password='sftp_password')
sftp_connection = ssh.open_sftp()

回答by Eugene Morozov

Standard module ftplibdoesn't support proxies. It seems the only solution is to write your own customized version of the ftplib.

标准模块ftplib不支持代理。似乎唯一的解决方案是编写您自己的ftplib.

回答by Lucas Cimon

Here is workaround using requests, tested with a squid proxy that does NOT support CONNECT tunneling:

这是使用requests不支持 CONNECT 隧道的鱿鱼代理进行测试的解决方法：

def ftp_fetch_file_through_http_proxy(host, user, password, remote_filepath, http_proxy, output_filepath):
    """
    This function let us to make a FTP RETR query through a HTTP proxy that does NOT support CONNECT tunneling.
    It is equivalent to: curl -x $HTTP_PROXY --user $USER:$PASSWORD ftp://$FTP_HOST/path/to/file
    It returns the 'Last-Modified' HTTP header value from the response.

    More precisely, this function sends the following HTTP request to $HTTP_PROXY:
        GET ftp://$USER:$PASSWORD@$FTP_HOST/path/to/file HTTP/1.1
    Note that in doing so, the host in the request line does NOT match the host we send this packet to.

    Python `requests` lib does not let us easily "cheat" like this.
    In order to achieve what we want, we need:
    - to mock urllib3.poolmanager.parse_url so that it returns a (host,port) pair indicating to send the request to the proxy
    - to register a connection adapter to the 'ftp://' prefix. This is basically a HTTP adapter but it uses the FULL url of
    the resource to build the request line, instead of only its relative path.
    """
    url = 'ftp://{}:{}@{}/{}'.format(user, password, host, remote_filepath)
    proxy_host, proxy_port = http_proxy.split(':')

    def parse_url_mock(url):
        return requests.packages.urllib3.util.url.parse_url(url)._replace(host=proxy_host, port=proxy_port, scheme='http')

    with open(output_filepath, 'w+b') as output_file, patch('requests.packages.urllib3.poolmanager.parse_url', new=parse_url_mock):
        session = requests.session()
        session.mount('ftp://', FTPWrappedInFTPAdapter())
        response = session.get(url)
        response.raise_for_status()
        output_file.write(response.content)
        return response.headers['last-modified']


class FTPWrappedInFTPAdapter(requests.adapters.HTTPAdapter):
    def request_url(self, request, _):
        return request.url