Java HTTPS URL 的基本代理身份验证返回 HTTP/1.0 407 需要代理身份验证

Basic proxy authentication for HTTPS URLs returns HTTP/1.0 407 Proxy Authentication Required

提问by John

I want to use a proxy with basic authentication (username, password) for a connection (and only this connection) in Java. The following code works for HTTP URLs (e.g. "http://www.google.com"):

我想在 Java 中使用具有基本身份验证（用户名、密码）的代理进行连接（并且仅此连接）。以下代码适用于 HTTP URL（例如“ http://www.google.com”）：

URL url = new URL("http://www.google.com");
HttpURLConnection httpURLConnection = null;
InetSocketAddress proxyLocation = new InetSocketAddress(proxyHost, proxyPort);
Proxy proxy = new Proxy(Proxy.Type.HTTP, proxyLocation);
httpURLConnection = (HttpURLConnection) url.openConnection(proxy);
// Works for HTTP only! Doesn't work for HTTPS!
String encoded = new sun.misc.BASE64Encoder().encodeBuffer((proxyUserName + ":" + proxyPassword).getBytes()).replace("\r\n", "");
httpURLConnection.setRequestProperty("Proxy-Authorization", "Basic " + encoded);
InputStream is = httpURLConnection.getInputStream();
InputStreamReader isr = new InputStreamReader(is); 
int data = isr.read();
while(data != -1){
  char c = (char) data;
  data = isr.read();
  System.out.print(c);
}
isr.close();

The code doesn't work for HTTPS URLs (e.g. "https://www.google.com"), though! I get java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 407 Proxy Authentication Required"when I try to access an HTTPS URL.

但是，该代码不适用于 HTTPS URL（例如“ https://www.google.com”）！我收到java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 407 Proxy Authentication Required"的时候我尝试访问HTTPS URL。

This code works for HTTP and HTTPS:

此代码适用于 HTTP 和 HTTPS：

URL url = new URL("https://www.google.com");
HttpURLConnection httpURLConnection = null;
InetSocketAddress proxyLocation = new InetSocketAddress(proxyHost, proxyPort);
Proxy proxy = new Proxy(Proxy.Type.HTTP, proxyLocation);
httpURLConnection = (HttpURLConnection) url.openConnection(proxy);
// Works for HTTP and HTTPS, but sets a global default!
Authenticator.setDefault(new Authenticator() {
  protected PasswordAuthentication getPasswordAuthentication() {
    return new PasswordAuthentication(proxyUserName, proxyPassword.toCharArray());
  }
});
InputStream is = httpURLConnection.getInputStream();
InputStreamReader isr = new InputStreamReader(is); 
int data = isr.read();
while(data != -1){
  char c = (char) data;
  data = isr.read();
  System.out.print(c);
}
isr.close();

The problem with the 2nd code is that it sets a new default Authenticatorand I don't want to do that, because this proxy is only used by a part of the application and a different part of the application could be using a different proxy. I don't want to set a global default for the whole application. Is there a way to get the 1st code to work with HTTPS or a way to use an Authenticatorwithout setting it as default?

第二个代码的问题在于它设置了一个新的默认值Authenticator，我不想这样做，因为此代理仅由应用程序的一部分使用，而应用程序的不同部分可能使用不同的代理。我不想为整个应用程序设置全局默认值。有没有办法让第一个代码与 HTTPS 一起使用，或者有一种方法可以在Authenticator不将其设置为默认值的情况下使用？

I have to use java.net.HttpURLConnection, because I'm overriding a method of a class which has to return an HttpURLConnection, so I can't use Apache HttpClient.

我必须使用java.net.HttpURLConnection，因为我要覆盖一个必须返回 的类的方法HttpURLConnection，所以我不能使用 Apache HttpClient。

采纳答案by Ferrybig

You can extend ProxiedHttpsConnectionand handle all the low level related stuff by yourself.

您可以自己扩展ProxiedHttpsConnection和处理所有低级相关的东西。

The following steps need to be done to make a connection over a HTTP proxy to a https website:

要通过 HTTP 代理连接到 https 网站，需要执行以下步骤：

Note: the communication with the proxy and http server should be in ASCII7.

注意：与代理和 http 服务器的通信应该是ASCII7。

1. Send CONNECT stackoverflow.com:443 HTTP/1.0\r\nto the proxy
2. Send your authentication: Proxy-Authorization: Basic c2F5WW91SGF2ZVNlZW5UaGlzSW5UaGVDb21tZW50cw==\r\n.
3. End the first request: \r\n
4. Read the response from the proxy until you see the combination "\r\n\r\n".
5. Parse the first line of the response you got from the proxy and check if it starts with HTTP/1.0 200.
6. Start a SSL session in place over the existing connection.
7. Send the start of a http request: GET /questions/3304006/persistent-httpurlconnection-in-java HTTP/1.0\r\n
8. Set the proper Host header: Host: stackoverflow.com\r\n
9. End the request to the http server: \r\n
10. Read till \r\nand parse first line as status message
11. Read till end of stream for request body

1. 发送CONNECT stackoverflow.com:443 HTTP/1.0\r\n给代理
2. 发送您的身份验证：Proxy-Authorization: Basic c2F5WW91SGF2ZVNlZW5UaGlzSW5UaGVDb21tZW50cw==\r\n。
3. 结束第一个请求： \r\n
4. 读取来自代理的响应，直到您看到组合“\r\n\r\n”。
5. 解析您从代理获得的响应的第一行，并检查它是否以HTTP/1.0 200.
6. 通过现有连接启动 SSL 会话。
7. 发送 http 请求的开始： GET /questions/3304006/persistent-httpurlconnection-in-java HTTP/1.0\r\n
8. 设置正确的主机头： Host: stackoverflow.com\r\n
9. 结束对http服务器的请求： \r\n
10. 阅读直到\r\n并将第一行解析为状态消息
11. 读取到请求正文的流结束

When we want to implement the HttpUrlConnection class, there are a few things we also need to consider:

当我们要实现 HttpUrlConnection 类时，还需要考虑以下几点：

• At the time the class is constructed, the class should store data for future connections, but NOT make it directly
• Any methods can be called in any order
• The closure of the OutputStreammeans the data transfer is done, not that the connection must finish
• Every api uses the methods in a different order
• HTTP headers are case insensitive, java maps are case sensitive.

• 在构造类时，类应该存储数据以供将来连接使用，但不要直接存储
• 可以以任何顺序调用任何方法
• 关闭OutputStream意味着数据传输完成，而不是连接必须完成
• 每个 api 以不同的顺序使用这些方法
• HTTP 标头不区分大小写，java 映射区分大小写。

Quickly said, there are just many pitfalls

快说，只是有很多陷阱

In the class I designed, it uses boolean flags to remember if the connectmethod and the afterPostClosuremethods are called, it also has support if getInputStream()is called before the OutputStreamis closed.

在我设计的类中，它使用布尔标志来记住connect方法和afterPostClosure方法是否被调用，它还支持 ifgetInputStream()在OutputStream关闭之前被调用。

This class also uses as little wrapping as possible over the streams returned by the socket, to prevent being really complex.

此类还在套接字返回的流上使用尽可能少的包装，以防止变得非常复杂。

public class ProxiedHttpsConnection extends HttpURLConnection {

    private final String proxyHost;
    private final int proxyPort;
    private static final byte[] NEWLINE = "\r\n".getBytes();//should be "ASCII7"

    private Socket socket;
    private final Map<String, List<String>> headers = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
    private final Map<String, List<String>> sendheaders = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
    private final Map<String, List<String>> proxyheaders = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
    private final Map<String, List<String>> proxyreturnheaders = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
    private int statusCode;
    private String statusLine;
    private boolean isDoneWriting;

    public ProxiedHttpsConnection(URL url,
            String proxyHost, int proxyPort, String username, String password)
            throws IOException {
        super(url);
        socket = new Socket();
        this.proxyHost = proxyHost;
        this.proxyPort = proxyPort;
        String encoded = Base64.encode((username + ":" + password).getBytes())
                .replace("\r\n", "");
        proxyheaders.put("Proxy-Authorization", new ArrayList<>(Arrays.asList("Basic " + encoded)));
    }

    @Override
    public OutputStream getOutputStream() throws IOException {
        connect();
        afterWrite();
        return new FilterOutputStream(socket.getOutputStream()) {
            @Override
            public void write(byte[] b, int off, int len) throws IOException {
                out.write(String.valueOf(len).getBytes());
                out.write(NEWLINE);
                out.write(b, off, len);
                out.write(NEWLINE);
            }

            @Override
            public void write(byte[] b) throws IOException {
                out.write(String.valueOf(b.length).getBytes());
                out.write(NEWLINE);
                out.write(b);
                out.write(NEWLINE);
            }

            @Override
            public void write(int b) throws IOException {
                out.write(String.valueOf(1).getBytes());
                out.write(NEWLINE);
                out.write(b);
                out.write(NEWLINE);
            }

            @Override
            public void close() throws IOException {
                afterWrite();
            }

        };
    }

    private boolean afterwritten = false;

    @Override
    public InputStream getInputStream() throws IOException {
        connect();
        return socket.getInputStream();

    }

    @Override
    public void setRequestMethod(String method) throws ProtocolException {
        this.method = method;
    }

    @Override
    public void setRequestProperty(String key, String value) {
        sendheaders.put(key, new ArrayList<>(Arrays.asList(value)));
    }

    @Override
    public void addRequestProperty(String key, String value) {
        sendheaders.computeIfAbsent(key, l -> new ArrayList<>()).add(value);
    }

    @Override
    public Map<String, List<String>> getHeaderFields() {
        return headers;
    }

    @Override
    public void connect() throws IOException {
        if (connected) {
            return;
        }
        connected = true;
        socket.setSoTimeout(getReadTimeout());
        socket.connect(new InetSocketAddress(proxyHost, proxyPort), getConnectTimeout());
        StringBuilder msg = new StringBuilder();
        msg.append("CONNECT ");
        msg.append(url.getHost());
        msg.append(':');
        msg.append(url.getPort() == -1 ? 443 : url.getPort());
        msg.append(" HTTP/1.0\r\n");
        for (Map.Entry<String, List<String>> header : proxyheaders.entrySet()) {
            for (String l : header.getValue()) {
                msg.append(header.getKey()).append(": ").append(l);
                msg.append("\r\n");
            }
        }

        msg.append("Connection: close\r\n");
        msg.append("\r\n");
        byte[] bytes;
        try {
            bytes = msg.toString().getBytes("ASCII7");
        } catch (UnsupportedEncodingException ignored) {
            bytes = msg.toString().getBytes();
        }
        socket.getOutputStream().write(bytes);
        socket.getOutputStream().flush();
        byte reply[] = new byte[200];
        byte header[] = new byte[200];
        int replyLen = 0;
        int headerLen = 0;
        int newlinesSeen = 0;
        boolean headerDone = false;
        /* Done on first newline */
        InputStream in = socket.getInputStream();
        while (newlinesSeen < 2) {
            int i = in.read();
            if (i < 0) {
                throw new IOException("Unexpected EOF from remote server");
            }
            if (i == '\n') {
                if (newlinesSeen != 0) {
                    String h = new String(header, 0, headerLen);
                    String[] split = h.split(": ");
                    if (split.length != 1) {
                        proxyreturnheaders.computeIfAbsent(split[0], l -> new ArrayList<>()).add(split[1]);
                    }
                }
                headerDone = true;
                ++newlinesSeen;
                headerLen = 0;
            } else if (i != '\r') {
                newlinesSeen = 0;
                if (!headerDone && replyLen < reply.length) {
                    reply[replyLen++] = (byte) i;
                } else if (headerLen < reply.length) {
                    header[headerLen++] = (byte) i;
                }
            }
        }

        String replyStr;
        try {
            replyStr = new String(reply, 0, replyLen, "ASCII7");
        } catch (UnsupportedEncodingException ignored) {
            replyStr = new String(reply, 0, replyLen);
        }

        // Some proxies return http/1.1, some http/1.0 even we asked for 1.0
        if (!replyStr.startsWith("HTTP/1.0 200") && !replyStr.startsWith("HTTP/1.1 200")) {
            throw new IOException("Unable to tunnel. Proxy returns \"" + replyStr + "\"");
        }
        SSLSocket s = (SSLSocket) ((SSLSocketFactory) SSLSocketFactory.getDefault())
                .createSocket(socket, url.getHost(), url.getPort(), true);
        s.startHandshake();
        socket = s;
        msg.setLength(0);
        msg.append(method);
        msg.append(" ");
        msg.append(url.toExternalForm().split(String.valueOf(url.getPort()), -2)[1]);
        msg.append(" HTTP/1.0\r\n");
        for (Map.Entry<String, List<String>> h : sendheaders.entrySet()) {
            for (String l : h.getValue()) {
                msg.append(h.getKey()).append(": ").append(l);
                msg.append("\r\n");
            }
        }
        if (method.equals("POST") || method.equals("PUT")) {
            msg.append("Transfer-Encoding: Chunked\r\n");
        }
        msg.append("Host: ").append(url.getHost()).append("\r\n");
        msg.append("Connection: close\r\n");
        msg.append("\r\n");
        try {
            bytes = msg.toString().getBytes("ASCII7");
        } catch (UnsupportedEncodingException ignored) {
            bytes = msg.toString().getBytes();
        }
        socket.getOutputStream().write(bytes);
        socket.getOutputStream().flush();
    }

    private void afterWrite() throws IOException {
        if (afterwritten) {
            return;
        }
        afterwritten = true;
        socket.getOutputStream().write(String.valueOf(0).getBytes());
        socket.getOutputStream().write(NEWLINE);
        socket.getOutputStream().write(NEWLINE);
        byte reply[] = new byte[200];
        byte header[] = new byte[200];
        int replyLen = 0;
        int headerLen = 0;
        int newlinesSeen = 0;
        boolean headerDone = false;
        /* Done on first newline */
        InputStream in = socket.getInputStream();
        while (newlinesSeen < 2) {
            int i = in.read();
            if (i < 0) {
                throw new IOException("Unexpected EOF from remote server");
            }
            if (i == '\n') {
                if (headerDone) {
                    String h = new String(header, 0, headerLen);
                    String[] split = h.split(": ");
                    if (split.length != 1) {
                        headers.computeIfAbsent(split[0], l -> new ArrayList<>()).add(split[1]);
                    }
                }
                headerDone = true;
                ++newlinesSeen;
                headerLen = 0;
            } else if (i != '\r') {
                newlinesSeen = 0;
                if (!headerDone && replyLen < reply.length) {
                    reply[replyLen++] = (byte) i;
                } else if (headerLen < header.length) {
                    header[headerLen++] = (byte) i;
                }
            }
        }

        String replyStr;
        try {
            replyStr = new String(reply, 0, replyLen, "ASCII7");
        } catch (UnsupportedEncodingException ignored) {
            replyStr = new String(reply, 0, replyLen);
        }

        /* We asked for HTTP/1.0, so we should get that back */
        if ((!replyStr.startsWith("HTTP/1.0 200")) && !replyStr.startsWith("HTTP/1.1 200")) {
            throw new IOException("Server returns \"" + replyStr + "\"");
        }
    }

    @Override
    public void disconnect() {
        try {
            socket.close();
        } catch (IOException ex) {
            Logger.getLogger(ProxiedHttpsConnection.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

    @Override
    public boolean usingProxy() {
        return true;
    }
}

Current bugs with the above code:

上述代码的当前错误：

• Streams are not closed on errors during post
• Streams are not closed during errors with the initial contact with the proxy
• It doesn't support http redirects
• It doesn't support the http 1.1 things like chunked and gzip encoding, but this is no problem as we announce ourselves as a http1.0 client.

• 发布期间不会因错误而关闭流
• 在与代理的初始联系发生错误期间不会关闭流
• 它不支持 http 重定向
• 它不支持像分块和 gzip 编码这样的 http 1.1 东西，但这没有问题，因为我们宣布自己是 http1.0 客户端。

The above code can be used like:

上面的代码可以像这样使用：

    ProxiedHttpsConnection n = new ProxiedHttpsConnection(
            new URL("https://stackoverflow.com:443/questions/3304006/persistent-httpurlconnection-in-java"), 
            "proxy.example.com", 8080, "root", "flg83yvem#");
    n.setRequestMethod("GET");
    n.addRequestProperty("User-Agent", "Java test https://stackoverflow.com/users/1542723/ferrybig");
    //try (OutputStream out = n.getOutputStream()) {
    //  out.write("Hello?".getBytes());
    //}
    try (InputStream in = n.getInputStream()) {
        byte[] buff = new byte[1024];
        int length;
        while ((length = in.read(buff)) >= 0) {
            System.out.write(buff, 0, length);
        }
    }

If you are going to use this with a kind of proxy selector, you should check the protocol of the url to see if its http or https, if its http, don't use this class, and instead attach the header manually like:

如果您打算将它与一种代理选择器一起使用，您应该检查 url 的协议，看看它是 http 还是 https，如果是 http，不要使用此类，而是手动附加标头，如：

httpURLConnection.setRequestProperty("Proxy-Authorization", "Basic " + encoded);

Why not using httpsUrlConnection.setSSLSocketFactory

为什么不使用httpsUrlConnection.setSSLSocketFactory

While java has this method, attempts to use it will show you why it won't work, java just keeps calling the createSocket(Socket s, String host, int port, boolean autoClose)with an already open connection, making it impossible to do the proxy stuff manually.

虽然 java 有这个方法，尝试使用它会告诉你为什么它不起作用，java 只是createSocket(Socket s, String host, int port, boolean autoClose)用一个已经打开的连接调用，因此无法手动执行代理操作。

回答by mico

Can you use HttpsUrlConnection? It extends HttpUrlConnection, so casting to HttpUrlConnection may be ok when returning from the class.

你可以使用 HttpsUrlConnection 吗？它扩展了 HttpUrlConnection，因此从类返回时转换为 HttpUrlConnection 可能没问题。

The code is similar, instead of HttpUrlConnection use one with https in the name.

代码类似，而不是 HttpUrlConnection 使用名称中带有 https 的代码。

Use following code:

使用以下代码：

if (testUrlHttps.getProtocol().toLowerCase().equals("https")) {
   trustAllHosts();
   HttpsURLConnection https = (HttpsURLConnection) url.openConnection();
   https.setHostnameVerifier(DO_NOT_VERYFY);
   urlCon = https;
} else {
   urlCon = (HttpURLConnection) url.openConnection();
}

Sources:

资料来源：

[1] https://docs.oracle.com/javase/7/docs/api/javax/net/ssl/HttpsURLConnection.html

[1] https://docs.oracle.com/javase/7/docs/api/javax/net/ssl/HttpsURLConnection.html

[2] HttpURLConnection - "https://" vs. "http://"(snippet)

[2] HttpURLConnection - "https://" vs. "http://"(snippet)

回答by whbogado

Unfortunately there is no simple solution for what you are trying to achieve. Your 1st code doesn't work with HTTPS because you are setting the authentication header directly. Since the client encrypts all data, the proxy server has no way of extracting any information from the request.

不幸的是，您要实现的目标没有简单的解决方案。您的第一个代码不适用于 HTTPS，因为您是直接设置身份验证标头。由于客户端加密所有数据，代理服务器无法从请求中提取任何信息。

In fact, HTTPS and proxy servers work in opposing ways. The proxy server wants to see all the data that flows between the client and the final server and take action based on what it sees. On the other hand the HTTPS protocol encrypts all data so that no one can see the data until it reaches the final destination. The encryption algorithm is negotiated between the client and the final destination so that the proxy server can't decrypt any information, in fact it can't even know which protocol the client is using.

事实上，HTTPS 和代理服务器以相反的方式工作。代理服务器希望查看在客户端和最终服务器之间流动的所有数据，并根据所看到的内容采取行动。另一方面，HTTPS 协议对所有数据进行加密，以便在数据到达最终目的地之前没有人可以看到数据。加密算法是在客户端和最终目的地之间协商的，因此代理服务器无法解密任何信息，实际上它甚至无法知道客户端使用的是哪种协议。

To use a proxy server on a HTTPS connection, the client has to establish a tunnel. To do this, it has to issue a CONNECT command directly to the proxy, for example:

要在 HTTPS 连接上使用代理服务器，客户端必须建立隧道。为此，它必须直接向代理发出 CONNECT 命令，例如：

CONNECT www.google.com:443 HTTP/1.0

and send the credentials to authenticate with the proxy server.

并发送凭据以通过代理服务器进行身份验证。

If the connection is successful the client can send and receive data through the connection. The proxy server is completely blind to the data. The data only passes through it on its way between the client and the server.

如果连接成功，客户端可以通过连接发送和接收数据。代理服务器对数据完全视而不见。数据仅在客户端和服务器之间通过它。

When you execute url.openConnection(proxy)on a HTTP URL it returns an instance of HttpURLConnection, when run on a HTTPS URL like in your 2nd code it retuns an instance of HttpsURLConnection.

当您url.openConnection(proxy)在 HTTP URL 上执行时，它返回 的实例HttpURLConnection，当在 HTTPS URL 上运行时（如您的第二个代码中），它返回 的实例HttpsURLConnection。

You are receiving the 407 error code because the proxy server can't extract he authentication information from the header you have sent. Looking at the exception stack we can see that the exception is thrown at sun.net.www.protocol.http.HttpURLConnection.doTunneling()which issues the CONNECT command to establish the HTTPS tunnel through the proxy. In the source code for sun.net.www.protocol.http.HttpURLConnectionwe can see:

您收到 407 错误代码是因为代理服务器无法从您发送的标头中提取身份验证信息。查看异常堆栈，我们可以看到抛出异常，该异常sun.net.www.protocol.http.HttpURLConnection.doTunneling()发出 CONNECT 命令以通过代理建立 HTTPS 隧道。在源代码中sun.net.www.protocol.http.HttpURLConnection我们可以看到：

/* We only have a single static authenticator for now.
 * REMIND:  backwards compatibility with JDK 1.1.  Should be
 * eliminated for JDK 2.0.
 */
private static HttpAuthenticator defaultAuth;

So it seems that the default authenticator is the only way to provide the proxy credentials.

因此，默认身份验证器似乎是提供代理凭据的唯一方法。

To do what you want, you would have to go down to the connection level and handle the HTTP protocol yourself because you have to talk to the proxy server not directly to the Google server.

要执行您想要的操作，您必须深入到连接级别并自己处理 HTTP 协议，因为您必须与代理服务器对话，而不是直接与 Google 服务器对话。

回答by Abhishek Anand

Ok this is what you need to do,

好的，这就是你需要做的，

public class ProxyAuth extends Authenticator {
    private PasswordAuthentication auth;

    ProxyAuth(String user, String password) {
        auth = new PasswordAuthentication(user, password == null ? new char[]{} : password.toCharArray());
    }

    protected PasswordAuthentication getPasswordAuthentication() {
        return auth;
    }
}

.

.

public class ProxySetup {
    public HttpURLConnection proxySetup(String urlInput)
    {
        URL url;
        try {
            url = new URL(urlInput);

            Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("10.66.182.100", 80)); // or whatever your proxy is
            HttpURLConnection uc = (HttpURLConnection)url.openConnection(proxy);
            System.setProperty("https.proxyHost", "10.66.182.100");
            System.setProperty("https.proxyPort", "80");
            System.setProperty("http.proxyHost", "10.66.182.100");
            System.setProperty("http.proxyPort", "80");
            String encoded = new String(Base64.encodeBase64(("domain\Username" + ":" + "Password").getBytes()));

            uc.setRequestProperty("Proxy-Authorization", "Basic " + encoded);
            Authenticator.setDefault(new ProxyAuth("domain\Username", "Password"));

            System.out.println("ProxySetup : proxySetup");
            return uc;
        } catch (Exception e) {
            // TODO Auto-generated catch block
            System.out.println("ProxySetup : proxySetup - Failed");
            e.printStackTrace();
        }
        return null;
    }
}

Use it like.

使用它喜欢。

HttpURLConnection conn = new ProxySetup().proxySetup(URL)