java 如何使用相同的 TLS 会话通过数据连接连接到 FTPS 服务器？

Question

提问by Martin Prikryl

Environment: I'm using Sun Java JDK 1.8.0_60 on 64-bit Windows 7, using Spring Integration 4.1.6 (which internally appears to use Apache Commons Net 3.3 for FTPS access).

环境：我在 64 位 Windows 7 上使用 Sun Java JDK 1.8.0_60，使用 Spring Integration 4.1.6（内部似乎使用 Apache Commons Net 3.3 进行 FTPS 访问）。

I'm attempting to integrate with our application an automatic download from our client's FTPS server. I've done so successfully with SFTP servers using Spring Integration without any trouble for other clients without issues, but this is the first time a client has required us to use FTPS, and getting it to connect has been very puzzling. While in my real application I'm configuring Spring Integration using XML beans, to try to understand what's not working I'm using the following test code (though I'm anonymizing the actual host/username/password here):

我正在尝试与我们的应用程序集成从我们客户端的 FTPS 服务器自动下载。我已经使用 Spring Integration 在 SFTP 服务器上成功地做到了这一点，其他客户端没有任何问题而没有任何问题，但这是客户端第一次要求我们使用 FTPS，让它连接起来非常令人费解。在我的实际应用程序中，我正在使用 XML bean 配置 Spring Integration，为了尝试了解什么不起作用，我使用了以下测试代码（尽管我在这里对实际主机/用户名/密码进行了匿名处理）：

final DefaultFtpsSessionFactory sessionFactory = new DefaultFtpsSessionFactory();
sessionFactory.setHost("XXXXXXXXX");
sessionFactory.setPort(990);
sessionFactory.setUsername("XXXXXXX");
sessionFactory.setPassword("XXXXXXX");
sessionFactory.setClientMode(2);
sessionFactory.setFileType(2);
sessionFactory.setUseClientMode(true);
sessionFactory.setImplicit(true);
sessionFactory.setTrustManager(TrustManagerUtils.getAcceptAllTrustManager());
sessionFactory.setProt("P");
sessionFactory.setProtocol("TLSv1.2");
sessionFactory.setProtocols(new String[]{"TLSv1.2"});
sessionFactory.setSessionCreation(true);
sessionFactory.setCipherSuites(new String[]{"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"});

final FtpSession session = sessionFactory.getSession();
//try {
    final FTPFile[] ftpFiles = session.list("/");
    logger.debug("FtpFiles: {}", (Object[]) ftpFiles);
//} catch (Exception ignored ) {}
session.close();

I'm running this code with -Djavax.net.debug=allto get all the TLS debugging information printed.

我正在运行此代码以-Djavax.net.debug=all打印所有 TLS 调试信息。

The main "control" connection to the FTPS server works fine, but when it tries to open the data connection for the list (or any other data connection I've tried), I get a javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake, caused by java.io.EOFException: SSL peer shut down incorrectly. If I uncomment the swallowing-exceptions catch block around the session.listcommand, then I can see (though the javax.net.debug output) that the server sent the following message after rejecting the data connection SSL handshake:

到 FTPS 服务器的主要“控制”连接工作正常，但是当它尝试打开列表的数据连接（或我尝试过的任何其他数据连接）时，我得到一个javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake, 由java.io.EOFException: SSL peer shut down incorrectly. 如果我取消注释session.list命令周围的swallowing-exceptions catch块，那么我可以看到（通过javax.net.debug输出）服务器在拒绝数据连接SSL握手后发送了以下消息：

main, READ: TLSv1.2 Application Data, length = 129
Padded plaintext after DECRYPTION:  len = 105
0000: 34 35 30 20 54 4C 53 20   73 65 73 73 69 6F 6E 20  450 TLS session 
0010: 6F 66 20 64 61 74 61 20   63 6F 6E 6E 65 63 74 69  of data connecti
0020: 6F 6E 20 68 61 73 20 6E   6F 74 20 72 65 73 75 6D  on has not resum
0030: 65 64 20 6F 72 20 74 68   65 20 73 65 73 73 69 6F  ed or the sessio
0040: 6E 20 64 6F 65 73 20 6E   6F 74 20 6D 61 74 63 68  n does not match
0050: 20 74 68 65 20 63 6F 6E   74 72 6F 6C 20 63 6F 6E   the control con
0060: 6E 65 63 74 69 6F 6E 0D   0A                       nection..

What appears to be happening (and this is my first time dealing with FTPS, though I've dealt with plain FTP before) is that the way the server ensures authentication and encryption over both the control and data connections is that after a "normal" TLS connection to establish the control connection and authentication happens there, each data connection requires the client to connect with the same TLS session. This makes sense to me as how it's supposed to be work, but the Apache Commons Net FTPS implementation doesn't seem to be doing that. It seems to be trying to establish a new TLS session, and so the server is rejecting the attempt.

似乎正在发生的事情（这是我第一次处理 FTPS，虽然我之前处理过普通 FTP）是服务器确保对控制和数据连接进行身份验证和加密的方式是在“正常”之后TLS 连接建立控制连接和身份验证发生在那里，每个数据连接都要求客户端使用相同的 TLS 会话进行连接。这对我来说是有意义的，因为它应该如何工作，但 Apache Commons Net FTPS 实现似乎并没有这样做。它似乎正在尝试建立一个新的 TLS 会话，因此服务器拒绝了该尝试。

Based on this question about resuming SSL sessions in JSSE, it appears that Java assumes or requires a different session for each host/post combination. My hypothesis is that since the FTPS data connection is on a different port than the control connection, it's not finding the existing session and is trying to establish a new one, so the connection fails.

基于这个关于在 JSSE 中恢复 SSL 会话的问题，Java 似乎假设或需要每个主机/帖子组合的不同会话。我的假设是，由于 FTPS 数据连接位于与控制连接不同的端口上，因此它没有找到现有会话并试图建立一个新会话，因此连接失败。

I see three main possibilities:

我看到了三种主要的可能性：

The server is not following the FTPS standard in requiring the same TLS session on the data port as on the control port. I can connect to the server fine (using the same host/user/password as I'm trying to use in my code) using FileZilla 3.13.1. The server identifies itself as "FileZilla Server 0.9.53 beta" upon login, so perhaps this is some sort of proprietary FileZilla way of doing things, and there's something odd I need to do to convince Java to use the same TLS session.
The Apache Commons Net client doesn't actually follow the FTPS standard, and only allows some subset that doesn't allow for securing the data connections. This would seem odd, as it appears to be the standard way of connecting to FTPS from within Java.
I'm completely missing something and misdiagnosing this.

服务器不遵循 FTPS 标准，要求数据端口上的 TLS 会话与控制端口上的相同。我可以使用 FileZilla 3.13.1 连接到服务器（使用与我在代码中尝试使用的相同的主机/用户/密码）。服务器在登录时将自己标识为“FileZilla Server 0.9.53 beta”，所以这可能是某种专有的 FileZilla 做事方式，我需要做一些奇怪的事情来说服 Java 使用相同的 TLS 会话。
Apache Commons Net 客户端实际上并不遵循 FTPS 标准，并且只允许某些不允许保护数据连接的子集。这看起来很奇怪，因为它似乎是从 Java 内部连接到 FTPS 的标准方式。
我完全错过了一些东西并误诊了这一点。

I'd appreciate any direction you can provide as to how to connect to this kind of FTPS server. Thank you.

我很感激你能提供的关于如何连接到这种 FTPS 服务器的任何方向。谢谢你。

Answer 1

回答by Martin Prikryl

Indeed some FTP(S) servers do require that the TLS/SSL session is reused for the data connection. This is a security measure by which the server can verify that the data connection is used by the same client as the control connection.

实际上，某些 FTP(S) 服务器确实要求将 TLS/SSL 会话重用于数据连接。这是一种安全措施，服务器可以通过它验证数据连接是否与控制连接由同一客户端使用。

Some references for common FTP servers:

常见FTP服务器的一些参考：

vsftpd: https://scarybeastsecurity.blogspot.com/2009/02/vsftpd-210-released.html
FileZilla server: https://svn.filezilla-project.org/filezilla?view=revision&revision=6661
ProFTPD: http://www.proftpd.org/docs/contrib/mod_tls.html#TLSOptions(NoSessionReuseRequireddirective)

vsftpd：https: //scarybeastsecurity.blogspot.com/2009/02/vsftpd-210-released.html
FileZilla 服务器：https://svn.filezilla-project.org/filezilla ?view =revision &revision =6661
ProFTPD：http: //www.proftpd.org/docs/contrib/mod_tls.html#TLSOptions（NoSessionReuseRequired指令）

What may help you with the implementation is that Cyberduck FTP(S) client does support TLS/SSL session reuse and it uses Apache Commons Net library:

可以帮助您实现的是 Cyberduck FTP(S) 客户端确实支持 TLS/SSL 会话重用，并且它使用 Apache Commons Net 库：

https://trac.cyberduck.io/ticket/5087- Reuse Session key on data connection

See its FTPClient.javacode (extends Commons Net FTPSClient), particularly its override of _prepareDataSocket_method:

@Override
protected void _prepareDataSocket_(final Socket socket) throws IOException {
    if(preferences.getBoolean("ftp.tls.session.requirereuse")) {
        if(socket instanceof SSLSocket) {
            // Control socket is SSL
            final SSLSession session = ((SSLSocket) _socket_).getSession();
            if(session.isValid()) {
                final SSLSessionContext context = session.getSessionContext();
                context.setSessionCacheSize(preferences.getInteger("ftp.ssl.session.cache.size"));
                try {
                    final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
                    sessionHostPortCache.setAccessible(true);
                    final Object cache = sessionHostPortCache.get(context);
                    final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
                    method.setAccessible(true);
                    method.invoke(cache, String.format("%s:%s", socket.getInetAddress().getHostName(),
                            String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT), session);
                    method.invoke(cache, String.format("%s:%s", socket.getInetAddress().getHostAddress(),
                            String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT), session);
                }
                catch(NoSuchFieldException e) {
                    // Not running in expected JRE
                    log.warn("No field sessionHostPortCache in SSLSessionContext", e);
                }
                catch(Exception e) {
                    // Not running in expected JRE
                    log.warn(e.getMessage());
                }
            }
            else {
                log.warn(String.format("SSL session %s for socket %s is not rejoinable", session, socket));
            }
        }
    }
}

It seems that the _prepareDataSocket_method was added to Commons Net FTPSClientspecifically to allow the TLS/SSL session reuse implementation:
https://issues.apache.org/jira/browse/NET-426
A native support for the reuse is still pending:
https://issues.apache.org/jira/browse/NET-408
You will obviously need to override the Spring Integration DefaultFtpsSessionFactory.createClientInstance()to return your custom FTPSClientimplementation with the session reuse support.

https://trac.cyberduck.io/ticket/5087- 在数据连接上重用会话密钥

查看它的FTPClient.java代码（扩展 Commons Net FTPSClient），特别是它对method 的覆盖_prepareDataSocket_：

@Override
protected void _prepareDataSocket_(final Socket socket) throws IOException {
    if(preferences.getBoolean("ftp.tls.session.requirereuse")) {
        if(socket instanceof SSLSocket) {
            // Control socket is SSL
            final SSLSession session = ((SSLSocket) _socket_).getSession();
            if(session.isValid()) {
                final SSLSessionContext context = session.getSessionContext();
                context.setSessionCacheSize(preferences.getInteger("ftp.ssl.session.cache.size"));
                try {
                    final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
                    sessionHostPortCache.setAccessible(true);
                    final Object cache = sessionHostPortCache.get(context);
                    final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
                    method.setAccessible(true);
                    method.invoke(cache, String.format("%s:%s", socket.getInetAddress().getHostName(),
                            String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT), session);
                    method.invoke(cache, String.format("%s:%s", socket.getInetAddress().getHostAddress(),
                            String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT), session);
                }
                catch(NoSuchFieldException e) {
                    // Not running in expected JRE
                    log.warn("No field sessionHostPortCache in SSLSessionContext", e);
                }
                catch(Exception e) {
                    // Not running in expected JRE
                    log.warn(e.getMessage());
                }
            }
            else {
                log.warn(String.format("SSL session %s for socket %s is not rejoinable", session, socket));
            }
        }
    }
}

似乎该_prepareDataSocket_方法是FTPSClient专门添加到 Commons Net 中以允许 TLS/SSL 会话重用实现的：https:
//issues.apache.org/jira/browse/NET-426
对重用的原生支持仍然悬而未决：https:
//issues.apache.org/jira/browse/NET-408
您显然需要覆盖 Spring IntegrationDefaultFtpsSessionFactory.createClientInstance()以返回FTPSClient具有会话重用支持的自定义实现。

The above solution does not work on its own anymore since JDK 8u161.

自 JDK 8u161 起，上述解决方案不再单独工作。

According to JDK 8u161 Update Release Notes(and the answer by @Laurent):

根据JDK 8u161 更新发行说明（以及@Laurent的回答）：

Added TLS session hash and extended master secret extension support
...
In case of compatibility issues, an application may disable negotiation of this extension by setting the System Property jdk.tls.useExtendedMasterSecretto falsein the JDK

添加了 TLS 会话哈希和扩展的主密钥扩展支持
...
在兼容性问题的情况下，应用程序可以通过在 JDK 中设置系统属性jdk.tls.useExtendedMasterSecret来禁用此扩展的协商false

I.e., you can call this to fix the problem:

即，您可以调用它来解决问题：

System.setProperty("jdk.tls.useExtendedMasterSecret", "false");

Though this should be a considered a workaround only. I do not know a proper solution.

尽管这应该仅被视为一种解决方法。我不知道一个合适的解决方案。

Answer 2

回答by Laurent Grousset

You can use this SSLSessionReuseFTPSClient class :

您可以使用此 SSLSessionReuseFTPSClient 类：

import java.io.IOException;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.net.Socket;
import java.util.Locale;

import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSessionContext;
import javax.net.ssl.SSLSocket;

import org.apache.commons.net.ftp.FTPSClient;

public class SSLSessionReuseFTPSClient extends FTPSClient {

    // adapted from:
    // https://trac.cyberduck.io/browser/trunk/ftp/src/main/java/ch/cyberduck/core/ftp/FTPClient.java
    @Override
    protected void _prepareDataSocket_(final Socket socket) throws IOException {
        if (socket instanceof SSLSocket) {
            // Control socket is SSL
            final SSLSession session = ((SSLSocket) _socket_).getSession();
            if (session.isValid()) {
                final SSLSessionContext context = session.getSessionContext();
                try {
                    final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
                    sessionHostPortCache.setAccessible(true);
                    final Object cache = sessionHostPortCache.get(context);
                    final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
                    method.setAccessible(true);
                    method.invoke(cache, String
                            .format("%s:%s", socket.getInetAddress().getHostName(), String.valueOf(socket.getPort()))
                            .toLowerCase(Locale.ROOT), session);
                    method.invoke(cache, String
                            .format("%s:%s", socket.getInetAddress().getHostAddress(), String.valueOf(socket.getPort()))
                            .toLowerCase(Locale.ROOT), session);
                } catch (NoSuchFieldException e) {
                    throw new IOException(e);
                } catch (Exception e) {
                    throw new IOException(e);
                }
            } else {
                throw new IOException("Invalid SSL Session");
            }
        }
    }
}

And With openJDK 1.8.0_161 :

并使用 openJDK 1.8.0_161 ：

We must set :

我们必须设置：

System.setProperty("jdk.tls.useExtendedMasterSecret", "false");

according to http://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html

根据 http://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html

Added TLS session hash and extended master secret extension support

添加了 TLS 会话哈希和扩展的主密钥扩展支持

In case of compatibility issues, an application may disable negotiation of this extension by setting the System Property jdk.tls.useExtendedMasterSecret to false in the JDK

在兼容性问题的情况下，应用程序可以通过在 JDK 中将系统属性 jdk.tls.useExtendedMasterSecret 设置为 false 来禁用此扩展的协商

Answer 3

回答by NotX

To make Martin Prikryl's suggestion work for me I had to store the key not only under socket.getInetAddress().getHostName()but also under socket.getInetAddress().getHostAddress(). (Solution stolen from here.)

为了让 Martin Prikryl 的建议对我有用，我不仅必须将密钥存储socket.getInetAddress().getHostName()在socket.getInetAddress().getHostAddress(). （从这里窃取的解决方案。）

java 如何使用相同的 TLS 会话通过数据连接连接到 FTPS 服务器？

提问by Martin Prikryl

回答by Martin Prikryl

回答by Laurent Grousset

回答by NotX

相关推荐

最近更新

标签

java 如何使用相同的 TLS 会话通过数据连接连接到 FTPS 服务器？

提问by Martin Prikryl

回答by Martin Prikryl

回答by Laurent Grousset

回答by NotX

相关推荐

java 使用 JPA 的表排他锁

java 如何在 Intellij 中启用 CSS 编辑

java 将外部资源文件夹添加到 Spring Boot

java 在 JavaFX Spinner 中手动输入文本不会更新值（除非用户按 ENTER）

相关推荐

最近更新

标签