.net 如何提高 FtpWebRequest 的性能？

How to improve the Performance of FtpWebRequest?

提问by A9S6

I have an application written in .NET 3.5 that uses FTP to upload/download files from a server. The app works fine but there are performance issues:

我有一个用 .NET 3.5 编写的应用程序，它使用 FTP 从服务器上传/下载文件。该应用程序运行良好，但存在性能问题：

1. It takes a lot of time to make connection to the FTP server. The FTP server is on a different network and has Windows 2003 Server (IIS FTP). When multiple files are queued for upload, the change from one file to another creates a new connection using FTPWebRequest and it takes a lot of time (around 8-10 seconds).

2. Is is possible to re-use the connection? I am not very sure about the KeepAlive property. Which connections are kept alive and reused.

3. The IIS-FTP on Windows Server 2003 does not support SSL so anyone can easily see the username/password through a packet sniffer such as WireShark. I found that windows Server 2008 supports SSL over FTP in its new version if IIS 7.0.

1. 连接到 FTP 服务器需要很多时间。FTP 服务器位于不同的网络上，并且具有 Windows 2003 Server (IIS FTP)。当多个文件排队等待上传时，从一个文件到另一个文件的更改会使用 FTPWebRequest 创建一个新连接，这需要很长时间（大约 8-10 秒）。

2. 是否可以重新使用连接？我不太确定 KeepAlive 属性。哪些连接保持活动并重用。

3. Windows Server 2003 上的 IIS-FTP 不支持 SSL，因此任何人都可以通过数据包嗅探器（例如 WireShark）轻松查看用户名/密码。我发现 Windows Server 2008 在其新版本中支持 SSL over FTP，如果 IIS 7.0.

I basically want to improve the upload/download performance of my application. Any ideas will be appreciated.

我基本上想提高我的应用程序的上传/下载性能。任何想法将不胜感激。

** Please note that 3 is not an issue but I would like people to have comments on it

** 请注意，3 不是问题，但我希望人们对此发表评论

回答by Syd

I have done some experimentation (uploading about 20 files on various sizes) on FtpWebRequest with the following factors

我在 FtpWebRequest 上做了一些实验（上传大约 20 个不同大小的文件），有以下因素

KeepAlive= true/false

KeepAlive= 真/假

ftpRequest.KeepAlive = isKeepAlive;

Connnection Group Name= UserDefined or null

连接组名称= UserDefined 或 null

ftpRequest.ConnectionGroupName = "MyGroupName";

Connection Limit= 2 (default) or 4 or 8

连接限制= 2（默认）或 4 或 8

ftpRequest.ServicePoint.ConnectionLimit = ConnectionLimit;

Mode= Synchronous or Async

模式= 同步或异步

see this example

看这个例子

My results:

我的结果：

1. Use ConnectionGroupName,KeepAlive=true took (21188.62 msec)

2. Use ConnectionGroupName,KeepAlive=false took (53449.00 msec)

3. No ConnectionGroupName,KeepAlive=false took (40335.17 msec)

4. Use ConnectionGroupName,KeepAlive=true;async=true,connections=2 took (11576.84 msec)

5. Use ConnectionGroupName,KeepAlive=true;async=true,connections=4 took (10572.56 msec)

6. Use ConnectionGroupName,KeepAlive=true;async=true,connections=8 took (10598.76 msec)

1. 使用 ConnectionGroupName,KeepAlive=true 花了（21188.62 毫秒）

2. 使用 ConnectionGroupName,KeepAlive=false 花了（53449.00 毫秒）

3. 没有 ConnectionGroupName,KeepAlive=false (40335.17 毫秒)

4. 使用 ConnectionGroupName,KeepAlive=true;async=true,connections=2 花了（11576.84 毫秒）

5. 使用 ConnectionGroupName,KeepAlive=true;async=true,connections=4 花了（10572.56 毫秒）

6. 使用 ConnectionGroupName,KeepAlive=true;async=true,connections=8 花了（10598.76 毫秒）

Conclusions

结论

1. FtpWebRequesthas been designed to support an internal connection pool. To ensure, the connection pool is used, we must make sure the ConnectionGroupNameis being set.

2. Setting up a connection is expensive. If we are connecting to the same ftp server using the same credentials, having the keep alive flag set to true will minimize the number of connections setup.

3. Asynchronous is the recommended way if you have a lot of files to ftp.

4. The default number of connections is 2. In my environment, a connection limit of 4 will give to me the most overall performance gain. Increasing the number of connections may or may not improve performance. I would recommend that you have the connection limit as a configuration parameter so that you can tune this parameter in your environment.

1. FtpWebRequest旨在支持内部连接池。为了确保使用连接池，我们必须确保ConnectionGroupName正在设置。

2. 建立连接的成本很高。如果我们使用相同的凭据连接到同一个 ftp 服务器，将 keep alive 标志设置为 true 将最小化连接设置的数量。

3. 如果你有很多文件要 ftp，异步是推荐的方式。

4. 默认连接数为 2。在我的环境中，连接限制为 4 将为我带来最大的整体性能提升。增加连接数可能会也可能不会提高性能。我建议您将连接限​​制作为配置参数，以便您可以在您的环境中调整此参数。

Hope you would find this useful.

希望你会发现这很有用。

回答by Remus Rusanu

It doesn't matter if the individual connections take long to connect as long as you can launch many in parallel. If you have many items to transfer (say hundreds) then it makes sense to launch tens and even hundreds of WebRequests in parallel, using the asynchronous methods like BeginGetRequestStreamand BeginGetResponse. I worked on projects that faced similar problems (long connect/authenticate times) but by issuing many calls in parallel the overall throughput was actually very good.

只要您可以并行启动多个连接，单个连接是否需要很长时间连接都没有关系。如果您有许多项目要传输（比如数百个），那么使用BeginGetRequestStream和BeginGetResponse等异步方法并行启动数十个甚至数百个 WebRequest 是有意义的。我从事过面临类似问题（连接/身份验证时间长）的项目，但通过并行发出许多调用，整体吞吐量实际上非常好。

Also it makes a huge difference if you use the async methodsor the synchronous one, as soon as you have many (tens, hundreds) of requests. This applies not only to your WebRequests methods, but also to your Stream read/writemethods you'll use after obtaining the upload/download stream. The Improving .Net Performance and Scalabilitybook is a bit outdated, but much of its advice still stands, and is free to read online.

如果您使用异步方法或同步方法，一旦您有许多（数十、数百）个请求，它也会产生巨大的差异。这不仅适用于您的 WebRequests 方法，还适用于您在获取上传/下载流后将使用的Stream读/写方法。《提高 .Net 性能和可扩展性》一书有点过时，但其中的大部分建议仍然有效，并且可以在线免费阅读。

One thing to consider is that the ServicePointManagerclass sits there lurking in the Framwework with one sole purpose: to ruin your performance. Make sure you obtain the ServicePointof your URL and change the ConnectionLimitto a reasonable value (at least as high as how many concurrent requests you intend).

需要考虑的一件事是ServicePointManager类潜伏在框架中，其目的只有一个：破坏您的性能。确保您获得了您的 URL的 ServicePoint并将ConnectionLimit更改为一个合理的值（至少与您想要的并发请求数一样高）。

回答by Dave Jarvis

Debug Network

调试网络

A few tricks for simple network debugging:

简单网络调试的几个技巧：

1. Check the response times when you ping the FTP server from the application server.
2. Check the response times for a trace route (tracertfrom a DOS shell).
3. Transfer a file from the command-line using the ftpcommand.
4. Connect to the FTP server via Telnet: telnet server 21.

1. 检查从应用程序服务器 ping FTP 服务器时的响应时间。
2. 检查跟踪路由的响应时间（tracert从 DOS shell）。
3. 使用ftp命令从命令行传输文件。
4. 通过 Telnet 连接到 FTP 服务器：telnet server 21。

The results will provide clues to solving the problem.

结果将为解决问题提供线索。

Network Hardware

网络硬件

For a slow trace route:

对于慢跟踪路由：

• Determine why the two computers are having network issues.
• Upgrade the network hardware between the slowest link.

• 确定两台计算机出现网络问题的原因。
• 升级最慢链路之间的网络硬件。

Network Configuration

网络配置

For a slow ping:

对于慢 ping：

• Check the network configuration on each machine.
• Ensure the settings are optimal.

• 检查每台机器上的网络配置。
• 确保设置最佳。

Validate API

验证 API

A slow command-line FTP session will tell you that the problem is not isolated to the FTP API you are using. It does not eliminate the API as a potential problem, but certainly makes it less likely.

缓慢的命令行 FTP 会话会告诉您问题并非与您使用的 FTP API 无关。它不会消除 API 作为潜在问题，但肯定会降低它的可能性。

Network Errors

网络错误

If packets are being dropped between the source and destination, ping will tell you. You might have to increase the packet size to 1500 bytes to see any errors.

如果数据包在源和目标之间被丢弃，ping 会告诉你。您可能需要将数据包大小增加到 1500 字节才能看到任何错误。

FTP Queue Server

FTP 队列服务器

If you have no control over the destination FTP server, have an intermediary server receive uploaded files. The intermediary then sends the files to the remote server at whatever speed it can. This gives the illusion that the files are being sent quickly. However, if the files must exist on the remote server as soon as they are uploaded, then this solution might not be viable.

如果您无法控制目标 FTP 服务器，请让中间服务器接收上传的文件。然后中介以任何速度将文件发送到远程服务器。这给人一种文件正在快速发送的错觉。但是，如果文件一上传就必须存在于远程服务器上，则此解决方案可能不可行。

FTP Server Software

FTP 服务器软件

Use a different FTP daemon on the FTP server, such as ProFTPdas a Windows service. (ProFTPd has plug-ins for various databases that allow authentication using SQL queries.)

在 FTP 服务器上使用不同的 FTP 守护程序，例如ProFTPd作为Windows 服务。（ProFTPd 有各种数据库的插件，允许使用 SQL 查询进行身份验证。）

FTP Server Operating System

FTP 服务器操作系统

A Unix-based operating system might be a better option than a Microsoft-based one.

基于 Unix 的操作系统可能比基于 Microsoft 的操作系统更好。

FTP Client Software

FTP客户端软件

There are a number of different APIs for sending and receiving files via FTP. It might take some work to make your application modular enough that you can simply plug in a new file transfer service. A few different APIs are listed as answers here.

有许多不同的 API 可用于通过 FTP 发送和接收文件。可能需要一些工作才能使您的应用程序足够模块化，以便您可以简单地插入新的文件传输服务。此处列出了一些不同的 API 作为答案。

Alternate Protocol

替代协议

If FTP is not an absolute requirement, try:

如果 FTP 不是绝对要求，请尝试：

• a Windows network drive
• HTTPS
• scp, rsync, or similar programs (Cygwinmight be required)

• Windows 网络驱动器
• HTTPS
• scp、rsync 或类似程序（可能需要Cygwin）

回答by Salar

This link describes ConnectionGroupName and KeepAlive affects: WebRequest ConnectionGroupName

此链接描述了 ConnectionGroupName 和 KeepAlive 的影响： WebRequest ConnectionGroupName

回答by Josh

You should definitely check out BITSwhich is a big improvement over FTP. The clear-text passwords aren't the only weakness in FTP. There's also the issue of predicting the port it will open for a passive upload or download and just overall difficulty when clients are using NAT or firewalls.

您绝对应该查看BITS，这是对 FTP 的重大改进。明文密码并不是 FTP 的唯一弱点。还有一个问题是预测它将为被动上传或下载打开的端口以及客户端使用 NAT 或防火墙时的整体困难。

BITS works over HTTP/HTTPS using IIS extensions and supports queued uploads and downloads that can be scheduled at low priority. It's overall just a lot more flexible than FTP if you are using Windows on the client and server.

BITS 使用 IIS 扩展通过 HTTP/HTTPS 工作，并支持可按低优先级安排的排队上传和下载。如果您在客户端和服务器上使用 Windows，它总体上比 FTP 灵活得多。

BITS for PowerShell

PowerShell 的 BITS

BITS for .NET

.NET 位

回答by shahkalpesh

Look at this page - http://www.ietf.org/rfc/rfc959.txt

看看这个页面 - http://www.ietf.org/rfc/rfc959.txt

It says of using different port when connecting to be able to reuse the connection.
Does that work?

它说连接时使用不同的端口能够重用连接。
那样有用吗？

回答by tomfanning

Personally I have migrated all of our apps away from using FTP for file upload/download, and instead rolled a solution based on XML Web Services in ASP.NET.

我个人已经将我们所有的应用程序从使用 FTP 进行文件上传/下载迁移，而是在 ASP.NET 中推出了基于 XML Web 服务的解决方案。

Performance is much improved, security is as much or as little as you want to code (and you can use the stuff built in to .NET) and it can all go over SSL with no issues.

性能大大提高，安全性与您想编码的一样多或少（并且您可以使用 .NET 中内置的东西）并且它可以毫无问题地通过 SSL。

Our success rate getting our clients' connections out through their own firewalls is FAR better than running FTP.

我们通过他们自己的防火墙让我们的客户连接出去的成功率比运行 FTP 要好得多。

回答by Max Toro

I strongly suggest Starksoft FTP/FTPS Component for .NET and Mono. It has a connection object that you can cache and reuse.

我强烈建议Starksoft FTP/FTPS Component for .NET 和 Mono。它有一个可以缓存和重用的连接对象。

回答by paolo Giusti

To resolve the problem about performance you simply need to set:

要解决有关性能的问题，您只需设置：

ftpRequest.ConnectionGroupName = "MyGroupName"; ftpRequest.KeepAlive = false; ftpRequest.ServicePoint.CloseConnectionGroup("MyGroupName");

ftpRequest.ConnectionGroupName = "MyGroupName"; ftpRequest.KeepAlive = false; ftpRequest.ServicePoint.CloseConnectionGroup("MyGroupName");

回答by paolo Giusti

I know it's an old thread, but I recently had to go through a similar situation.

我知道这是一个旧线程，但我最近不得不经历类似的情况。

We needed to download 70+ XML files from an ftp server in under 25 minutes without opening more than 5 connections during that time-frame.

我们需要在 25 分钟内从 ftp 服务器下载 70 多个 XML 文件，并且在这段时间内打开的连接不超过 5 个。

These were all the alternatives we tried:

这些是我们尝试过的所有替代方案：

• wget - It was fast, but every GET meant a new connection. We had to stop due to the amount of connections created. We were having some issues with GETM that's well-documented in the web so that wasn't an option.
• FtpWebRequest - Every Create call would log a new connection, even though we used the KeepAlive and ConnectionGroupName properties. Plus it was very slow.
• Webclient - We didn't check if it created a new connection for every file (although I assume it does), but it copied 1 file/minute. So it wasn't an option.

• wget - 速度很快，但每个 GET 都意味着一个新的连接。由于创建的连接数量，我们不得不停下来。我们在使用 GETM 时遇到了一些问题，这些问题在网络上有详细记录，所以这不是一个选项。
• FtpWebRequest - 每个 Create 调用都会记录一个新连接，即使我们使用了 KeepAlive 和 ConnectionGroupName 属性。此外，它非常缓慢。
• Webclient - 我们没有检查它是否为每个文件创建了一个新连接（尽管我认为它确实如此），但它复制了 1 个文件/分钟。所以这不是一个选择。

We ended up using old-fashioned ftp batch script. It's fast and I only use one connection to download all the files. It isn't flexible, but it's much faster than everything else I've tried (75 files in under 20 minutes).

我们最终使用了老式的 ftp 批处理脚本。它很快，我只使用一个连接来下载所有文件。它不灵活，但比我尝试过的所有其他方法都要快得多（20 分钟内 75 个文件）。