使文件传输更高效 Java
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/7500059/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Making file transfer more efficient Java
提问by David
I have two wireless computers connected to an N wireless router. Each of these PCs are connected at between 108-150Mbps.
我有两台无线计算机连接到 N 无线路由器。这些 PC 中的每一个都以 108-150Mbps 的速度连接。
Theoretically, I should be able to transfer at 13.5MB/s to 18.75MB/s, under the absolute best of conditions.
从理论上讲，在绝对最佳条件下，我应该能够以 13.5MB/s 到 18.75MB/s 的速度进行传输。
The first computer (that is sending), uses a very fast SSD, which is around 100MB/s if I remember correctly. CPU usage also stays below 20%.
第一台计算机（正在发送）使用非常快的 SSD，如果我没记错的话大约是 100MB/s。CPU 使用率也保持在 20% 以下。
It sent 1960273535 bytes (1.8GB) in 656367ms. That's 2.8MB/s (22 out of 108 Megabits). When I open up task manager, I see that only 25-27% of the network connection is being used.
它在 656367 毫秒内发送了 1960273535 个字节（1.8GB）。那是 2.8MB/s（108 兆位中的 22 位）。当我打开任务管理器时，我看到只有 25-27% 的网络连接正在使用。
I am looking for any ideas, suggestions, or improvements that can make the transfer faster (over a network). I was thinking of buffering the file from the disk on a thread and sending the buffered data from another thread but I'm not sure if it's a good idea. Here is the SSCCE:
我正在寻找可以加快传输速度（通过网络）的任何想法、建议或改进。我正在考虑在一个线程上从磁盘缓冲文件并从另一个线程发送缓冲数据，但我不确定这是否是个好主意。这是 SSCCE：
Host:
主持人：
import java.io.*;
import java.net.*;
public class Host {
public static void main(String[] args) throws IOException {
ServerSocket servsock = new ServerSocket(15064);
Socket sock = servsock.accept();
long time = System.currentTimeMillis();
OutputStream out = sock.getOutputStream();
FileInputStream fileInputStream = new FileInputStream("C:\complete.rar");
byte [] buffer = new byte[64*1024];
int bytesRead = 0;
long totalSent = 0;
while ( (bytesRead = fileInputStream.read(buffer)) != -1)
{
if (bytesRead > 0)
{
out.write(buffer, 0, bytesRead);
totalSent += bytesRead;
System.out.println("sent " + totalSent);
}
}
sock.close();
System.out.println("Sent " + totalSent + " bytes in "
+ (System.currentTimeMillis() - time) + "ms.");
}
}
Client:
客户：
import java.io.*;
import java.net.*;
public class Client {
public static void main(String[] args) throws Exception {
Socket sock = new Socket("127.0.0.1", 15064);
InputStream in = sock.getInputStream();
FileOutputStream fileOutputStream = new FileOutputStream("output.rar");
byte [] buffer = new byte[64*1024];
int bytesRead = 0;
while ( (bytesRead = in.read(buffer)) != -1)
fileOutputStream.write(buffer, 0, bytesRead);
sock.close();
fileOutputStream.close();
}
}
Edit: I tried mapping a network drive and sending the file over that, and windows did even worse - 2.35MB/s. According to this article http://tinyurl.com/634qaqgmapping a network drive is faster than FTP, and I also don't have the time to stay playing around and setting up the FTP server.
编辑：我尝试映射网络驱动器并通过该驱动器发送文件，Windows 的表现更糟 - 2.35MB/s。根据这篇文章http://tinyurl.com/634qaqg映射网络驱动器比 FTP 快，而且我也没有时间继续玩和设置 FTP 服务器。
Edit2: After changing the timer, turns out it was transferring at 3MB/s over WiFi. I hate the "theoretical" throughput. When I buy something, I want to know it's REAL performance. It turns out the code is indeed limited by WiFi speeds. I am still open to suggestions though.
Edit2：更改计时器后，结果是通过 WiFi 以 3MB/s 的速度传输。我讨厌“理论”吞吐量。当我买东西时，我想知道它的真实性能。事实证明，代码确实受到 WiFi 速度的限制。不过，我仍然愿意接受建议。
Edit 3: After running the program on 100Mbps LAN, it managed to transfer the file at 11.8MB/s. That's pretty good considering that the maximum transfer rate is 12.5MB/s.
编辑 3：在 100Mbps LAN 上运行程序后，它设法以 11.8MB/s 的速度传输文件。考虑到最大传输速率为 12.5MB/s，这已经相当不错了。
采纳答案by NPE
At 2.8MB/s, it is unlikely that the slowness has anything to do with your code. It is almost certainly due to the wireless network not being able to achieve the theoretical throughput (possibly due to environmental conditions).
在 2.8MB/s 时，缓慢不太可能与您的代码有关。几乎可以肯定是由于无线网络无法达到理论吞吐量（可能是由于环境条件）。
It's easy to test whether this is the case: simply time a large ftpor scpfile transfer between the same two computers and see what kind of throughput you're seeing.
测试是否是这种情况很容易：只需对相同的两台计算机之间的大型ftp或scp文件传输进行计时，然后查看您看到的吞吐量类型。
回答by Peter Lawrey
I suggest you try the following code which prints
我建议你试试下面的打印代码
Wed Oct 26 14:21:03 BST 2011: Accepted a connection
Wed Oct 26 14:21:13 BST 2011: Transfer rate was 3212.5 MB/s
on the server and on the client prints
在服务器和客户端打印
Wed Oct 26 14:21:03 BST 2011 Sending for 10.0 seconds.
Wed Oct 26 14:21:13 BST 2011 ... sent.
Wed Oct 26 14:21:13 BST 2011 ... received 33691287552
Send and received 3212.8 MB/s
Note: the total amount transferred is double this as everything sent client to server is sent server to client.
注意：传输的总金额是这个的两倍，因为从客户端发送到服务器的所有内容都是从服务器发送到客户端的。
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.nio.ByteBuffer;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Date;
public class EchoServerMain {
public static void main(String... args) throws IOException {
int port = args.length < 1 ? 55555 : Integer.parseInt(args[0]);
ServerSocketChannel ss = ServerSocketChannel.open();
ss.socket().bind(new InetSocketAddress(port));
while (!ss.socket().isClosed()) {
SocketChannel s = ss.accept();
System.out.println(new Date() + ": Accepted a connection");
long start = System.nanoTime();
ByteBuffer bytes = ByteBuffer.allocateDirect(32*1024);
int len;
long total = 0;
// Thank you @EJP, for a more elegant single loop.
while ((len = s.read(bytes)) >= 0 || bytes.position() > 0) {
bytes.flip();
s.write(bytes);
bytes.compact();
total += len;
}
long time = System.nanoTime() - start;
System.out.printf(new Date() + ": Transfer rate was %.1f MB/s%n", total * 1e9 / 1024 / 1024 / time);
}
ss.close();
}
}
and
和
import java.io.EOFException;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.UnknownHostException;
import java.nio.ByteBuffer;
import java.nio.channels.SocketChannel;
import java.util.Date;
public class EchoClientMain {
public static void main(String ... args) throws IOException {
String hostname = args.length < 1 ? "localhost" : args[0];
int port = args.length < 2 ? 55555 : Integer.parseInt(args[1]);
double seconds = args.length < 3 ? 10 : Double.parseDouble(args[2]);
SocketChannel s = SocketChannel.open(new InetSocketAddress(hostname, port));
s.configureBlocking(false);
ByteBuffer bytes = ByteBuffer.allocateDirect(32*1024);
System.out.printf(new Date()+ " Sending for %.1f seconds.%n", seconds);
long start = System.nanoTime();
long dataSent = 0, dataReceived = 0;
// run for 10 seconds.
while(start + seconds*1e9 > System.nanoTime()) {
bytes.clear();
int wlen = s.write(bytes);
if (wlen < 0) throw new IOException();
dataSent += wlen;
bytes.clear();
int rlen = s.read(bytes);
if (rlen < 0) throw new EOFException();
dataReceived += rlen;
}
System.out.println(new Date()+ " ... sent.");
while(dataReceived < dataSent) {
bytes.clear();
int rlen = s.read(bytes);
if (rlen < 0) throw new EOFException();
dataReceived += rlen;
}
s.close();
long time = System.nanoTime() - start;
System.out.println(new Date()+ " ... received "+dataReceived);
System.out.printf("Send and received %.1f MB/s%n", dataReceived * 1e9/1024/1024/time);
}
}
回答by Piskvor left the building
Test the same program using a wired, Fast Ethernet (100Mbit/s) link between the computers (and then possibly using a 1Gbit link). That way, you'll see whether the transfer rate is actually limited by your program or by the link.
使用计算机之间的有线快速以太网 (100Mbit/s) 链接（然后可能使用 1Gbit 链接）测试相同的程序。这样，您将看到传输速率实际上是否受到程序或链接的限制。
回答by user207421
Just set a very large socket send buffer, and if possible set a very large socket receive buffer at the receiver. Code 'optimizations' contribute basically nothing to these scenarios.
只需设置一个非常大的套接字发送缓冲区，并在可能的情况下在接收器处设置一个非常大的套接字接收缓冲区。代码“优化”对这些场景基本上没有任何贡献。
回答by Peter
Your timer is wrong ! you should start it after you accept the connection not when you start the host
你的计时器错了！您应该在接受连接后启动它，而不是在启动主机时启动它
Did you try increasing your buffer size ?
您是否尝试增加缓冲区大小？
