Linux 通过 UDP 套接字发送缓冲区时延迟有时会增加

Delay increases sometimes when sending buffer via UDP socket

提问by Helder AC

I've developed an MPEG-ts Streamer. It reads the packets from a file and sends them at the right pace to the receiver.

我开发了一个 MPEG-ts Streamer。它从文件中读取数据包，并以正确的速度将它们发送到接收器。

Now everything works fine excluding that I have some lags quite often. I have searched for every possible error in my code. I have optimized my program quite a bit performance wise.

现在一切正常，除了我经常有一些滞后。我在我的代码中搜索了所有可能的错误。我已经对我的程序进行了相当多的性能优化。

Now I keep a log with the time it takes the sendto()function to send the packet and I also log the difference between when the packet should have been sent and when it is sent.

现在，我记录了sendto()函数发送数据包所需的时间，并记录了应该发送数据包的时间和发送数据包的时间之间的差异。

I noticed that every time the packets are a lot later than average, the time it took the sendto()to send the previous packet is much higher than normal too.

我注意到每次数据包比平均值晚很多时sendto()，发送前一个数据包所花费的时间也比正常情况高得多。

This indicates me that it is the sendto()that is causing those lags every time it somehow takes longer to send a packet. I am using UDP sockets.

这表明我sendto()每次发送数据包需要更长的时间时都会导致这些滞后。我正在使用 UDP 套接字。

Am I maybe doing something wrong with the sockets? Could it be possible that the socket buffer is full and that it actually takes longer to send the packet? Or am I missing something? Is there maybe a way to accelerate the socket or to make it not fill it's buffer completely before sending?

我可能在插座上做错了什么吗？是否有可能套接字缓冲区已满并且发送数据包实际上需要更长的时间？或者我错过了什么？有没有办法加速套接字或让它在发送之前不完全填满它的缓冲区？

As this is meant to stream video I am quite depending on the performance, mostly for HD where the lags happen more often as the amount of packets is much higher.

因为这是为了流式传输视频，所以我非常依赖于性能，主要是对于 HD，因为数据包的数量要高得多，所以延迟发生得更频繁。

采纳答案by Seth Noble

There are only two reasons for sendto()to block:

sendto()阻塞的原因只有两个：

• Your outgoing UDP buffer is full, and it needs to wait until space frees up
• The CPU scheduler simply decides to do something else for a while, as it can with any syscall.

• 您的传出 UDP 缓冲区已满，需要等待空间释放
• CPU 调度程序只是决定在一段时间内做其他事情，就像它可以处理任何系统调用一样。

Check the size of your outgoing buffer:

检查输出缓冲区的大小：

int buff_size;
int len = sizeof(buff_size);

err = getsockopt(s,SOL_SOCKET,SO_SNDBUF,(char*)&size,&len);

Some linux systems default to absurdly small send buffers (only a few kilobytes), so you may need to set it to something larger.

一些 linux 系统默认发送缓冲区非常小（只有几千字节），因此您可能需要将其设置为更大的缓冲区。

If buffer is larger and sendto()is briefly blocking anyway (how long exactly?) then that's probably just the cost of doing business. Even on a LAN, and certainly on a WAN, latency is going to vary a lot.

如果缓冲区更大并且sendto()无论如何都会短暂阻塞（具体多久？），那么这可能只是做生意的成本。即使在 LAN 上，当然在 WAN 上，延迟也会有很大差异。

Update

更新

To increase the system limit on UDP buffer sizes:

要增加系统对 UDP 缓冲区大小的限制：

sysctl -w net.core.wmem_max=1048576
sysctl -w net.core.rmem_max=1048576

You can make this permanent by adding the following lines to /etc/sysctl.conf:

您可以通过将以下行添加到以下行来使其永久化/etc/sysctl.conf：

net.core.wmem_max=1048576
net.core.rmem_max=1048576

To take advantage of this within your application, you need to use setsockopt():

要在您的应用程序中利用这一点，您需要使用setsockopt()：

int len, trysize, gotsize;
len = sizeof(int);
trysize = 1048576+32768;
do {
   trysize -= 32768;
   setsockopt(s,SOL_SOCKET,SO_SNDBUF,(char*)&trysize,len);
   err = getsockopt(s,SOL_SOCKET,SO_SNDBUF,(char*)&gotsize,&len);
   if (err < 0) { perror("getsockopt"); break; }
} while (gotsize < trysize);
printf("Size set to %d\n",gotsize);

Repeat the same thing for SO_RCVBUF. The loop is important because many systems silently enforce a maximum which is less than the max you set with sysctland when setsockopt()fails, it leaves the previous value untouched. So you have to try many different values until you get one that sticks. Its also important to not test for (gotsize == trysize)because on some systems the result which is set is not actually the same as the one you requested.

对 重复同样的事情SO_RCVBUF。循环很重要，因为许多系统会默默地强制执行一个最大值，该最大值小于您设置的最大值，sysctl并且当setsockopt()失败时，它会保持先前的值不变。所以你必须尝试许多不同的值，直到你得到一个合适的值。不测试也很重要，(gotsize == trysize)因为在某些系统上，设置的结果实际上与您请求的结果不同。

回答by m0skit0

Impossible to say without the code, but here are a few hints:

没有代码就不能说，但这里有一些提示：

• It's possible that your code works just fine, and those delays are caused by OS scheduler.
• Usually, optimizing your code too early could actually lead to performance drops. Optimization is last step.

• 您的代码可能运行良好，而这些延迟是由操作系统调度程序引起的。
• 通常，过早优化代码实际上可能会导致性能下降。优化是最后一步。

You can try to "nicer" your process. If this improves performance, it's an OS scheduler issue.

您可以尝试“更好”您的流程。如果这提高了性能，则是操作系统调度程序问题。

回答by Aaron Digulla

[EDIT]Replace the receiver with something that has a buffer. Or maybe try to send more data than the receiver really needs - it really should have a buffer already. I doubt that there is anyone in the world who builds set-top boxes andwho doesn't know that there are network lags andwho isn't bankrupt already.

[编辑]用具有缓冲区的东西替换接收器。或者尝试发送比接收者真正需要的更多的数据——它真的应该已经有一个缓冲区了。我怀疑世界上是否有人制造机顶盒，但谁不知道网络滞后，谁还没有破产。

From your explanation, I doubt that the cause is in your code. There are many reasons why it can be slower:

根据您的解释，我怀疑原因出在您的代码中。速度变慢的原因有很多：

• The OS might decide to give another process control since you're doing I/O.
• The network buffer might be full
• The network might be congested
• The virus scanner might be interested in the same data that you're reading and steal too many cycles.
• If you're on Windows: The Aero desktop renders something (the IRQ of the graphics card usually has a higher priority than that of the network card).

• 由于您正在进行 I/O，操作系统可能会决定给予另一个进程控制。
• 网络缓冲区可能已满
• 网络可能拥塞
• 病毒扫描程序可能对您正在阅读的相同数据感兴趣并窃取太多周期。
• 如果您使用的是 Windows：Aero 桌面会渲染一些东西（显卡的 IRQ 通常比网卡的优先级更高）。

Conclusion: Your problem is that the receiver has no buffer. There are really very many reasons that can cause a delay in the sending of the UDP packets and all of them are not under the control of your code.

结论：您的问题是接收器没有缓冲区。确实有很多原因会导致 UDP 数据包的发送延迟，并且所有这些原因都不受您的代码的控制。

---

---

Original Answer

原答案

UDP is not a reliable protocol. There is no guarantee that packets arrive at the receiver in time. UDP just has a smaller overhead than TCP but that comes at the cost of reliability.

UDP 不是一个可靠的协议。无法保证数据包及时到达接收方。UDP 的开销比 TCP 小，但这是以可靠性为代价的。

The usual solution is to send more data than the client needs. So for example, you send the first 10 seconds at full throttle. This allows the client to cache some data to play over lags.

通常的解决方案是发送比客户端需要更多的数据。例如，您在全速发送前 10 秒。这允许客户端缓存一些数据以在延迟上播放。

You should also change the client to send back the length of the cached stream to the server every few seconds. The server can then calculate how many packets to send at full throttle to keep the cache on the client side filled.

您还应该更改客户端以每隔几秒钟将缓存流的长度发送回服务器。然后服务器可以计算全速发送多少数据包以保持客户端的缓存被填满。

回答by mac

Unlike TCP, UDP does not buffer data. Source code would definitively help to understand things better. How big are the packets you are sending? In UDP it would be best to keep the payload at the size of the MTU (1500 bytes)

与 TCP 不同，UDP 不缓冲数据。源代码绝对有助于更好地理解事物。您发送的数据包有多大？在 UDP 中，最好将有效负载保持在 MTU 的大小（1500 字节）