Javascript 如何设置一个非常快的 node.js UDP 服务器

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/10232717/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-24 00:25:40  来源:igfitidea点击:

How to setup a very fast node.js UDP server

javascriptnode.jsudp

提问by Panthro

I'm quite new to Node.js and I have a request for an application that would receive a payload of UDP packets and process it.

我对 Node.js 很陌生,我有一个应用程序的请求,该应用程序将接收 UDP 数据包的有效负载并对其进行处理。

I'm talking about more than 400 messages per second, which would reach something like 200.000 messages/minute.

我说的是每秒 400 多条消息,这将达到大约 200.000 条消息/分钟。

I have written a code to setup a UDP server (grabbed from docs here http://nodejs.org/api/all.html#all_udp_datagram_socketsactually) but it's losing something around 5% of the packets.

我已经编写了一个代码来设置一个 UDP 服务器(实际上是从这里的文档http://nodejs.org/api/all.html#all_udp_datagram_sockets 中获取的)但是它丢失了大约 5% 的数据包。

What I really need to develop is a server which would get the packet and send it to another workerdo the job with the message. But looks like threading in node.js is a nightmare.

我真正需要开发的是一个服务器,它可以获取数据包并将其发送给另一个处理消息的工作人员。但看起来 node.js 中的线程是一场噩梦。

This is my core as is:

这是我的核心:

var dgram = require("dgram");
var fs = require("fs");
var stream = fs.createWriteStream("received.json",{ flags: 'w',
  encoding: "utf8",
  mode: 0666 });

var server = dgram.createSocket("udp4");
server.on("message", function (msg, rinfo) {
  console.log("server got: " + msg + " from " +
    rinfo.address + ":" + rinfo.port);
    stream.write(msg);
});

server.on("listening", function () {
  var address = server.address();
  console.log("server listening " +
      address.address + ":" + address.port);
});

server.bind(41234);
// server listening 0.0.0.0:41234

回答by Mustafa

You are missing concepts, NodeJS is not meant to be multi-thread in terms of you mean, requests should be handled in a cycle. No other thread exists so no context-switches happens. In a multi-core environment, you can create a cluster via node's cluster module, I have a blog-post about this here.

您缺少概念,就您而言,NodeJS 并不意味着是多线程的,请求应该在一个循环中处理。不存在其他线程,因此不会发生上下文切换。在多核环境中,您可以通过节点的集群模块创建集群,我在这里有一篇关于此的博客文章。

You set the parent proceses to fork child processes, and ONLY child processes should bind to a port. Your parent proceses will handle the load balancing between children.

您将父进程设置为 fork 子进程,并且只有子进程应该绑定到端口。您的父进程将处理子进程之间的负载平衡。

Note: In my blog post, I made i < os.cpus().length / 2;but it should be i < os.cpus().length;

注意:在我的博客文章中,我做了 i < os.cpus().length / 2;但它应该是 i < os.cpus().length;

回答by J_McCaffrey

I wrote a soap/xml forwarding service with a similar structure, and found that the info would come in 2 packets. I needed to update my code to detect 2 halves of the message and put them back together. This payload size thing may be more of an HTTP issue than a udp issue, but my suggestion is that you add logging to write out everything you are receiving and then go over it with a fine tooth comb. It looks like you would be logging what you are getting now, but you may have to dig into the 5% that you are losing.

我用类似的结构写了一个soap/xml转发服务,发现信息会分成2个数据包。我需要更新我的代码以检测消息的两半并将它们重新组合在一起。这种有效载荷大小的问题可能更像是 HTTP 问题而不是 udp 问题,但我的建议是您添加日志记录以写出您收到的所有内容,然后用细齿梳进行检查。看起来你会记录你现在得到的东西,但你可能不得不挖掘你正在失去的 5%。

How do you know its 5%? if you send that traffic again, will it always be 5%? are the same messages always lost.

你怎么知道它的5%?如果您再次发送该流量,它是否总是 5%?总是丢失相同的消息。

I built a UDP server for voip/sip call data using ruby and Event Machine, and so far things have been working well. (I'm curious about your test approach though, I was doing everything over netcat, or a small ruby client, I never did 10k messages)

我使用 ruby​​ 和 Event Machine 为 voip/sip 呼叫数据构建了一个 UDP 服务器,到目前为止一切都运行良好。(不过,我对您的测试方法很好奇,我是通过 netcat 或小型 ruby​​ 客户端做所有事情的,我从来没有做过 10k 条消息)

回答by Walt Howard

A subtle tip here. Why did you use UDP for a stream? You need to use TCP for streams. The UDP protocol sends datagrams, discrete messages. It will not break them apart on you behind the scenes. What you send is what you receive with UDP.(IP fragmentation is a different issue I'm not talking about that here). You don't have to be concerned with re-assembling a stream on the other side. That's one of the main advantages of using UDP instead of TCP. Also, if you are doing localhost to localhost you don't have to worry about losing packets due to network hiccups. You could lose packets if you overflow the network stack buffers though so give yourself big ones if you are doing high speed data transfer. So, forget about the stream, just use UDP send:

这里有一个微妙的提示。你为什么使用 UDP 作为流?您需要对流使用 TCP。UDP 协议发送数据报、离散消息。它不会在幕后将它们分开。您发送的是您使用 UDP 接收的内容。(IP 碎片是一个不同的问题,我不在这里讨论)。您不必担心在另一侧重新组装流。这是使用 UDP 而不是 TCP 的主要优势之一。此外,如果您正在执行 localhost 到 localhost,您不必担心由于网络故障而丢失数据包。如果您溢出网络堆栈缓冲区,您可能会丢失数据包,因此如果您正在进行高速数据传输,请给自己大的数据包。因此,忘记流,只需使用 UDP 发送:

var udp_server = dgram.createSocket({ type: 'udp4', reuseAddr: true, recvBufferSize: 20000000 // <<== mighty big buffers });

var udp_server = dgram.createSocket({ type: 'udp4', repeatAddr: true, recvBufferSize: 20000000 // <<== 强大的缓冲区 });

udp_server.send("Badabing, badaboom", remote_port, remote_address);

udp_server.send("Badabing, badaboom", remote_port, remote_address);

Go was developed by Google to deal with the proliferation of languages that occurs in modern tech shops. (it's crazy, I agree). I cannot use it because its culture and design prohibit using Exceptions which are the most important feature modern languages have for removing the huge amount of clutter added by old fashion error handling. Other than that, it's fine but that's a show stopper for me.

Go 由 Google 开发,用于应对现代科技商店中出现的语言激增。(这很疯狂,我同意)。我不能使用它,因为它的文化和设计禁止使用异常,这是现代语言最重要的特性,用于消除旧式错误处理添加的大量混乱。除此之外,这很好,但这对我来说是个障碍。