通常，Node.js 如何处理 10,000 个并发请求？

How, in general, does Node.js handle 10,000 concurrent requests?

提问by g_b

I understand that Node.js uses a single-thread and an event loop to process requests only processing one at a time (which is non-blocking). But still, how does that work, lets say 10,000 concurrent requests. The event loop will process all the requests? Would not that take too long?

我知道 Node.js 使用单线程和事件循环来处理请求，一次只处理一个（非阻塞）。但是，它是如何工作的，比如说 10,000 个并发请求。事件循环会处理所有的请求吗？那不会花太长时间吗？

I can not understand (yet) how it can be faster than a multi-threaded web server. I understand that multi-threaded web server will be more expensive in resources (memory, CPU), but would not it still be faster? I am probably wrong; please explain how this single-thread is faster in lots of requests, and what it typically does (in high level) when servicing lots of requests like 10,000.

我无法理解（还）它如何比多线程 Web 服务器更快。我知道多线程 Web 服务器在资源（内存、CPU）方面会更昂贵，但它不会更快吗？我可能错了；请解释这个单线程如何在大量请求中更快，以及它在处理大量请求（如 10,000）时通常会做什么（在高级别）。

And also, will that single-thread scale well with that large amount? Please bear in mind that I am just starting to learn Node.js.

而且，那个单线程能在这么大的量下很好地扩展吗？请记住，我才刚刚开始学习 Node.js。

回答by slebetman

If you have to ask this question then you're probably unfamiliar with what most web applications/services do. You're probably thinking that all software do this:

如果您不得不问这个问题，那么您可能不熟悉大多数 Web 应用程序/服务的作用。您可能认为所有软件都这样做：

user do an action
       │
       v
 application start processing action
   └──> loop ...
          └──> busy processing
 end loop
   └──> send result to user

However, this is not how web applications, or indeed any application with a database as the back-end, work. Web apps do this:

然而，这不是 Web 应用程序或任何以数据库为后端的应用程序的工作方式。Web 应用程序执行以下操作：

user do an action
       │
       v
 application start processing action
   └──> make database request
          └──> do nothing until request completes
 request complete
   └──> send result to user

In this scenario, the software spend most of its running time using 0% CPU time waiting for the database to return.

在这种情况下，软件的大部分运行时间都使用 0% 的 CPU 时间等待数据库返回。

Multithreaded network app:

多线程网络应用：

Multithreaded network apps handle the above workload like this:

多线程网络应用程序处理上述工作负载是这样的：

request ──> spawn thread
              └──> wait for database request
                     └──> answer request
request ──> spawn thread
              └──> wait for database request
                     └──> answer request
request ──> spawn thread
              └──> wait for database request
                     └──> answer request

So the thread spend most of their time using 0% CPU waiting for the database to return data. While doing so they have had to allocate the memory required for a thread which includes a completely separate program stack for each thread etc. Also, they would have to start a thread which while is not as expensive as starting a full process is still not exactly cheap.

所以线程大部分时间都在使用 0% 的 CPU 等待数据库返回数据。在这样做时，他们不得不为一个线程分配所需的内存，该线程包括一个完全独立的程序堆栈，用于每个线程等。此外，他们必须启动一个线程，虽然它不像启动一个完整进程那样昂贵，但仍然不完全便宜的。

Singlethreaded event loop

单线程事件循环

Since we spend most of our time using 0% CPU, why not run some code when we're not using CPU? That way, each request will still get the same amount of CPU time as multithreaded applications but we don't need to start a thread. So we do this:

既然我们大部分时间都在使用 0% 的 CPU，那么为什么不在不使用 CPU 的情况下运行一些代码呢？这样，每个请求仍将获得与多线程应用程序相同的 CPU 时间，但我们不需要启动线程。所以我们这样做：

request ──> make database request
request ──> make database request
request ──> make database request
database request complete ──> send response
database request complete ──> send response
database request complete ──> send response

In practice both approaches return data with roughly the same latency since it's the database response time that dominates the processing.

实际上，这两种方法都以大致相同的延迟返回数据，因为数据库响应时间决定了处理过程。

The main advantage here is that we don't need to spawn a new thread so we don't need to do lots and lots of malloc which would slow us down.

这里的主要优点是我们不需要产生一个新线程，所以我们不需要做很多会减慢我们速度的 malloc。

Magic, invisible threading

神奇的隐形穿线

The seemingly mysterious thing is how both the approaches above manage to run workload in "parallel"? The answer is that the database is threaded. So our single-threaded app is actually leveraging the multi-threaded behaviour of another process: the database.

看似神秘的事情是上述两种方法如何设法“并行”运行工作负载？答案是数据库是线程化的。所以我们的单线程应用实际上是在利用另一个进程的多线程行为：数据库。

Where singlethreaded approach fails

单线程方法失败的地方

A singlethreaded app fails big if you need to do lots of CPU calculations before returning the data. Now, I don't mean a for loop processing the database result. That's still mostly O(n). What I mean is things like doing Fourier transform (mp3 encoding for example), ray tracing (3D rendering) etc.

如果您需要在返回数据之前进行大量 CPU 计算，则单线程应用程序会失败。现在，我不是指处理数据库结果的 for 循环。这仍然主要是 O(n)。我的意思是做傅立叶变换（例如 mp3 编码）、光线追踪（3D 渲染）等。

Another pitfall of singlethreaded apps is that it will only utilise a single CPU core. So if you have a quad-core server (not uncommon nowdays) you're not using the other 3 cores.

单线程应用程序的另一个缺陷是它只会使用单个 CPU 内核。因此，如果您有一个四核服务器（现在并不少见），您就不会使用其他 3 个内核。

Where multithreaded approach fails

多线程方法失败的地方

A multithreaded app fails big if you need to allocate lots of RAM per thread. First, the RAM usage itself means you can't handle as many requests as a singlethreaded app. Worse, malloc is slow. Allocating lots and lots of objects (which is common for modern web frameworks) means we can potentially end up being slower than singlethreaded apps. This is where node.js usually win.

如果您需要为每个线程分配大量 RAM，则多线程应用程序会失败。首先，RAM 使用本身意味着您无法处理与单线程应用程序一样多的请求。更糟糕的是，malloc 很慢。分配大量对象（这在现代 Web 框架中很常见）意味着我们可能最终比单线程应用程序慢。这是 node.js 通常获胜的地方。

One use-case that end up making multithreaded worse is when you need to run another scripting language in your thread. First you usually need to malloc the entire runtime for that language, then you need to malloc the variables used by your script.

最终使多线程变得更糟的一个用例是当您需要在您的线程中运行另一种脚本语言时。首先，您通常需要为该语言分配整个运行时，然后您需要对脚本使用的变量进行分配。

So if you're writing network apps in C or go or java then the overhead of threading will usually not be too bad. If you're writing a C web server to serve PHP or Ruby then it's very easy to write a faster server in javascript or Ruby or Python.

因此，如果您使用 C 或 go 或 java 编写网络应用程序，那么线程的开销通常不会太糟糕。如果您正在编写一个 C Web 服务器来为 PHP 或 Ruby 提供服务，那么用 javascript、Ruby 或 Python 编写一个更快的服务器非常容易。

Hybrid approach

混合方法

Some web servers use a hybrid approach. Nginx and Apache2 for example implement their network processing code as a thread pool of event loops. Each thread runs an event loop simultaneously processing requests single-threaded but requests are load-balanced among multiple threads.

一些 Web 服务器使用混合方法。例如，Nginx 和 Apache2 将它们的网络处理代码实现为事件循环的线程池。每个线程运行一个事件循环同时处理单线程请求，但请求在多个线程之间进行负载平衡。

Some single-threaded architectures also use a hybrid approach. Instead of launching multiple threads from a single process you can launch multiple applications - for example, 4 node.js servers on a quad-core machine. Then you use a load balancer to spread the workload amongst the processes.

一些单线程架构也使用混合方法。您可以启动多个应用程序，而不是从单个进程启动多个线程 - 例如，四核机器上的 4 个 node.js 服务器。然后您使用负载平衡器在进程之间分配工作负载。

In effect the two approaches are technically identical mirror-images of each other.

实际上，这两种方法在技术上是彼此相同的镜像。

回答by chriskelly

What you seem to be thinking is that most of the processing is handled in the node event loop. Node actually farms off the I/O work to threads. I/O operations typically take orders of magnitude longer than CPU operations so why have the CPU wait for that? Besides, the OS can handle I/O tasks very well already. In fact, because Node does not wait around it achieves much higher CPU utilisation.

您似乎在想的是，大部分处理都是在节点事件循环中处理的。Node 实际上将 I/O 工作转移到线程上。I/O 操作通常比 CPU 操作花费的时间长几个数量级，那么为什么要让 CPU 等待呢？此外，操作系统已经可以很好地处理 I/O 任务。事实上，因为 Node 不会等待，它实现了更高的 CPU 利用率。

By way of analogy, think of NodeJS as a waiter taking the customer orders while the I/O chefs prepare them in the kitchen. Other systems have multiple chefs, who take a customers order, prepare the meal, clear the table and only then attend to the next customer.

打个比方，把 NodeJS 想象成一个服务员在接客户订单，而 I/O 厨师在厨房准备。其他系统有多个厨师，他们接受客户的订单，准备饭菜，清理桌子，然后才照顾下一个客户。

回答by sheltond

I understand that Node.js uses a single-thread and an event loop to process requests only processing one at a time (which is non-blocking).

我知道 Node.js 使用单线程和事件循环来处理请求，一次只处理一个（非阻塞）。

I could be misunderstanding what you've said here, but "one at a time" sounds like you may not be fully understanding the event-based architecture.

我可能会误解你在这里所说的，但“一次一个”听起来你可能没有完全理解基于事件的架构。

In a "conventional" (non event-driven) application architecture, the process spends a lot of time sitting around waiting for something to happen. In an event-based architecture such as Node.js the process doesn't just wait, it can get on with other work.

在“传统”（非事件驱动）应用程序架构中，该过程花费大量时间等待某事发生。在基于事件的体系结构中，例如 Node.js，过程不只是等待，它还可以继续进行其他工作。

For example: you get a connection from a client, you accept it, you read the request headers (in the case of http), then you start to act on the request. You might read the request body, you will generally end up sending some data back to the client (this is a deliberate simplification of the procedure, just to demonstrate the point).

例如：您从客户端获得连接，接受它，读取请求标头（在 http 的情况下），然后开始对请求采取行动。您可能会阅读请求正文，通常最终会将一些数据发送回客户端（这是对过程的故意简化，只是为了说明这一点）。

At each of these stages, most of the time is spent waiting for some data to arrive from the other end - the actual time spent processing in the main JS thread is usually fairly minimal.

在每个阶段，大部分时间都花在等待一些数据从另一端到达——在主 JS 线程中花费的实际处理时间通常相当少。

When the state of an I/O object (such as a network connection) changes such that it needs processing (e.g. data is received on a socket, a socket becomes writable, etc) the main Node.js JS thread is woken with a list of items needing to be processed.

当 I/O 对象（例如网络连接）的状态发生变化以致需要处理时（例如在套接字上接收到数据，套接字变为可写等），主 Node.js JS 线程将被一个列表唤醒需要处理的项目。

It finds the relevant data structure and emits some event on that structure which causes callbacks to be run, which process the incoming data, or write more data to a socket, etc. Once all of the I/O objects in need of processing have been processed, the main Node.js JS thread will wait again until it's told that more data is available (or some other operation has completed or timed out).

它找到相关的数据结构并在该结构上发出一些事件，导致运行回调，处理传入的数据，或将更多数据写入套接字等。 一旦所有需要处理的 I/O 对象都已完成处理后，主 Node.js JS 线程将再次等待，直到它被告知有更多数据可用（或其他一些操作已完成或超时）。

The next time that it is woken, it could well be due to a different I/O object needing to be processed - for example a different network connection. Each time, the relevant callbacks are run and then it goes back to sleep waiting for something else to happen.

下次它被唤醒时，很可能是由于需要处理不同的 I/O 对象——例如不同的网络连接。每次，相关的回调都会运行，然后它会重新进入睡眠状态，等待其他事情发生。

The important point is that the processing of different requests is interleaved, it doesn't process one request from start to end and then move onto the next.

重要的一点是，不同请求的处理是交错进行的，它不会从头到尾处理一个请求，然后再转到下一个请求。

To my mind, the main advantage of this is that a slow request (e.g. you're trying to send 1MB of response data to a mobile phone device over a 2G data connection, or you're doing a really slow database query) won't block faster ones.

在我看来，这样做的主要优点是缓慢的请求（例如，您尝试通过 2G 数据连接向移动电话设备发送 1MB 的响应数据，或者您正在执行非常慢的数据库查询）不会t 阻止更快的。

In a conventional multi-threaded web server, you will typically have a thread for each request being handled, and it will process ONLY that request until it's finished. What happens if you have a lot of slow requests? You end up with a lot of your threads hanging around processing these requests, and other requests (which might be very simple requests that could be handled very quickly) get queued behind them.

在传统的多线程 Web 服务器中，您通常会为每个正在处理的请求设置一个线程，并且它只会处理该请求，直到它完成。如果您有很多缓慢的请求会怎样？您最终会遇到很多线程在处理这些请求，而其他请求（可能是可以非常快速处理的非常简单的请求）在它们后面排队。

There are plenty of others event-based systems apart from Node.js, and they tend to have similar advantages and disadvantages compared with the conventional model.

除了 Node.js 之外，还有很多其他基于事件的系统，与传统模型相比，它们往往具有相似的优点和缺点。

I wouldn't claim that event-based systems are faster in every situation or with every workload - they tend to work well for I/O-bound workloads, not so well for CPU-bound ones.

我不会声称基于事件的系统在每种情况下或每种工作负载下都更快——它们往往适用于受 I/O 限制的工作负载，而不适用于受 CPU 限制的工作负载。

回答by sudheer nunna

Single Threaded Event Loop Model Processing Steps:

单线程事件循环模型处理步骤：

• Clients Send request to Web Server.

• Node JS Web Server internally maintains a Limited Thread pool to provide services to the Client Requests.

• Node JS Web Server receives those requests and places them into a Queue. It is known as “Event Queue”.

• Node JS Web Server internally has a Component, known as “Event Loop”. Why it got this name is that it uses indefinite loop to receive requests and process them.

• Event Loop uses Single Thread only. It is main heart of Node JS Platform Processing Model.

• Event Loop checks any Client Request is placed in Event Queue. If not then wait for incoming requests for indefinitely.

• If yes, then pick up one Client Request from Event Queue

  1. Starts process that Client Request
  2. If that Client Request Does Not requires any Blocking IO Operations, then process everything, prepare response and send it back to client.
  3. If that Client Request requires some Blocking IO Operations like interacting with Database, File System, External Services then it will follow different approach
• Checks Threads availability from Internal Thread Pool
• Picks up one Thread and assign this Client Request to that thread.

• That Thread is responsible for taking that request, process it, perform Blocking IO operations, prepare response and send it back to the Event Loop

  very nicely explained by @Rambabu Posa for more explanation go throw this Link

• 客户端向 Web 服务器发送请求。

• Node JS Web Server 内部维护了一个有限线程池来为客户端请求提供服务。

• Node JS Web Server 接收这些请求并将它们放入队列中。它被称为“事件队列”。

• Node JS Web Server 内部有一个组件，称为“事件循环”。之所以有这个名字，是因为它使用无限循环来接收请求并处理它们。

• 事件循环仅使用单线程。它是 Node JS 平台处理模型的核心。

• 事件循环检查任何放置在事件队列中的客户端请求。如果没有，则无限期地等待传入请求。

• 如果是，则从事件队列中提取一个客户端请求

  1. 启动客户端请求的进程
  2. 如果该客户端请求不需要任何阻塞 IO 操作，则处理所有内容，准备响应并将其发送回客户端。
  3. 如果该客户端请求需要一些阻塞 IO 操作，例如与数据库、文件系统、外部服务交互，那么它将遵循不同的方法
• 从内部线程池检查线程可用性
• 选取一个线程并将此客户端请求分配给该线程。

• 该线程负责接收该请求、处理它、执行阻塞 IO 操作、准备响应并将其发送回事件循环

  @Rambabu Posa 很好地解释了更多解释，请抛出此链接

回答by Aman Gupta

Adding to slebetman answer: When you say Node.JScan handle 10,000 concurrent requests they are essentially non-blocking requests i.e. these requests are majorly pertaining to database query.

添加到 slebetman 答案：当您说Node.JS可以处理 10,000 个并发请求时，它们本质上是非阻塞请求，即这些请求主要与数据库查询有关。

Internally, event loopof Node.JSis handling a thread pool, where each thread handles a non-blocking requestand event loop continues to listen to more request after delegating work to one of the thread of the thread pool. When one of the thread completes the work, it send a signal to the event loopthat it has finished aka callback. Event loopthen process this callback and send the response back.

在内部，event loopofNode.JS正在处理 a thread pool，其中每个线程处理 anon-blocking request并且事件循环在将工作委派给thread pool. 当其中一个线程完成工作时，它会向 发送一个信号，event loop表示它已完成 aka callback。Event loop然后处理此回调并将响应发回。

As you are new to NodeJS, do read more about nextTickto understand how event loop works internally. Read blogs on http://javascriptissexy.com, they were really helpful for me when I started with JavaScript/NodeJS.

由于您是 NodeJS 的新手，请阅读更多内容nextTick以了解事件循环在内部是如何工作的。阅读http://javascriptissexy.com 上的博客，当我开始使用 JavaScript/NodeJS 时，它们对我很有帮助。

回答by rranj

Adding to slebetman's answer for more clarity on what happens while executing the code.

添加到slebetman的答案以更清楚地了解执行代码时发生的情况。

The internal thread pool in nodeJs just has 4 threads by default. and its not like the whole request is attached to a new thread from the thread pool the whole execution of request happens just like any normal request (without any blocking task) , just that whenever a request has any long running or a heavy operation like db call ,a file operation or a http request the task is queued to the internal thread pool which is provided by libuv. And as nodeJs provides 4 threads in internal thread pool by default every 5th or next concurrent request waits until a thread is free and once these operations are over the callback is pushed to the callback queue. and is picked up by event loop and sends back the response.

nodeJs 中的内部线程池默认只有 4 个线程。它不像整个请求附加到线程池中的新线程，请求的整个执行就像任何普通请求（没有任何阻塞任务）一样发生，只是每当请求有任何长时间运行或像 db 这样的繁重操作时调用、文件操作或 http 请求将任务排队到由 libuv 提供的内部线程池。由于 nodeJs 默认在内部线程池中提供 4 个线程，每 5 个或下一个并发请求等待一个线程空闲，一旦这些操作结束，回调就会被推送到回调队列中。并被事件循环接收并发回响应。

Now here comes another information that its not once single callback queue, there are many queues.

现在又来了一个信息，它不是一个单一的回调队列，有很多队列。

1. NextTick queue
2. Micro task queue
3. Timers Queue
4. IO callback queue (Requests, File ops, db ops)
5. IO Poll queue
6. Check Phase queue or SetImmediate
7. close handlers queue

1. NextTick 队列
2. 微任务队列
3. 定时器队列
4. IO 回调队列（请求、文件操作、数据库操作）
5. IO 轮询队列
6. 检查阶段队列或 SetImmediate
7. 关闭处理程序队列

Whenever a request comes the code gets executing in this order of callbacks queued.

每当请求到来时，代码就会按照排队的回调顺序执行。

It is not like when there is a blocking request it is attached to a new thread. There are only 4 threads by default. So there is another queueing happening there.

它不像当有一个阻塞请求时它附加到一个新线程。默认只有 4 个线程。所以那里发生了另一个排队。

Whenever in a code a blocking process like file read occurs , then calls a function which utilises thread from thread pool and then once the operation is done , the callback is passed to the respective queue and then executed in the order.

每当在代码中发生文件读取等阻塞过程时，就会调用一个函数，该函数利用线程池中的线程，然后一旦操作完成，回调将传递到相应的队列，然后按顺序执行。

Everything gets queued based on the the type of callback and processed in the order mentioned above.

一切都根据回调的类型排队，并按上述顺序处理。