java Jetty 和其他容器如何在坚持 Servlet 规范的同时利用 NIO？

How do Jetty and other containers leverage NIO while sticking to the Servlet specification?

提问by rationalrevolt

I'm new to NIO, and I am trying to figure out how Jetty leverages NIO.

我是 NIO 的新手，我正在尝试弄清楚 Jetty 如何利用 NIO。

My understanding of how traditional servlet containers that use Blocking IO service a request is as follows:

我对使用Blocking IO服务请求的传统servlet容器如何理解如下：

1. A request arrives
2. A thread is allocated to process the request and the servlet method (doGetetc) is invoked
3. Servlet method is handed an InputStreamand OutputStream
4. The servlet method reads from the InputStreamand writes to the OutputStream
5. The InputStreamand OutputStreamare basically tied to the respective streams of the underlying Socket

1. 一个请求到达
2. 分配一个线程来处理请求并doGet调用servlet 方法（等）
3. Servlet 方法被传递给一个InputStream和OutputStream
4. servlet 方法从 读取InputStream并写入OutputStream
5. 的InputStream和OutputStream基本上与基础的相应流Socket

What is different when an NIO connector is used? My guess is along the following lines:

使用 NIO 连接器时有何不同？我的猜测如下：

1. A request arrives
2. Jetty uses NIO connector and buffers the entire request asynchronously
3. Once request has been read completely wrap the buffer in an InputStream
4. Create an empty response buffer (wrapped in an OutputStream)
5. Allocate a thread and invoke the servlet method (doGetetc) handing the above wrapper streams
6. Servlet method writes to the wrapped (buffered) response stream and returns from the servlet method
7. Jetty uses NIO to write the response buffer contents to the underlying SocketChannel

1. 一个请求到达
2. Jetty 使用 NIO 连接器并异步缓冲整个请求
3. 一旦请求被完全读取，将缓冲区包裹在一个 InputStream
4. 创建一个空的响应缓冲区（包装在 中OutputStream）
5. 分配一个线程并调用doGet处理上述包装流的 servlet 方法（等）
6. Servlet 方法写入包装（缓冲）响应流并从 servlet 方法返回
7. Jetty 使用 NIO 将响应缓冲区内容写入底层 SocketChannel

From the Jetty documentation, I found the following:

从 Jetty 文档中，我发现了以下内容：

SelectChannelConnector- This connector uses efficient NIO buffers with a non-blocking threading model. Jetty uses Direct NIO buffers, and allocates threads only to connections with requests. Synchronization simulates blocking for the servlet API, and any unflushed content at the end of request handling is written asynchronously.

SelectChannelConnector- 此连接器使用具有非阻塞线程模型的高效 NIO 缓冲区。Jetty 使用 Direct NIO 缓冲区，并且只为有请求的连接分配线程。同步模拟 servlet API 的阻塞，并且在请求处理结束时任何未刷新的内容都是异步写入的。

I'm not sure I understand what Synchronization simulates blocking for the servlet APImeans?

我不确定我明白什么Synchronization simulates blocking for the servlet API意思？

回答by gregw

You don't have it exactly correct. When jetty uses an NIO connector (and 9 only supports NIO) it works as follows:

你没有完全正确。当 jetty 使用 NIO 连接器（并且 9 只支持 NIO）时，它的工作方式如下：

1. Idle state as a few threads (1-4 depending on # cores) calling the selector, looking for IO activity. This has been scaled to over 1,000,000 connections on Jetty.

2. When selector sees IO activity, it calls a handle method on the connection, which either:

   • something else has registered that it is blocked waiting for IO for this connection, so in that case the selector just wakes up whoever was blocked.
   • otherwise a thread is dispatched to handle the connection.

3. If a thread is dispatched, then it will attempt to read the connection and parse it. What happens now depends on if the connection is http, spdy, http2 or websocket.

   • for http, if the request headers are complete, the thread goes on to call the handling of the request (eventually this gets to the servlet) without waiting for any content.
   • for http2/spdy another dispatch is required, but see the discussion about Eat-What-You-Kill strategy on the list: http://dev.eclipse.org/mhonarc/lists/jetty-dev/msg02166.html
   • for websocket the message handling is called.

4. Once a thread is dispatched to a servlet, it looks to it like the servlet IO is blocking, but underneath the level of HttpInputStream and HttpOutputStream all the IO is async with callbacks. The blocking API uses a special blocking callback to achieve blocking. This means that if the servlet chooses to use async IO, then it is just bypassing the blocking callback and using the async API more or less directly.

5. A servlet can suspend using request.startAsync, in which case the dispatched thread is returned to the thread pool, but the associated connection is not marked as interested in IO. Async IO can be performed, but a AsyncContext event is need to either reallocate a thread or to re-enroll the connection for IO activity once the async cycle is complete.

1. 空闲状态为几个线程（1-4 取决于 # 个内核）调用选择器，寻找 IO 活动。这已在 Jetty 上扩展到超过 1,000,000 个连接。

2. When selector sees IO activity, it calls a handle method on the connection, which either:

   • 其他东西已经注册它被阻止等待此连接的 IO，因此在这种情况下，选择器只会唤醒被阻止的任何人。
   • 否则将调度一个线程来处理连接。

3. 如果一个线程被分派，那么它将尝试读取连接并解析它。现在会发生什么取决于连接是 http、spdy、http2 还是 websocket。

   • 对于 http，如果请求头是完整的，线程会继续调用请求的处理（最终这会到达 servlet），而无需等待任何内容。
   • 对于 http2/spdy 需要另一个调度，但请参阅列表中关于 Eat-What-You-Kill 策略的讨论：http: //dev.eclipse.org/mhonarc/lists/jetty-dev/msg02166.html
   • 对于 websocket，消息处理被调用。

4. 一旦一个线程被分派到一个 servlet，它看起来就像 servlet IO 是阻塞的，但是在 HttpInputStream 和 HttpOutputStream 的级别之下，所有的 IO 都是与回调异步的。阻塞 API 使用特殊的阻塞回调来实现阻塞。这意味着如果 servlet 选择使用异步 IO，那么它只是绕过阻塞回调并或多或少直接使用异步 API。

5. servlet 可以使用 request.startAsync 挂起，在这种情况下，分派的线程会返回到线程池，但关联的连接不会被标记为对 IO 感兴趣。可以执行异步 IO，但需要 AsyncContext 事件重新分配线程或在异步周期完成后重新注册 IO 活动的连接。

This view is slightly complicated by http2 and spdy, which are multiplexed, so they can involve an extra dispatch.

这个视图被 http2 和 spdy 稍微复杂了一点，它们是多路复用的，因此它们可能涉及额外的调度。

Any HTTP framework that does not dispatch can go really really fast in benchmark code, but when faced with a real application that can do silly things like block on databases, files system, REST services etc... then lack of dispatch just means that one connection can hold up all the other connections on the system.

任何不分派的 HTTP 框架在基准代码中都可以非常快地运行，但是当面对一个真正的应用程序时，它可以做一些愚蠢的事情，比如阻塞数据库、文件系统、REST 服务等……那么缺乏分派就意味着一个连接可以阻止系统上的所有其他连接。

For some more info on how jetty handles async and dispatch see:

有关码头如何处理异步和调度的更多信息，请参阅：

• https://webtide.com/asynchronous-callbacks/
• https://webtide.com/jetty-9-goes-fast-with-mechanical-sympathy/
• https://webtide.com/avoiding-parallel-slowdown-in-jetty-9/
• https://webtide.com/lies-damned-lies-and-benchmarks-2/
• https://webtide.com/jetty-in-techempower-benchmarks/

• https://webtide.com/asynchronous-callbacks/
• https://webtide.com/jetty-9-goes-fast-with-mechanical-sympathy/
• https://webtide.com/avoiding-parallel-slowdown-in-jetty-9/
• https://webtide.com/lies-damned-lies-and-benchmarks-2/
• https://webtide.com/jetty-in-techempower-benchmarks/