Java NIO：如何通过非阻塞 I/O 知道 SocketChannel read() 何时完成

Java NIO: How to know when SocketChannel read() is complete with non-blocking I/O

提问by Riyad Kalla

I am currently using a non-blocking SocketChannel (Java 1.6) to act as a client to a Redis server. Redis accepts plain-text commands directly over a socket, terminated by CRLF and responds in-like, a quick example:

我目前使用非阻塞 SocketChannel (Java 1.6) 作为 Redis 服务器的客户端。Redis 直接通过套接字接受纯文本命令，由 CRLF 终止并以类似方式响应，一个简单的例子：

SEND: 'PING\r\n'

RECV: '+PONG\r\n'

发送：'PING\r\n'

RECV: '+PONG\r\n'

Redis can also return huge replies (depending on what you are asking for) with many sections of \r\n-terminated data all as part of a single response.

Redis 还可以返回大量回复（取决于您的要求），其中包含许多 \r\n 终止的数据部分，全部作为单个响应的一部分。

I am using a standard while(socket.read() > 0) {//append bytes}loop to read bytes from the socket and re-assemble them client side into a reply.

我正在使用标准的while(socket.read() > 0) {//append bytes}循环从套接字读取字节并将它们重新组装到客户端作为回复。

NOTE: I am not using a Selector, just multiple, client-side SocketChannels connected to the server, waiting to service send/receive commands.

注意：我没有使用选择器，只是连接到服务器的多个客户端 SocketChannel，等待服务发送/接收命令。

What I'm confused about is the contractof the SocketChannel.read()method in non-blocking mode, specifically, how to know when the server is done sending and I have the entire message.

我感到困惑的是合同中的SocketChannel.read（）方法在非阻塞模式，具体而言，当服务器进行发送，我有整个邮件怎么知道。

I have a few methods to protect against returning too fast and giving the server a chance to reply, but the one thing I'm stuck on is:

我有一些方法可以防止返回太快并让服务器有机会回复，但我坚持的一件事是：

1. Is it ever possible for read()to return bytes, then on a subsequent call return no bytes, but on another subsequent call again return some bytes?

1. read()是否有可能返回字节，然后在后续调用中不返回字节，但在另一个后续调用中再次返回一些字节？

Basically, can I trust that the server is done responding to me if I have received at least 1 byte and eventually read()returns 0 then I know I'm done, or is it possible the server was just busy and might sputter back some more bytes if I wait and keep trying?

基本上，如果我收到至少 1 个字节并最终read()返回 0 ，我可以相信服务器已经完成对我的响应，然后我知道我已经完成了，或者服务器是否可能只是忙并且可能会回馈一些如果我等待并继续尝试更多字节？

If it cankeep sending bytes even after a read() has returned 0 bytes (after previous successful reads) then I have no idea how to tell when the server is done talking to me and in-fact am confused how java.io.* style communications would even know when the server is "done" either.

如果即使在 read() 返回 0 字节（之前成功读取之后）之后它仍然可以继续发送字节，那么我不知道如何判断服务器何时完成与我的对话，实际上我很困惑 java.io.*样式通信甚至会知道服务器何时“完成”。

As you guys know read never returns -1 unless the connection is dead and these are standard long-lived DB connections, so I won't be closing and opening them on each request.

正如你们所知， read 永远不会返回 -1 ，除非连接死了，而且这些是标准的长期数据库连接，所以我不会在每个请求时关闭和打开它们。

I know a popular response (atleast for these NIO questions) have been to look at Grizzly, MINA or Netty -- if possible I'd really like to learn how this all works in it's raw state before adopting some 3rd party dependencies.

我知道一个流行的反应（至少对于这些 NIO 问题）是查看 Grizzly、MINA 或 Netty——如果可能的话，我真的很想在采用一些 3rd 方依赖项之前了解这一切在原始状态下是如何工作的。

Thank you.

谢谢你。

Bonus Question:

奖金问题：

I originally thought a blocking SocketChannel would be the way to go with this as I don't really want a caller to do anything until I process their command and give them back a reply anyway.

我最初认为阻塞 SocketChannel 将是解决此问题的方法，因为在我处理他们的命令并无论如何给他们回复之前，我真的不希望调用者做任何事情。

If that ends up being a better way to go, I was a bit confused seeing that SocketChannel.read() blocks as long as there aren't bytes sufficient to fill the given buffer... short of reading everything byte-by-byte I can't figure out how this default behavior is actually meant to be used... I never know the exactsize of the reply coming back from the server, so my calls to SocketChannel.read() always block until a time out (at which point I finally see that the content was sitting in the buffer).

如果这最终成为一种更好的方法，那么看到 SocketChannel.read() 阻塞，只要没有足够的字节来填充给定的缓冲区，我就有点困惑......没有逐字节读取所有内容我无法弄清楚这个默认行为实际上是如何使用的......我永远不知道从服务器返回的回复的确切大小，所以我对 SocketChannel.read() 的调用总是阻塞直到超时（此时我终于看到内容位于缓冲区中）。

I'm not real clear on the right way to use the blocking method since it always hangs up on a read.

我不太清楚使用阻塞方法的正确方法，因为它总是在读取时挂起。

采纳答案by Bert F

If it can keep sending bytes even after a read() has returned 0 bytes (after previous successful reads) then I have no idea how to tell when the server is done talking to me and in-fact am confused how java.io.* style communications would even know when the server is "done" either.

如果即使在 read() 返回 0 字节（之前成功读取之后）之后它仍然可以继续发送字节，那么我不知道如何判断服务器何时完成与我的对话，实际上我很困惑 java.io.*样式通信甚至会知道服务器何时“完成”。

Read and follow the protocol:

阅读并遵循协议：

http://redis.io/topics/protocol

http://redis.io/topics/protocol

The spec describes the possible types of replies and how to recognize them. Some are line terminated, while multi-line responses include a prefix count.

该规范描述了可能的回复类型以及如何识别它们。有些是行终止的，而多行响应包括前缀计数。

Replies

Redis will reply to commands with different kinds of replies. It is possible to check the kind of reply from the first byte sent by the server:

• With a single line reply the first byte of the reply will be "+"
• With an error message the first byte of the reply will be "-"
• With an integer number the first byte of the reply will be ":"
• With bulk reply the first byte of the reply will be "$"
• With multi-bulk reply the first byte of the reply will be "*"

Single line reply

A single line reply is in the form of a single line string starting with "+" terminated by "\r\n". ...

...

Multi-bulk replies

Commands like LRANGE need to return multiple values (every element of the list is a value, and LRANGE needs to return more than a single element). This is accomplished using multiple bulk writes, prefixed by an initial line indicating how many bulk writes will follow.

回复

Redis 将使用不同类型的回复来回复命令。可以从服务器发送的第一个字节检查回复的类型：

• 对于单行回复，回复的第一个字节将是“+”
• 对于错误消息，回复的第一个字节将是“-”
• 对于整数，回复的第一个字节将是“：”
• 批量回复时，回复的第一个字节将是“$”
• 对于多批量回复，回复的第一个字节将是“*”

单行回复

单行回复采用单行字符串的形式，以 "+" 开头，以 "\r\n" 结尾。...

...

多批回复

像 LRANGE 这样的命令需要返回多个值（列表的每个元素都是一个值，而 LRANGE 需要返回多个元素）。这是使用多个批量写入完成的，以初始行作为前缀，指示将遵循多少批量写入。

---

---

Is it ever possible for read() to return bytes, then on a subsequent call return no bytes, but on another subsequent call again return some bytes? Basically, can I trust that the server is done responding to me if I have received at least 1 byte and eventually read() returns 0 then I know I'm done, or is it possible the server was just busy and might sputter back some more bytes if I wait and keep trying?

read() 是否有可能返回字节，然后在后续调用中不返回字节，但在另一个后续调用中再次返回一些字节？基本上，如果我收到至少 1 个字节并最终 read() 返回 0，我可以相信服务器已经完成对我的响应，然后我知道我已经完成了，或者服务器是否可能只是忙并且可能会回馈一些如果我等待并继续尝试更多字节？

Yes, that's possible. Its not just due to the server being busy, but network congestion and downed routes can cause data to "pause". The data is a stream that can "pause" anywhere in the stream without relation to the application protocol.

是的，这是可能的。这不仅是因为服务器繁忙，而且网络拥塞和路由中断会导致数据“暂停”。数据是一个流，它可以在流中的任何地方“暂停”，而与应用程序协议无关。

Keep reading the stream into a buffer. Peek at the first character to determine what type of response to expect. Examine the buffer after each successful read until the buffer contains the full message according to the specification.

继续将流读入缓冲区。查看第一个字符以确定预期的响应类型。每次成功读取后检查缓冲区，直到缓冲区包含符合规范的完整消息。

---

---

I originally thought a blocking SocketChannel would be the way to go with this as I don't really want a caller to do anything until I process their command and give them back a reply anyway.

我最初认为阻塞 SocketChannel 将是解决此问题的方法，因为在我处理他们的命令并无论如何给他们回复之前，我真的不希望调用者做任何事情。

I think you're right. Based on my quick-look at the spec, blocking reads wouldn't work for this protocol. Since it looks line-based, BufferedReadermay help, but you still need to know how to recognize when the response is complete.

我觉得你是对的。根据我对规范的快速浏览，阻止读取不适用于该协议。由于它看起来是基于行的，BufferedReader可能会有所帮助，但您仍然需要知道如何识别响应何时完成。

回答by Erick Robertson

Look to your Redis specifications for this answer.

请查看您的 Redis 规范以获取此答案。

It's not against the rules for a call to .read()to return 0 bytes on one call and 1 or more bytes on a subsequent call. This is perfectly legal. If anything were to cause a delay in delivery, either because of network lag or slowness in the Redis server, this could happen.

调用.read()在一次调用中返回 0 字节并在后续调用中返回 1 个或更多字节并不违反规则。这是完全合法的。如果有任何事情导致交付延迟，无论是由于网络延迟还是 Redis 服务器速度缓慢，都可能发生这种情况。

The answer you seek is the same answer to the question: "If I connected manually to the Redis server and sent a command, how could I know when it was done sending the response to me so that I can send another command?"

您寻求的答案与以下问题的答案相同：“如果我手动连接到 Redis 服务器并发送命令，我怎么知道它何时完成向我发送响应以便我可以发送另一个命令？”

The answer must be found in the Redis specification. If there's not a global token that the server sends when it is done executing your command, then this may be implemented on a command-by-command basis. If the Redis specifications do not allow for this, then this is a fault in the Redis specifications. They should tell you how to tell when they have sent all their data. This is why shells have command prompts. Redis should have an equivalent.

答案必须在 Redis 规范中找到。如果服务器在执行完您的命令后没有发送全局令牌，那么这可能会在逐个命令的基础上实现。如果Redis规范不允许这样做，那么这是Redis规范中的错误。他们应该告诉您如何判断他们何时发送了所有数据。这就是 shell 具有命令提示符的原因。Redis 应该有一个等价的。

In the case that Redis does not have this in their specifications, then I would suggest putting in some sort of timer functionality. Code your thread handling the socket to signal that a command is completed after no data has been received for a designated period of time, like five seconds. Choose a period of time that is significantly longer than the longest command takes to execute on the server.

如果 Redis 的规范中没有这个，那么我建议加入某种计时器功能。对处理套接字的线程进行编码，以在指定的时间段内（如五秒）未接收到数据后发出命令已完成的信号。选择比在服务器上执行的最长命令要长得多的时间段。

回答by user207421

I am using a standard while(socket.read() > 0) {//append bytes} loop

我正在使用标准的 while(socket.read() > 0) {//append bytes} 循环

That is not a standard technique in NIO. You muststore the result of the read in a variable, and test it for:

这不是 NIO 的标准技术。您必须将读取的结果存储在一个变量中，并对其进行测试：

1. -1, indicating EOS, meaning you should close the channel
2. zero, meaning there was no data to read, meaning you should return to the select() loop, and
3. a positive value, meaning you have read that many bytes, which you should then extract and remove from the ByteBuffer (get()/compact()) before continuing.

1. -1，表示EOS，意味着你应该关闭通道
2. 零，意味着没有要读取的数据，意味着您应该返回到 select() 循环，并且
3. 一个正值，意味着您已经读取了那么多字节，然后您应该在继续之前从 ByteBuffer (get()/compact()) 中提取和删除这些字节。

回答by igaz

It's been a long time, but . . .

已经很久了，但是。. .

I am currently using a non-blocking SocketChannel

我目前正在使用非阻塞 SocketChannel

Just to be clear, SocketChannels are blocking by default; to make them non-blocking, one must explicitly invoke SocketChannel#configureBlocking(false)

需要明确的是，SocketChannels 默认是阻塞的；为了使它们非阻塞，必须显式调用SocketChannel#configureBlocking(false)

I'll assume you did that

我会假设你这样做了

I am not using a Selector

我没有使用选择器

Whoa; that's the problem; if you are going to use non-blocking Channels, then you should always use a Selector (at least for reads); otherwise, you run into the confusion you described, viz. read(ByteBuffer) == 0doesn't mean anything (well, it means that there are no bytes in the tcp receive buffer at this moment).

哇；那就是问题所在; 如果您打算使用非阻塞通道，那么您应该始终使用选择器（至少对于读取而言）；否则，你会遇到你描述的混乱，即。read(ByteBuffer) == 0并不意味着什么（当然，这意味着有在TCP没有字节接收缓冲区在这一刻）。

It's analogous to checking your mailbox and it's empty; does it mean that the letter will never arrive? was never sent?

这类似于检查您的邮箱并且它是空的；这是否意味着这封信永远不会到达？从来没有发送过？

What I'm confused about is the contract of the SocketChannel.read() method in non-blocking mode, specifically, how to know when the server is done sending and I have the entire message.

我感到困惑的是非阻塞模式下 SocketChannel.read() 方法的契约，具体来说，如何知道服务器何时完成发送以及我拥有完整的消息。

There is a contract -> if a Selector has selected a Channel for a read operation, then the next invocation of SocketChannel#read(ByteBuffer)is guaranteed to return > 0(assuming there's room in the ByteBuffer arg)

有一个约定 -> 如果 Selector 选择了一个 Channel 进行读取操作，则SocketChannel#read(ByteBuffer)保证下一次调用返回 > 0（假设 ByteBuffer arg 中有空间）

Which is why you use a Selector, and because it can in one select call "select" 1Ks of SocketChannels that have bytes ready to read

这就是您使用 Selector 的原因，并且因为它可以在一次选择调用中“选择”1Ks 的 SocketChannels，这些 SocketChannels 已准备好读取字节

Now there's nothing wrong with using SocketChannels in their default blocking mode; and given your description (a client or two), there's probably no reason to as its simpler; but if you want to use non-blocking Channels, use a Selector

现在在默认阻塞模式下使用 SocketChannel 没有任何问题；根据您的描述（一两个客户），可能没有理由认为它更简单；但如果你想使用非阻塞通道，请使用选择器