将二进制 NodeJS 缓冲区转换为 JavaScript ArrayBuffer

Question

提问by Drake Amara

How can I convert a NodeJS binary buffer into a JavaScript ArrayBuffer?

如何将 NodeJS 二进制缓冲区转换为 JavaScript ArrayBuffer？

Answer 1

回答by Martin Thomson

Instances of Bufferare also instances of Uint8Arrayin node.js 4.x and higher. Thus, the most efficient solution is to access the buf.bufferproperty directly, as per https://stackoverflow.com/a/31394257/1375574. The Buffer constructor also takes an ArrayBufferView argument if you need to go the other direction.

的实例Buffer也是Uint8Arraynode.js 4.x 及更高版本中的实例。因此，最有效的解决方案是buf.buffer直接访问该属性，如https://stackoverflow.com/a/31394257/1375574。如果您需要向另一个方向移动，则 Buffer 构造函数还需要一个 ArrayBufferView 参数。

Note that this will not create a copy, which means that writes to any ArrayBufferView will write through to the original Buffer instance.

请注意，这不会创建副本，这意味着写入任何 ArrayBufferView 都将写入原始 Buffer 实例。

在旧版本中，node.js 将 ArrayBuffer 作为 v8 的一部分，但 Buffer 类提供了更灵活的 API。为了读取或写入 ArrayBuffer，您只需要创建一个视图并进行复制。

From Buffer to ArrayBuffer:

从缓冲区到数组缓冲区：

function toArrayBuffer(buf) {
    var ab = new ArrayBuffer(buf.length);
    var view = new Uint8Array(ab);
    for (var i = 0; i < buf.length; ++i) {
        view[i] = buf[i];
    }
    return ab;
}

From ArrayBuffer to Buffer:

从 ArrayBuffer 到 Buffer：

function toBuffer(ab) {
    var buf = Buffer.alloc(ab.byteLength);
    var view = new Uint8Array(ab);
    for (var i = 0; i < buf.length; ++i) {
        buf[i] = view[i];
    }
    return buf;
}

Answer 2

回答by ZachB

No dependencies, fastest, Node.js 4.x and later

无依赖，最快，Node.js 4.x 及更高版本

Buffers are Uint8Arrays, so you just need to slice (copy) its region of the backing ArrayBuffer.

Buffers 是Uint8Arrays，所以你只需要切片（复制）它的 backing 区域ArrayBuffer。

// Original Buffer
let b = Buffer.alloc(512);
// Slice (copy) its segment of the underlying ArrayBuffer
let ab = b.buffer.slice(b.byteOffset, b.byteOffset + b.byteLength);

The sliceand offset stuff is requiredbecause small Buffers (less than 4 kB by default, half the pool size) can be views on a shared ArrayBuffer. Without slicing, you can end up with an ArrayBuffercontaining data from another Buffer. See explanation in the docs.

在slice和偏移的东西是需要，因为小Buffer秒（小于4kB的默认状态下，一半的池大小）可以是在共享视图ArrayBuffer。如果不进行切片，您最终会得到一个ArrayBuffer包含来自另一个Buffer. 请参阅文档中的说明。

If you ultimately need a TypedArray, you can create one without copying the data:

如果您最终需要一个TypedArray，则可以在不复制数据的情况下创建一个：

// Create a new view of the ArrayBuffer without copying
let ui32 = new Uint32Array(b.buffer, b.byteOffset, b.byteLength / Uint32Array.BYTES_PER_ELEMENT);

No dependencies, moderate speed, any version of Node.js

无依赖，中等速度，任何版本的 Node.js

Use Martin Thomson's answer, which runs in O(n)time. (See also my replies to comments on his answer about non-optimizations. Using a DataView is slow. Even if you need to flip bytes, there are faster ways to do so.)

使用Martin Thomson 的答案，它在O(n)时间内运行。（另请参阅我对他关于非优化的回答的评论的回复。使用 DataView 很慢。即使您需要翻转字节，也有更快的方法。）

Dependency, fast, Node.js ≤ 0.12 or iojs 3.x

依赖，快，Node.js ≤ 0.12 或 iojs 3.x

You can use https://www.npmjs.com/package/memcpyto go in either direction (Buffer to ArrayBuffer and back). It's faster than the other answers posted here and is a well-written library. Node 0.12 through iojs 3.x require ngossen's fork (see this).

您可以使用https://www.npmjs.com/package/memcpy向任一方向前进（从 Buffer 到 ArrayBuffer 再返回）。它比此处发布的其他答案更快，并且是一个编写良好的库。节点 0.12 到 iojs 3.x 需要 ngossen 的 fork（请参阅此）。

Answer 3

回答by kraag22

"From ArrayBuffer to Buffer" could be done this way:

“从 ArrayBuffer 到 Buffer”可以这样完成：

var buffer = Buffer.from( new Uint8Array(ab) );

Answer 4

回答by David Fooks

A quicker way to write it

一种更快的书写方式

var arrayBuffer = new Uint8Array(nodeBuffer).buffer;

However, this appears to run roughly 4 times slower than the suggested toArrayBuffer function on a buffer with 1024 elements.

但是，在具有 1024 个元素的缓冲区上，这似乎比建议的 toArrayBuffer 函数慢 4 倍。

Answer 5

回答by Константин Ван

1. A `Buffer`is just a viewfor looking into an `ArrayBuffer`.

1.一种`Buffer`仅仅是一个视图用于寻找到的`ArrayBuffer`。

A Buffer, in fact, is a FastBuffer, which extends(inherits from) Uint8Array, which is an octet-unit view(“partial accessor”) of the actual memory, an ArrayBuffer.

A Buffer，实际上是一个FastBuffer，它extends（继承自）Uint8Array，它是实际内存的一个八位字节单元视图（“部分访问器”），一个ArrayBuffer.

/lib/buffer.js#L65-L73/lib/buffer.js#L65-L73^{Node.js 9.4.0}^{Node.js 9.4.0}

class FastBuffer extends Uint8Array {
  constructor(arg1, arg2, arg3) {
    super(arg1, arg2, arg3);
  }
}
FastBuffer.prototype.constructor = Buffer;
internalBuffer.FastBuffer = FastBuffer;

Buffer.prototype = FastBuffer.prototype;

2. The size of an `ArrayBuffer`and the size of its viewmay vary.

2. an`ArrayBuffer`的大小及其视图的大小可能会有所不同。

Reason #1: `Buffer.from(arrayBuffer[, byteOffset[, length]])`.

理由1： `Buffer.from(arrayBuffer[, byteOffset[, length]])`。

With Buffer.from(arrayBuffer[, byteOffset[, length]]), you can create a Bufferwith specifying its underlying ArrayBufferand the view's position and size.

使用Buffer.from(arrayBuffer[, byteOffset[, length]])，您可以创建一个Buffer并指定其底层ArrayBuffer和视图的位置和大小。

const test_buffer = Buffer.from(new ArrayBuffer(50), 40, 10);
console.info(test_buffer.buffer.byteLength); // 50; the size of the memory.
console.info(test_buffer.length); // 10; the size of the view.

Reason #2: `FastBuffer`'s memory allocation.

原因#2：`FastBuffer`的内存分配。

It allocates the memory in two different ways depending on the size.

它根据大小以两种不同的方式分配内存。

If the size is less than the half of the size of a memory pooland is not 0 (“small”): it makes use of a memory poolto prepare the required memory.
Else: it creates a dedicated ArrayBufferthat exactly fits the required memory.

如果大小小于内存池大小的一半并且不为 0（“小”）：它使用内存池来准备所需的内存。
否则：它创建了一个ArrayBuffer完全适合所需内存的专用内存。

/lib/buffer.js#L306-L320/lib/buffer.js#L306-L320^{Node.js 9.4.0}^{Node.js 9.4.0}

function allocate(size) {
  if (size <= 0) {
    return new FastBuffer();
  }
  if (size < (Buffer.poolSize >>> 1)) {
    if (size > (poolSize - poolOffset))
      createPool();
    var b = new FastBuffer(allocPool, poolOffset, size);
    poolOffset += size;
    alignPool();
    return b;
  } else {
    return createUnsafeBuffer(size);
  }
}

/lib/buffer.js#L98-L100/lib/buffer.js#L98-L100^{Node.js 9.4.0}^{Node.js 9.4.0}

function createUnsafeBuffer(size) {
  return new FastBuffer(createUnsafeArrayBuffer(size));
}

What do you mean by a “memory pool?”

“内存池”是什么意思？

A memory poolis a fixed-size pre-allocatedmemory block for keeping small-size memory chunks for Buffers. Using it keeps the small-size memory chunks tightly together, so prevents fragmentationcaused by separate management (allocation and deallocation) of small-size memory chunks.

甲存储器池是一个固定大小的预先分配用于保持小尺寸存储块用于存储块Buffer秒。使用它可以将小内存块紧密结合在一起，从而防止因单独管理（分配和释放）小内存块而导致碎片化。

In this case, the memory pools are ArrayBuffers whose size is 8 KiB by default, which is specified in Buffer.poolSize. When it is to provide a small-size memory chunk for a Buffer, it checks if the last memory pool has enough available memory to handle this; if so, it creates a Bufferthat “views”the given partial chunk of the memory pool, otherwise, it creates a new memory pool and so on.

在这种情况下，内存池是ArrayBuffers，默认大小为 8 KiB，在Buffer.poolSize. 当它要为 a 提供一个小尺寸的内存块时Buffer，它会检查最后一个内存池是否有足够的可用内存来处理这个；如果是的话，它会创建一个Buffer该“意见”的内存池的给定的部分块，否则，将创建一个新的内存池等。

You can access the underlying ArrayBufferof a Buffer. The Buffer's bufferproperty(that is, inherited from Uint8Array) holds it. A “small”Buffer's bufferproperty is an ArrayBufferthat represents the entire memory pool.So in this case, the ArrayBufferand the Buffervaries in size.

您可以访问底层ArrayBuffer的Buffer。该Buffer的buffer财产（即从继承Uint8Array）持有它。一个“小”Buffer的buffer属性是ArrayBuffer代表整个内存池。所以在这种情况下，ArrayBuffer和的Buffer大小不同。

const zero_sized_buffer = Buffer.allocUnsafe(0);
const small_buffer = Buffer.from([0xC0, 0xFF, 0xEE]);
const big_buffer = Buffer.allocUnsafe(Buffer.poolSize >>> 1);

// A `Buffer`'s `length` property holds the size, in octets, of the view.
// An `ArrayBuffer`'s `byteLength` property holds the size, in octets, of its data.

console.info(zero_sized_buffer.length); /// 0; the view's size.
console.info(zero_sized_buffer.buffer.byteLength); /// 0; the memory..'s size.
console.info(Buffer.poolSize); /// 8192; a memory pool's size.

console.info(small_buffer.length); /// 3; the view's size.
console.info(small_buffer.buffer.byteLength); /// 8192; the memory pool's size.
console.info(Buffer.poolSize); /// 8192; a memory pool's size.

console.info(big_buffer.length); /// 4096; the view's size.
console.info(big_buffer.buffer.byteLength); /// 4096; the memory's size.
console.info(Buffer.poolSize); /// 8192; a memory pool's size.

3. So we need to extract the memory it “views.”

3. 所以我们需要提取它“查看”的内存。

An ArrayBufferis fixed in size, so we need to extract it out by making a copy of the part. To do this, we use Buffer's byteOffsetpropertyand lengthproperty, which are inherited from Uint8Array, and the ArrayBuffer.prototype.slicemethod, which makes a copy of a part of an ArrayBuffer. The slice()-ing method herein was inspired by @ZachB.

An 的ArrayBuffer大小是固定的，因此我们需要通过制作零件的副本来提取它。要做到这一点，我们使用Buffer的byteOffset财产和length财产，这是从继承Uint8Array，并且该ArrayBuffer.prototype.slice方法，这使得一个部分的副本ArrayBuffer。这里的slice()-ing 方法受到@ZachB 的启发。

const test_buffer = Buffer.from(new ArrayBuffer(10));
const zero_sized_buffer = Buffer.allocUnsafe(0);
const small_buffer = Buffer.from([0xC0, 0xFF, 0xEE]);
const big_buffer = Buffer.allocUnsafe(Buffer.poolSize >>> 1);

function extract_arraybuffer(buf)
{
    // You may use the `byteLength` property instead of the `length` one.
    return buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.length);
}

// A copy -
const test_arraybuffer = extract_arraybuffer(test_buffer); // of the memory.
const zero_sized_arraybuffer = extract_arraybuffer(zero_sized_buffer); // of the... void.
const small_arraybuffer = extract_arraybuffer(small_buffer); // of the part of the memory.
const big_arraybuffer = extract_arraybuffer(big_buffer); // of the memory.

console.info(test_arraybuffer.byteLength); // 10
console.info(zero_sized_arraybuffer.byteLength); // 0
console.info(small_arraybuffer.byteLength); // 3
console.info(big_arraybuffer.byteLength); // 4096

4. Performance improvement

4. 性能提升

If you're to use the results as read-only, or it is okay to modify the input Buffers' contents, you can avoid unnecessary memory copying.

如果您要将结果用作只读，或者可以修改输入Buffers 的内容，则可以避免不必要的内存复制。

const test_buffer = Buffer.from(new ArrayBuffer(10));
const zero_sized_buffer = Buffer.allocUnsafe(0);
const small_buffer = Buffer.from([0xC0, 0xFF, 0xEE]);
const big_buffer = Buffer.allocUnsafe(Buffer.poolSize >>> 1);

function obtain_arraybuffer(buf)
{
    if(buf.length === buf.buffer.byteLength)
    {
        return buf.buffer;
    } // else:
    // You may use the `byteLength` property instead of the `length` one.
    return buf.subarray(0, buf.length);
}

// Its underlying `ArrayBuffer`.
const test_arraybuffer = obtain_arraybuffer(test_buffer);
// Just a zero-sized `ArrayBuffer`.
const zero_sized_arraybuffer = obtain_arraybuffer(zero_sized_buffer);
// A copy of the part of the memory.
const small_arraybuffer = obtain_arraybuffer(small_buffer);
// Its underlying `ArrayBuffer`.
const big_arraybuffer = obtain_arraybuffer(big_buffer);

console.info(test_arraybuffer.byteLength); // 10
console.info(zero_sized_arraybuffer.byteLength); // 0
console.info(small_arraybuffer.byteLength); // 3
console.info(big_arraybuffer.byteLength); // 4096

Answer 6

回答by Feross

Use the following excellent npm package: to-arraybuffer.

使用以下优秀NPM包：to-arraybuffer。

Or, you can implement it yourself. If your buffer is called buf, do this:

或者，您可以自己实现它。如果您的缓冲区被调用buf，请执行以下操作：

buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength)

Answer 7

回答by Benny Neugebauer

You can think of an ArrayBufferas a typed Buffer.

您可以将 anArrayBuffer视为键入的Buffer.

An ArrayBuffertherefore always needs a type (the so-called "Array Buffer View"). Typically, the Array Buffer Viewhas a type of Uint8Arrayor Uint16Array.

一个ArrayBuffer因此总是需要一个类型（即所谓“数组缓冲区视图”）。通常，数组缓冲区视图的类型为Uint8Array或Uint16Array。

There is a good article from Renato Mangini on converting between an ArrayBuffer and a String.

Renato Mangini 有一篇关于在 ArrayBuffer 和 String 之间转换的好文章。

I have summarized the essential parts in a code example (for Node.js). It also shows how to convert between the typed ArrayBufferand the untyped Buffer.

我在一个代码示例中总结了基本部分（对于 Node.js）。它还展示了如何在 typedArrayBuffer和 untyped之间进行转换Buffer。

function stringToArrayBuffer(string) {
  const arrayBuffer = new ArrayBuffer(string.length);
  const arrayBufferView = new Uint8Array(arrayBuffer);
  for (let i = 0; i < string.length; i++) {
    arrayBufferView[i] = string.charCodeAt(i);
  }
  return arrayBuffer;
}

function arrayBufferToString(buffer) {
  return String.fromCharCode.apply(null, new Uint8Array(buffer));
}

const helloWorld = stringToArrayBuffer('Hello, World!'); // "ArrayBuffer" (Uint8Array)
const encodedString = new Buffer(helloWorld).toString('base64'); // "string"
const decodedBuffer = Buffer.from(encodedString, 'base64'); // "Buffer"
const decodedArrayBuffer = new Uint8Array(decodedBuffer).buffer; // "ArrayBuffer" (Uint8Array)

console.log(arrayBufferToString(decodedArrayBuffer)); // prints "Hello, World!"

Answer 8

回答by Dlabz

This Proxy will expose the buffer as any of the TypedArrays, without any copy. :

此代理将缓冲区公开为任何 TypedArrays，没有任何副本。：

https://www.npmjs.com/package/node-buffer-as-typedarray

It only works on LE, but can be easily ported to BE. Also, never got to actually test how efficient this is.

它仅适用于 LE，但可以轻松移植到 BE。此外，从来没有真正测试过它的效率。

Answer 9

回答by Gopalakrishna Palem

Now there is a very useful npm package for this: bufferhttps://github.com/feross/buffer

现在有一个非常有用的 npm 包：https: buffer//github.com/feross/buffer

It tries to provide an API that is 100% identical to node's Buffer API and allow:

它试图提供一个与节点的 Buffer API 100% 相同的 API 并允许：

convert typed array to buffer: https://github.com/feross/buffer#convert-typed-array-to-buffer
convert buffer to typed array: https://github.com/feross/buffer#convert-buffer-to-typed-array

将类型化数组转换为缓冲区：https: //github.com/feross/buffer#convert-typed-array-to-buffer
将缓冲区转换为类型化数组：https: //github.com/feross/buffer#convert-buffer-to-typed-array

and few more.

还有更多。

Answer 10

回答by Exitos

I tried the above for a Float64Array and it just did not work.

我为 Float64Array 尝试了上述方法，但它不起作用。

I ended up realising that really the data needed to be read 'INTO' the view in correct chunks. This means reading 8 bytes at a time from the source Buffer.

我最终意识到确实需要将数据以正确的块读取到视图中。这意味着一次从源缓冲区读取 8 个字节。

Anyway this is what I ended up with...

无论如何，这就是我最终的结果......

var buff = new Buffer("40100000000000004014000000000000", "hex");
var ab = new ArrayBuffer(buff.length);
var view = new Float64Array(ab);

var viewIndex = 0;
for (var bufferIndex=0;bufferIndex<buff.length;bufferIndex=bufferIndex+8)            {

    view[viewIndex] = buff.readDoubleLE(bufferIndex);
    viewIndex++;
}

将二进制 NodeJS 缓冲区转换为 JavaScript ArrayBuffer

提问by Drake Amara

回答by Martin Thomson

回答by ZachB

No dependencies, fastest, Node.js 4.x and later

无依赖，最快，Node.js 4.x 及更高版本

No dependencies, moderate speed, any version of Node.js

无依赖，中等速度，任何版本的 Node.js

Dependency, fast, Node.js ≤ 0.12 or iojs 3.x

依赖，快，Node.js ≤ 0.12 或 iojs 3.x

回答by kraag22

回答by David Fooks

回答by Константин Ван

1. A `Buffer`is just a viewfor looking into an `ArrayBuffer`.

1.一种`Buffer`仅仅是一个视图用于寻找到的`ArrayBuffer`。

2. The size of an `ArrayBuffer`and the size of its viewmay vary.

2. an`ArrayBuffer`的大小及其视图的大小可能会有所不同。

Reason #1: `Buffer.from(arrayBuffer[, byteOffset[, length]])`.

理由1： `Buffer.from(arrayBuffer[, byteOffset[, length]])`。

Reason #2: `FastBuffer`'s memory allocation.

原因#2：`FastBuffer`的内存分配。

What do you mean by a “memory pool?”

“内存池”是什么意思？

3. So we need to extract the memory it “views.”

3. 所以我们需要提取它“查看”的内存。

4. Performance improvement

4. 性能提升

回答by Feross

回答by Benny Neugebauer

回答by Dlabz

回答by Gopalakrishna Palem

回答by Exitos

相关推荐

最近更新

标签

将二进制 NodeJS 缓冲区转换为 JavaScript ArrayBuffer

提问by Drake Amara

回答by Martin Thomson

回答by ZachB

No dependencies, fastest, Node.js 4.x and later

无依赖，最快，Node.js 4.x 及更高版本

No dependencies, moderate speed, any version of Node.js

无依赖，中等速度，任何版本的 Node.js

Dependency, fast, Node.js ≤ 0.12 or iojs 3.x

依赖，快，Node.js ≤ 0.12 或 iojs 3.x

回答by kraag22

回答by David Fooks

回答by Константин Ван

1. A Bufferis just a viewfor looking into an ArrayBuffer.

1.一种Buffer仅仅是一个视图用于寻找到的ArrayBuffer。

2. The size of an ArrayBufferand the size of its viewmay vary.

2. anArrayBuffer的大小及其视图的大小可能会有所不同。

Reason #1: Buffer.from(arrayBuffer[, byteOffset[, length]]).

理由1： Buffer.from(arrayBuffer[, byteOffset[, length]])。

Reason #2: FastBuffer's memory allocation.

原因#2：FastBuffer的内存分配。

What do you mean by a “memory pool?”

“内存池”是什么意思？

3. So we need to extract the memory it “views.”

3. 所以我们需要提取它“查看”的内存。

4. Performance improvement

4. 性能提升

回答by Feross

回答by Benny Neugebauer

回答by Dlabz

回答by Gopalakrishna Palem

回答by Exitos

相关推荐

Javascript 使用多维数组创建多维对象

你能用 JavaScript 触发自定义 HTML5 表单错误吗？

Javascript 如何使用 jQuery 运行 MySQL 查询？

Javascript 如何计算数组中每个项目的出现次数？

相关推荐

最近更新

标签

1. A `Buffer`is just a viewfor looking into an `ArrayBuffer`.

1.一种`Buffer`仅仅是一个视图用于寻找到的`ArrayBuffer`。

2. The size of an `ArrayBuffer`and the size of its viewmay vary.

2. an`ArrayBuffer`的大小及其视图的大小可能会有所不同。

Reason #1: `Buffer.from(arrayBuffer[, byteOffset[, length]])`.

理由1： `Buffer.from(arrayBuffer[, byteOffset[, length]])`。

Reason #2: `FastBuffer`'s memory allocation.

原因#2：`FastBuffer`的内存分配。