Python PyTorch 内存模型：“torch.from_numpy()”与“torch.Tensor()”

PyTorch memory model: "torch.from_numpy()" vs "torch.Tensor()"

提问by kmario23

I'm trying to have an in-depth understanding of how PyTorch Tensor memory model works.

我试图深入了解 PyTorch Tensor 内存模型的工作原理。

# input numpy array
In [91]: arr = np.arange(10, dtype=float32).reshape(5, 2)

# input tensors in two different ways
In [92]: t1, t2 = torch.Tensor(arr), torch.from_numpy(arr)

# their types
In [93]: type(arr), type(t1), type(t2)
Out[93]: (numpy.ndarray, torch.FloatTensor, torch.FloatTensor)

# ndarray 
In [94]: arr
Out[94]: 
array([[ 0.,  1.],
       [ 2.,  3.],
       [ 4.,  5.],
       [ 6.,  7.],
       [ 8.,  9.]], dtype=float32)

---

---

I know that PyTorch tensors share the memory bufferof NumPy ndarrays. Thus, changing one will be reflected in the other. So, here I'm slicing and updating some values in the Tensor t2

我知道 PyTorch 张量共享NumPy ndarrays的内存缓冲区。因此，改变一个将反映在另一个中。所以，我在这里切片和更新张量中的一些值t2

In [98]: t2[:, 1] = 23.0

And as expected, it's updated in t2and arrsince they share the same memory buffer.

正如预期的那样，它更新了t2，arr因为它们共享相同的内存缓冲区。

In [99]: t2
Out[99]: 

  0  23
  2  23
  4  23
  6  23
  8  23
[torch.FloatTensor of size 5x2]


In [101]: arr
Out[101]: 
array([[  0.,  23.],
       [  2.,  23.],
       [  4.,  23.],
       [  6.,  23.],
       [  8.,  23.]], dtype=float32)

But, t1is also updated. Remember that t1was constructed using torch.Tensor()whereas t2was constructed using torch.from_numpy()

但是，t1也更新了。请记住，t1是使用构建的，torch.Tensor()而使用的t2是构建的torch.from_numpy()

In [100]: t1
Out[100]: 

  0  23
  2  23
  4  23
  6  23
  8  23
[torch.FloatTensor of size 5x2]

So, no matter whether we use torch.from_numpy()or torch.Tensor()to construct a tensor from an ndarray, allsuch tensors and ndarrays share the same memory buffer.

因此，无论我们是使用torch.from_numpy()还是torch.Tensor()从 ndarray 构造张量，所有这些张量和 ndarray 都共享相同的内存缓冲区。

Based on this understanding, my question is why does a dedicated function torch.from_numpy()exists when simply torch.Tensor()can do the job?

基于这种理解，我的问题是torch.from_numpy()，当torch.Tensor()可以完成工作时，为什么还存在专用功能？

I looked at the PyTorch documentation but it doesn't mention anything about this? Any ideas/suggestions?

我查看了 PyTorch 文档，但它没有提到任何关于此的内容？任何想法/建议？

回答by Viacheslav Kroilov

from_numpy()automatically inherits input array dtype. On the other hand, torch.Tensoris an alias for torch.FloatTensor.

from_numpy()自动继承输入数组dtype。另一方面，torch.Tensor是 的别名torch.FloatTensor。

Therefore, if you pass int64array to torch.Tensor, output tensor is float tensor and they wouldn't share the storage. torch.from_numpygives you torch.LongTensoras expected.

因此，如果您将int64数组传递给torch.Tensor，则输出张量是浮点张量，它们不会共享存储。按预期torch.from_numpy给你torch.LongTensor。

a = np.arange(10)
ft = torch.Tensor(a)  # same as torch.FloatTensor
it = torch.from_numpy(a)

a.dtype  # == dtype('int64')
ft.dtype  # == torch.float32
it.dtype  # == torch.int64

回答by Jadiel de Armas

The recommended way to build tensors in Pytorch is to use the following two factory functions: torch.tensorand torch.as_tensor.

在 Pytorch 中构建张量的推荐方法是使用以下两个工厂函数：torch.tensor和torch.as_tensor.

torch.tensoralwayscopies the data. For example, torch.tensor(x)is equivalent to x.clone().detach().

torch.tensor总是复制数据。例如，torch.tensor(x)相当于x.clone().detach()。

torch.as_tensoralways triesto avoid copies of the data. One of the cases where as_tensoravoids copying the data is if the original data is a numpy array.

torch.as_tensor总是试图避免数据的副本。之一，其中例as_tensor避免了复制数据是，如果原始数据是numpy的阵列。

回答by eric

This comes from _torch_docs.py; there is also a possible discussion on the "why" here.

这来自_torch_docs.py; 还有一个可能的讨论“为什么”在这里。

def from_numpy(ndarray): # real signature unknown; restored from __doc__
    """
    from_numpy(ndarray) -> Tensor

    Creates a :class:`Tensor` from a :class:`numpy.ndarray`.

    The returned tensor and `ndarray` share the same memory. 
    Modifications to the tensor will be reflected in the `ndarray` 
    and vice versa. The returned tensor is not resizable.

    Example::

        >>> a = numpy.array([1, 2, 3])
        >>> t = torch.from_numpy(a)
        >>> t
        torch.LongTensor([1, 2, 3])
        >>> t[0] = -1
        >>> a
        array([-1,  2,  3])
    """
    pass

Taken from the numpydocs:

取自numpy文档：

Different ndarrayscan share the same data, so that changes made in one ndarray may be visible in another. That is, an ndarraycan be a “view” to another ndarray, and the data it is referring to is taken care of by the “base” ndarray.

不同的ndarrays可以共享相同的数据，因此在一个 ndarray 中所做的更改可能在另一个中可见。也就是说，一个ndarray可以是另一个的“视图” ndarray，它所引用的数据由“基”处理ndarray。

Pytorch docs:

火炬docs：

If a numpy.ndarray, torch.Tensor, or torch.Storageis given, a new tensor that shares the same data is returned. If a Python sequence is given, a new tensor is created from a copy of the sequence.

如果给出 a numpy.ndarray、torch.Tensor或torch.Storage，则返回共享相同数据的新张量。如果给出了 Python 序列，则会从该序列的副本创建一个新的张量。