Python 了解 torch.nn.Parameter
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/50935345/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Understanding torch.nn.Parameter
提问by Vineet Pandey
I am new to pytorch and I have difficulty in understanding how torch.nn.Parameter()works.
我是 pytorch 的新手，我很难理解它是如何torch.nn.Parameter()工作的。
I have gone through the documentation in https://pytorch.org/docs/stable/nn.htmlbut could make a very little sense out of it.
我已经阅读了https://pytorch.org/docs/stable/nn.html 中的文档，但可能对其意义不大。
Can someone please help?
有人可以帮忙吗？
The code snippet that I am working on:
我正在处理的代码片段：
def __init__(self, weight):
super(Net, self).__init__()
# initializes the weights of the convolutional layer to be the weights of the 4 defined filters
k_height, k_width = weight.shape[2:]
# assumes there are 4 grayscale filters
self.conv = nn.Conv2d(1, 4, kernel_size=(k_height, k_width), bias=False)
self.conv.weight = torch.nn.Parameter(weight)
回答by Astha Sharma
I will break it down for you. Tensors, as you might know, are multi dimensional matrices. Parameter, in its raw form, is a tensor i.e. a multi dimensional matrix. It sub-classes the Variable class.
我会为你分解。您可能知道，张量是多维矩阵。参数，以其原始形式，是一个张量，即一个多维矩阵。它是 Variable 类的子类。
The difference between a Variable and a Parameter comes in when associated with a module. When a Parameter is associated with a module as a model attribute, it gets added to the parameter list automatically and can be accessed using the 'parameters' iterator.
当与模块关联时，变量和参数之间的区别就出现了。当参数作为模型属性与模块关联时，它会自动添加到参数列表中，并且可以使用“参数”迭代器进行访问。
Initially in Torch, a Variable (which could for example be an intermediate state) would also get added as a parameter of the model upon assignment. Later on there were use cases identified where a need to cache the variables instead of having them added to the parameter list was identified.
最初在 Torch 中，变量（例如可以是中间状态）也将在分配时作为模型的参数添加。后来确定了需要缓存变量而不是将它们添加到参数列表的用例。
One such case, as mentioned in the documentation is that of RNN, where in you need to save the last hidden state so you don't have to pass it again and again. The need to cache a Variable instead of having it automatically register as a parameter to the model is why we have an explicit way of registering parameters to our model i.e. nn.Parameter class.
文档中提到的一种情况是 RNN，其中您需要保存最后一个隐藏状态，这样您就不必一次又一次地传递它。需要缓存一个变量而不是让它自动注册为模型的参数，这就是为什么我们有一种显式的方式来向我们的模型注册参数，即 nn.Parameter 类。
For instance, run the following code -
例如，运行以下代码 -
import torch
import torch.nn as nn
from torch.optim import Adam
class NN_Network(nn.Module):
def __init__(self,in_dim,hid,out_dim):
super(NN_Network, self).__init__()
self.linear1 = nn.Linear(in_dim,hid)
self.linear2 = nn.Linear(hid,out_dim)
self.linear1.weight = torch.nn.Parameter(torch.zeros(in_dim,hid))
self.linear1.bias = torch.nn.Parameter(torch.ones(hid))
self.linear2.weight = torch.nn.Parameter(torch.zeros(in_dim,hid))
self.linear2.bias = torch.nn.Parameter(torch.ones(hid))
def forward(self, input_array):
h = self.linear1(input_array)
y_pred = self.linear2(h)
return y_pred
in_d = 5
hidn = 2
out_d = 3
net = NN_Network(in_d, hidn, out_d)
Now, check the parameter list associated with this model -
现在，检查与此模型关联的参数列表 -
for param in net.parameters():
print(type(param.data), param.size())
""" Output
<class 'torch.FloatTensor'> torch.Size([5, 2])
<class 'torch.FloatTensor'> torch.Size([2])
<class 'torch.FloatTensor'> torch.Size([5, 2])
<class 'torch.FloatTensor'> torch.Size([2])
"""
Or try,
或者试试，
list(net.parameters())
This can easily be fed to your optimizer -
这可以很容易地提供给您的优化器 -
opt = Adam(net.parameters(), learning_rate=0.001)
Also, note that Parameters have require_grad set by default.
另请注意，参数默认设置为 require_grad。
回答by prosti
Recent PyTorch releases just have Tensors, it came out the concept of the Variable has deprecated.
最近的 PyTorch 版本只有张量，它出来的变量的概念已被弃用。
Parametersare just Tensors limited to the module they are defined (in the module constructor __init__method).
参数只是受限于它们定义的模块的张量（在模块构造函数__init__方法中）。
They will appear inside module.parameters().
This comes handy when you build your custom modules, that learn thanks to these parameters gradient descent.
他们会出现在里面module.parameters()。当您构建自定义模块时，这会很方便，由于这些参数梯度下降而学习。
Anything that is true for the PyTorch tensors is true for parameters, since they are tensors.
任何对 PyTorch 张量适用的东西对参数也适用，因为它们是张量。
Additionally, if module goes to GPU, parameters goes as well. If module is saved parameters will also be saved.
此外，如果模块去 GPU，参数也会去。如果模块被保存，参数也将被保存。
There is a similar concept to model parameters called buffers.
有一个类似于模型参数的概念，称为buffers。
These are named tensors inside the module, but these tensors are not meant to learn via gradient descent, instead you can think these are like variables. You will update your named buffers inside module forward()as you like.
这些在模块内被命名为张量，但这些张量并不意味着通过梯度下降来学习，相反，您可以认为它们就像变量。您将根据需要更新模块内的命名缓冲区forward()。
For buffers, it is also true they will go to GPU with the module, and they will be saved together with the module.
对于buffers，也确实会和模块一起去GPU，和模块一起保存。
