It is an RNN type of structure. Because of the question setting, it is reasonable to make the initial value of the hidden state to be a concatenation of tensor and parameters. For the tensor part, the initial value is known, and for the parameter part, it is trainable. I try to torch.cat() parameter and tensor, and it seems not possible.
initialvec=torch.zeros(1,1,3)
hidden_train=torch.zeros(1,1,3)
hidden_train=nn.Parameter(hidden_train,requires_grad=True)
torch.cat(hidden_train,initialvec)
Error message
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-34-1f7bbc984497> in <module>
2 hidden_train=torch.zeros(1,1,3)
3 hidden_train=nn.Parameter(hidden_train,requires_grad=True)
----> 4 torch.cat(hidden_train,initialvec)
TypeError: cat(): argument 'tensors' (position 1) must be tuple of Tensors, not Parameter
What’s a good way to concatenate a parameter and tensor?
Thank you.
Best.
Yue
