[Solved] What is the right way to create a new Variable in forward funciton

#1

Hi, there,

I create a new Variable as the output to play the 4D Tensor batch multiplication in the forward function like this:

def _4D_bmm(self, batch1, batch2):
    x = Variable(torch.Tensor(batch1.size(0), batch1.size(1), batch1.size(3), batch2.size(3)))
    for i in range(batch1.size(0)):
        x[i] = torch.bmm(torch.transpose(batch1[i], 1, 2), batch2[i])
    return x

It can be passed forward, but an error will be thrown in the linear layers:

File "/home/usrname/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/usrname/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/usrname/anaconda2/lib/python2.7/site-packages/torch/nn/modules/linear.py", line 54, in forward
    return self._backend.Linear()(input, self.weight, self.bias)
  File "/home/usrname/anaconda2/lib/python2.7/site-packages/torch/nn/_functions/linear.py", line 10, in forward
    output.addmm_(0, 1, input, weight.t())
TypeError: addmm_ received an invalid combination of arguments - got (int, int, torch.FloatTensor, torch.cuda.FloatTensor), but expected one of:
 * (torch.FloatTensor mat1, torch.FloatTensor mat2)
 * (torch.SparseFloatTensor mat1, torch.FloatTensor mat2)
 * (float beta, torch.FloatTensor mat1, torch.FloatTensor mat2)
 * (float alpha, torch.FloatTensor mat1, torch.FloatTensor mat2)
 * (float beta, torch.SparseFloatTensor mat1, torch.FloatTensor mat2)
 * (float alpha, torch.SparseFloatTensor mat1, torch.FloatTensor mat2)
 * (float beta, float alpha, torch.FloatTensor mat1, torch.FloatTensor mat2)
 * (float beta, float alpha, torch.SparseFloatTensor mat1, torch.FloatTensor mat2)

So what is the right way to state a new Variable?
@apaszke @fmassa Any suggestions to help? Many thanks in advance.

#2 
def _4D_bmm(self, batch1, batch2):
    x =[]
    for i in range(batch1.size(0)):
        x.append( torch.bmm(torch.transpose(batch1[i], 1, 2), batch2[i]))
    return Variable(torch.cat(x)).cuda()

And the exception raises form that model’s parameters are in GPU, while the input is in CPU.

#3

Yeah, thanks. I figure it out.