I have a problem concatenating two features tensor when coding an encoder-decoder network in Pytorch. Specifically, in forward function of my network, I have tensor x of size batch x feature1, the result of GRU. And tensor y of size seqLen x batch x feature2. I want to concatenate x into each timestep of y, should output z of size seqLen x batch x (feature1 + feature2).
I have tried allocating a Variable and iterate each timestep, but it doesn’t work. My code looks like this:
z = Variable(torch.zeros(y.size(0), y.size(1), 1024))
for i in range(z.size(0)):
z[i] = torch.cat((y[i], x), 1)
z = self.decoderGRU(z, init_decoder_GRU)
decoded = self.classLinear(z.view(z.size(0)*z.size(1), z.size(2))) #error here.
AttributeError: 'tuple' object has no attribute 'view'
Just to be sure, you want to replicate x along the first (seqLen) dimension when concatenating it, right?
If so, this should do the trick for you and will be much more efficient:
# Replace the for loop with this line. It will expand x to be of size seqLen x batch x feature1
# and concatenate it with y in one go.
z = torch.cat([y, x.unsqueeze(0).expand(seqLen, *x.size())], 2)
About the error message: it seems that the error is related to something else - GRU returns two outputs - the output and hidden state (see the docs), so if you want to retrieve only the output you have to do this: