Mismatching size of the input training tensor and target

Hello everyone

I am trying to train my 1d conv autoencoder. In part of training the NN,I faced with the below error:

//////////////////////////////////////////////////////////////////////////////////////////////////
“Using a target size (torch.Size([2048, 1, 94])) that is different to the input size (torch.Size([2048, 1, 93])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
return F.mse_loss(input, target, reduction=self.reduction)”
//////////////////////////////////////////////////////////////////////////////////

I am quite sure that my input tensor has the right size. because just before running the training loop, when ever I checked the size of that it gave me the right answer.
My input size is (2040,1,94) which 2040 is batch number in every singel training, 1 is the input channel size and 94 is length of my signal.

below is my training codes:

#training loop


for epoch in range(num_epochs):
for data in dataloader:
img = data

    # ===================forward=====================
    output = model(img)
    loss = criterion(output, img)
    # ===================backward====================
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

If you change your code to the following, what does it print?

    output = model(img)
    print("out", output.size())
    print("img", img.size())
    loss = criterion(output,  img)
1 Like

HI
Thank you for your reply,
I did that. below you could see the results:

out torch.Size([2048, 1, 93])
img torch.Size([2048, 1, 94])

That looks like your model does not output the correct size… It could happen in many places, I’d recommend printing the shape of internal variables in the forward method of your network, so that you can pinpoint the exact location that introduces the error and debug that.

It would also help if you can share the code of your architecture! But it would be understandable if you can’t :slight_smile:

1 Like

Hi
Thank you for your reply.
My codes are not really something important :slight_smile:
Below you could be able to see them all:
//////////////////////////////////////////////////////////////////////////////////
#Autoencoder and Autodecoder conv layer…
class Autoencoder(nn.Module):

def init(self):
super(Autoencoder,self).init()

    self.encoder = nn.Sequential(
        nn.Conv1d(1,5,kernel_size=5, stride=2),
        nn.ReLU(True),
        nn.Conv1d(5,10,kernel_size=5, stride = 2),
        nn.ReLU(True),
        nn.Conv1d(10,20,kernel_size=5, stride = 2),
        nn.ReLU(True))
    self.decoder = nn.Sequential(          
        nn.ConvTranspose1d(20,10,kernel_size=5, stride = 2),
        nn.ReLU(True),
        nn.ConvTranspose1d(10,5,kernel_size=5, stride = 2),
        nn.ReLU(True),
        nn.ConvTranspose1d(5,1,kernel_size=5, stride = 2),
        nn.Sigmoid())

def forward(self,x):
x = self.encoder(x)
x = self.decoder(x)
return x
//////////////////////////////////////////////////////////////////
num_epochs = 32
batch_size = 2048
learning_rate = 1e-3
////////////////////////////////////////////////////////////
model = Autoencoder().cpu()
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate,
weight_decay=1e-5)
dataloader = DataLoader(data_pixel2, batch_size=batch_size, shuffle=True)
///////////////////////////////////////////////////////////////////////////////

#training loop


for epoch in range(num_epochs):
for data in dataloader:
img = data

    # ===================forward=====================
    output = model(img)
    print("out", output.size())
    print("img", img.size())
    loss = criterion(output, img)
    # ===================backward====================
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
# ===================log========================
print('epoch [{}/{}], loss:{:.4f}'
      .format(epoch+1, num_epochs, loss.data[0]))
if epoch % 10 == 0:
    pic = to_img(output.cpu().data)
    save_image(pic, './dc_img/image_{}.png'.format(epoch))

///////////////////////////////////////////////

Regards,

I think I know where the error comes from…

Your convolutions have a stride of 2. This means roughly (I hope I’m right) that each conv layer divides your input size by 2 (and removes a few elements on the borders due to no padding by default in the conv layer).

By printing the shape of the output of each layer, which you can do for the encoder with the following (the same with the decoder):

for module in self.encoder:
    x = module(x)
    print(x.shape)

You will see that the shape follows the following sequence: [94, 45, 21, 9] in the encoder, but the decoder gives [9, 21, 45, 93].

Now, I’m not familiar with the autoencoder architecture, specifically with how you need to setup your input sizes and kernel sizes for correct downsizing and upsizing… So you’ll need to check some documentation on that, I’m afraid this is the furthest I can help you.

I’ve just tried quickly changing the kernel size of the outermost convolutions (so the first conv in the encoder and the last conv transpose of the decoder) to 6 instead of 5, and this resulted in an output size of 94, but I’m not sure this is what you want or if it’s even valid…

P.S: if you format your code properly (by inserting it in a code block enclosed in two groups of three backticks `), it would be much easier to read it and help you! Copy-pasting the code directly failed, because the __init__ was changed to init due to markdown’s formatting shortcuts.

1 Like

HI
Thank you for your clear explanation. My problem actually solved, however when I run the training loop on my network loss of my NN was about 1088392064.000000 …

Do you have any idea ???

Is that the training loss? Does it start at this point or does it increase and stay there? Does it explode after that point?

That can be many things, but try reducing the learning rate first, and see how that goes.

Yes it is training loss as far as I understand.
Below you can see my codes in order to calculate the loss:
#training loop

for epoch in range(num_epochs):
for data in dataloader:
img = data
img = Variable(img)

output = model (img)
loss = criterion (output,img)

optimizer.zero_grad()
loss.backward()
optimizer.step()

if epoch%10 == 1:
  print ('\r Train Epoch : {}/{} \tLoss : {:.6f}'.format (epoch+1,num_epochs,loss))

I also wrote you down the loss of the last epoch :
Train Epoch : 72/80 Loss : 1088451712.000000
Train Epoch : 72/80 Loss : 1088443520.000000
Train Epoch : 72/80 Loss : 1088414208.000000
Train Epoch : 72/80 Loss : 1088391808.000000
Train Epoch : 72/80 Loss : 1088355584.000000
Train Epoch : 72/80 Loss : 1088334464.000000

I see.

Well the MSELoss can be quite large due to its nature, but this is still very high… It also depends on the nature of your problem (so your X and y values).

Also, does it start at the same values in the first epoch? This might indicate that your network is not learning anything.

1 Like

Hi Alex
Sorry for late replying back.
m.mmmm you are right.
Actually I realized that I did not normalized my data. Do you have any idea that how I can do it by using data loader module ? I tried to use below code, however I don’t have good feeling about that. I believe that there might be a way of normalizing in pytorch rather than do it by for loops
//////////////////////////////////////////////////////////////////////////////////////////////
#again reshape the data file
new_data = np.empty((256,256,94))
for i in range (256) :
for j in range (256) :
for k in range (94) :

  new_data[i,j,k] = (data[k,i,j]-(np.mean(data[:,i,j])))/((np.amax(data[:,i,j]))-(np.amin([data[:,i,j]])))

////////////////////////////////////////////////////////////////////////////////////////

Thank you

Indeed, this is far from efficient, and good thinking for the normalization, it slipped my mind!

I’m not sure if this is the right way for your kind of data (I’ve only worked on images), but in general, what you want to do is compute your statistics in advance, on the whole training set (do not include validation or test sets, as this would introduce some bias), and then normalize input data in the dataset (not the data loader!) as it gets called, using your previously computed stats.

1 Like