CNN produces strange ouput

1

Hey there.
I have a grayscale image as input and a grayscale image as output of my CNN. So the input should after enough epochs become similar to the ouput. But it just doesnt. The output seems to be pretty random. I tried more epochs and different learning rates but its always the same.

model = ConvNet().to(device)

criterion = nn.MSELoss()

optimizer = torch.optim.Adam(model.parameters(), lr = learning_rate,weight_decay=0.00001)

for epoch in range(num_epochs):

print('epoch:' + str(epoch+1))

for i, (images,ground_truths) in enumerate(dataloader):

    print(i+1)

    images = images.to(device)

    ground_truths = ground_truths.to(device)

    #zero gradients

    optimizer.zero_grad()

    #forward pass

    output = model(images)

    probe = output.squeeze(0).squeeze(0)

    probe = torch.detach(probe).numpy()

    probe = np.asarray(probe)

    probe = 255*probe

    print(np.max(probe))

    print(probe,'\n')

    

    loss = criterion(output,ground_truths)  

    print(loss)

    

    #Backwardpropagation and optimization

    if epoch == 40:

      optimizer = torch.optim.Adam(model.parameters(), lr = learning_rate_2)

    loss.backward()

    optimizer.step()

torch.save(model,'first_net.pth')
2

Hi, most likely it is because you detach model output before doing backward pass. I believe you have to keep it in the graph do backward, and then detach for some probe purpose.

3

I would like to correct my answer here. Detaching shouldn’t be the issue. I tried a simple dummy example. Consider the following snippet. This code can overfit on random input.

import torch
import torch.nn as nn

def train_model(model, data, actual_labels):
    model.train()
    optimizer.zero_grad()
    output = model(data)
    # detaching here
    probe = output.squeeze(0).squeeze(0)
    probe = torch.detach(probe).numpy()
    print(probe.max())

    loss = loss_function(output, actual_labels)
    loss.backward()
    optimizer.step()
    
    return loss.item()

data  = torch.randn(1000, 2048)
labels = torch.empty(1000, dtype=torch.long).random_(3)
loss_function = nn.CrossEntropyLoss()
model = nn.Linear(2048, 3)
optimizer = torch.optim.Adam(model.parameters())

for epoch in range(100):
    loss = train_model(model, data, labels)
    print(f"Epoch: {epoch}, loss: {loss}")

So, I think it should be some problem in your data processing pipeline or modeling. The first suggestion would be to try and overfit just single example and see if your network could learn this single example. If not, I would try to inspect some data/label mismatch in data processing code. And maybe some reshapes on the batch level in the model code which could introduce some mixing of data points. There are for sure a lot of other possibilities on how we can introduce silent bugs into our neural network related code. For example, I tend to reread this great blog post from Andrej Karpaty about neural networks being such a “leaky abstraction”.

4

Thanks for your answer. So as input i have a numpy array that i change to a pytorch tensor. My ground truth is also a numpy arra that is transformed to a tensor. the values in the numpy array represent the grayscale values of an infrared image. I tried it for only 1 picture and with different learning rates. But i just creates weird output and doesnt seem like its learning. I feel like the update of the parameters is pretty random.

5

Hi, Marcel. It is really hard to tell anything and debug without any code snippets. Also, I can suggest finding some working implementations of some similar tasks to the task you are doing now. Maybe reading some correct implementation can help you to figure out what you are missing in your code. For instance, you can take a look at some autoencoders. I think they are trying to achieve almost the same task. All the best!