Denoising Autoencoder

1

Hi everyone
I have a stupid question,
Is anyone knows that what should be the form of loss function in an Denoising Autoencoder?
should it be like below?;

loss = criterion (model (noisy_data),noise_less_data)
basically model (noisy_data) is the model will be trained with inputs that are corrupted data and loss function calculates the difference (here MSE) between output of the model and the data that are not noisy ?

In this way that makes no sense to me. because if we already have access to noiseless data, then what’s the point of building up a denoising autoencoder?

2

The goal would be to train the model to be able to denoise new data.
The same question might apply to why we would like to train a model to classify dogs and cats, if we already have the labels. :wink:

3 Likes
3

That totally makes sense. :slight_smile:
One more question:
Do u if there is any method (model) that can be do in an unsupervised way? I mean without seeing noiseless data, to Denise noisy data?
Tnx

4

This is most likely not the state of the art technique anymore, but have a look at Deep Image Prior.

1 Like
5

I have one more question,about that:

For both noiseless data as well as noisy input of the NN, I used dataloader (I have two data loader for each) Now I am a bit of worry because the shuffle option is set to True in both data loader. Do you think this might affect my training because I should compare same pixel from the noisy data with the same pixel from the noiseless data …

6

You should use one dataloader instead of two, and get the noise and noiseless data at the same time

1 Like
7

Can you please give me a clue ?
below u can see my DataLoader codes:

class CustomDataset(torch.utils.data.Dataset):
def init(self, data):
self.data = data

def __getitem__(self, index):
    return self.data[index]

def __len__(self):
    return len(self.data)

x = data_ll
separation = int(x.shape[0] * 0.8)
train = x[:separation]
test = x[separation:]

train_dataset = CustomDataset(train)
test_dataset = CustomDataset(test)

train_loader_ll = torch.utils.data.DataLoader(train_dataset, batch_size=2621,shuffle = False)
test_loader_ll = torch.utils.data.DataLoader(test_dataset, batch_size=655,shuffle = False)
print ("//////////////////////////Training dataset//////////////////////////")
for train_sample in train_loader_ll:
print(train_sample.shape)

print ("//////////////////////////Test dataset//////////////////////////")

for test_sample in test_loader_ll:
print(test_sample.shape)

Basically I use this twice. One for noisy data and the other for noiseless data.

8

What it’s the format of your data? I mean x have both noise and noise less data or only one of them?

9

x here is a tensor of the size (65536,1,92) and it is just noisy data.
I made one another class one trainload for the noiseless data.

10

I wish to build a Denoising autoencoder I just use a small definition from another PyTorch thread to add noise in the MNIST dataset. While training my model gives identical loss results. please tell me what I am doing wrong.

def add_noise(inputs):

noise = torch.randn_like(inputs)*0.3
return inputs + noise

Screenshot_120
Screenshot_1201116×729 34.1 KB

11

Could you post the model definition so that we could take a look, if you are accidentally detaching the computation graph?

1 Like
12

Thankyou @ptrblck for the reply. I am posting my model. and please tell me what is wrong

Screenshot_124
Screenshot_124701×617 35.5 KB

13

The model looks generally alright, but I’m not sure if you really want to apply the last F.relu on the output. As a debugging step you could remove it and rerun your training.

PS: you can post code snippets by wrapping them into three backticks ```, which would allow to copy and debug your code :wink:

1 Like