I am trying to understand how the data augmentation works in pytorch, so I started with the exemple in the official documentation the faces exemple from my understanding the augmentation in pytorch does not increase the number of samples (does not crete additional ones) but at every epoch it makes random alterations to the existing ones. know if I want to use data augmentation to make additional data. what modifications should I do this is the exemple of the documentation
import os
import torch
import pandas as pd
from skimage import io, transform
import numpy as np
import matplotlib.pyplot as plt
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms, utils
# Ignore warnings
import warnings
warnings.filterwarnings("ignore")
plt.ion() # interactive mode
class FaceLandmarksDataset(Dataset):
"""Face Landmarks dataset."""
def __init__(self, csv_file, root_dir, transform=None):
"""
Arguments:
csv_file (string): Path to the csv file with annotations.
root_dir (string): Directory with all the images.
transform (callable, optional): Optional transform to be applied
on a sample.
"""
self.landmarks_frame = pd.read_csv(csv_file)
self.root_dir = root_dir
self.transform = transform
def __len__(self):
return len(self.landmarks_frame)
def __getitem__(self, idx):
if torch.is_tensor(idx):
idx = idx.tolist()
img_name = os.path.join(self.root_dir,
self.landmarks_frame.iloc[idx, 0])
image = io.imread(img_name)
landmarks = self.landmarks_frame.iloc[idx, 1:]
landmarks = np.array([landmarks], dtype=float).reshape(-1, 2)
sample = {'image': image, 'landmarks': landmarks}
if self.transform:
sample = self.transform(sample)
return sample
# class Rescale (object):
#....
#class RandomCrop (object):
# ....
#class Totensor (object):
# ....
transformed_dataset = FaceLandmarksDataset(csv_file='data/faces/face_landmarks.csv',
root_dir='data/faces/',
transform=transforms.Compose([
Rescale(256),
RandomCrop(224),
ToTensor()
]))
for i, sample in enumerate(transformed_dataset):
print(i, sample['image'].size(), sample['landmarks'].size())
if i == 3:
break
dataloader = DataLoader(transformed_dataset, batch_size=4,
shuffle=True, num_workers=
I actually want to applied on the dataset I am working with .But first I wanted to understand how it works on a simple exemple.
the thing is that I have a small dataset and after the training I got stuck with 0.9 training accuracy but 0.6 validation accuracy .
I saw someone posting about it saying that data augmentation can be done by adding additional samples not only making alterations on existing ones
I wated to try to add aditionnal data perhaps the results gets better
But didn’t know how
I saw something about adding another loop in the get_item function is that correct @ptrblck
like this discussion I want to have for every image multiple augmented images and pass them to the network
It makes sense to add data augmentation, but I would not try to save the augmented samples.
The standard workflow would be to apply the random transformations inside the Dataset.__getitem__ method on each loaded (original) sample. This would create a new, augmented sample in each batch and you could still keep the original dataset small.
yes that’s what I want to apply the random transformations inside the getitem function is that any documentation or exemple that I can understand from how it should be done @ptrblck
You could check this tutorial, which shows how transforms are passed to the Dataset and used in the __getitem__. This tutorial also shows how a custom Dataset is created. In case you are using any built-in Dataset it should most likely provide a transform argument in its __init__.
just one other question in the getitem function it calls transforms and after that you define the transformations exemple and call them in the dataloader
is there a possibility to call the transformation exemple functions in the getitem directly ? @ptrblck
I’m not sure how to understand the question, so let me try to break down the logic.
The __getitem__ method first loads the img_path for the current index and loads the image via read_image. The corresponding label is loaded afterwards.
Next we check if self.transform is defined (default is None) and if so, we apply it on the image. Now image is transformed using the self.transform transformations, which are defined as ToTensor() in this example, but can contain a other (random) transformations, too.
The original image is now gone since the augmented tensor replaced image.
The same applies to the label using self.target_transform.
Finally, the image and label tensor are returned.
Sounds like your model is suffering from overfitting. Image augmentation is one approach to address it. Also, experimenting with a higher p in your dropout layers is another technique that can address overfitting and can be used in parallel to augmentation.
Saving augmented images for reuse would have the effect of increasing overfitting. Instead, you might try experimenting with increasing the number and level of transforms.
Just keep in mind that if your model is overfitting at 90% train and 60% val, the actual model performance might cap out in the 70-80% range. One naive approach for data scientists starting out is thinking that the 90% value is realistic and that they only need to bring up the validation to that level. But the reality is that 90% is likely unrealistic and an average of the train and val accuracy is a more realistic expectation for where your model performance caps out at, given that dataset.
When doing augmentations it is important you consider what augmentations make sense. Not all do.
I can’t tell for sure from your code but it looks like perhaps you are processing faces? That will limit the usable augmentations you can do. Mirroring a face top-to-bottom might not make sense but mirroring it left to right might. Offsetting horizontally and vertically might make sense. Small rotations might make sense but large ones not.
In my work, the images have no real “up” or “down” or “left” and “right” so we do rotations, left-right and top-bottom inversions, and random horizontal and vertical offsets. We create the augmentations and save them before training. We embed the augmentations in the image’s name so we know how the image was created and and augment image can always be tracked back to the original one. This allows us to do checks such as if all the augmented images classify in the same groups as the parent image.
I also require that all augmentations of an image must end up in the same set: Train, Test, or Validate.
yes I understand that but my problem is not with the augmentation
it’s with the size of the dataset it’s very small and I want to add addional sample to augment the size of the dataset not only making alterations to the existing ones
But th implementation in pytorch doesn’t do that so I was wondering about the modification that I should make to this script to make the aditional samples @JimS
in another post someone suggested to use multiple training loop inside the --init–function and in the training concatenate them and used them for the training