TypeError in DataLoader worker process 0 (Due to my transforms?)

1

Hi, I’m re-training an inception_v3 using a remote GPU with CUDA device.
I used these transforms for my dataset

train_set = datasets.ImageFolder(
root = “liG”,
transform = transforms.Compose([transforms.ToTensor(),
transforms.RandomRotation(20),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
transforms.Resize(299)]
))
data_loader= torch.utils.data.DataLoader(train_set,
batch_size=5,
shuffle=True,
num_workers=2)
Below is the cell that raises the error:
for X, y in data_loader:
print("Shape of X [N, C, H, W]: ", X.shape)
print("Shape of y: ", y.shape, y.dtype)
break

The error I’m getting is:

TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File “/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py”, line 202, in _worker_loop
data = fetcher.fetch(index)
File “/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py”, line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File “/opt/conda/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py”, line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File “/opt/conda/lib/python3.8/site-packages/torchvision/datasets/folder.py”, line 171, in getitem
sample = self.transform(sample)
File “/opt/conda/lib/python3.8/site-packages/torchvision/transforms/transforms.py”, line 60, in call
img = t(img)
File “/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 881, in _call_impl
result = self.forward(*input, **kwargs)
File “/opt/conda/lib/python3.8/site-packages/torchvision/transforms/transforms.py”, line 1236, in forward
fill = [float(f) for f in fill]
TypeError: ‘NoneType’ object is not iterable

2

Based on the error message it seems that fill is set to None in this line of code, which shouldn’t be the case, as it’s checked here.

I also cannot reproduce the issue using this code snippet:

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.RandomRotation(20),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
    transforms.Resize(299)
])

img = transforms.ToPILImage()(torch.randn(3, 224, 224))
out = transform(img)

Which PyTorch and torchvision versions are you using and are you seeing the same issue using my code?

3

@ptrblck Caught TypeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop data = fetcher.fetch(index) File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/lib/python3.8/dist-packages/torchvision/datasets/folder.py", line 229, in __getitem__ sample = self.loader(path) File "/usr/local/lib/python3.8/dist-packages/torchvision/transforms/transforms.py", line 95, in __call__ img = t(img) File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(*input, **kwargs) File "/usr/local/lib/python3.8/dist-packages/torchvision/transforms/transforms.py", line 661, in forward img = F.pad(img, self.padding, self.fill, self.padding_mode) File "/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional.py", line 523, in pad return F_pil.pad(img, padding=padding, fill=fill, padding_mode=padding_mode) File "/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py", line 154, in pad raise TypeError(f"img should be PIL Image. Got {type(img)}") TypeError: img should be PIL Image. Got <class 'str'>

this was the error I was facing ,Sam training my model on .npy files, can u pls help me out?

4

It seems you are passing a str to your transformation while a PIL.Image (or tensor) is expected.

5 
@torch.no_grad()
def evaluate(model, val_loader):
    model.eval()
    outputs = [model.validation_step(batch) for batch in val_loader]
    return model.validation_epoch_end(outputs)

def get_lr(optimizer):
    for param_group in optimizer.param_groups:
        return param_group['lr']

def fit_one_cycle(epochs, max_lr, model, train_loader, val_loader, 
                  weight_decay=0, grad_clip=None, opt_func=torch.optim.SGD):
    torch.cuda.empty_cache()
    history = []
    
    # Set up cutom optimizer with weight decay
    optimizer = opt_func(model.parameters(), max_lr, weight_decay=weight_decay)
    # Set up one-cycle learning rate scheduler
    sched = torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr, epochs=epochs, 
                                                steps_per_epoch=len(train_loader))
    
    for epoch in range(epochs):
        # Training Phase 
        model.train()
        train_losses = []
        lrs = []
        for batch in train_loader:
            loss = model.training_step(batch)
            train_losses.append(loss)
            loss.backward()
            
            # Gradient clipping
            if grad_clip: 
                nn.utils.clip_grad_value_(model.parameters(), grad_clip)
            
            optimizer.step()
            optimizer.zero_grad()
            
            # Record & update learning rate
            lrs.append(get_lr(optimizer))
            sched.step()
        
        # Validation phase
        result = evaluate(model, val_loader)
        result['train_loss'] = torch.stack(train_losses).mean().item()
        result['lrs'] = lrs
        model.epoch_end(epoch, result)
        history.append(result)
    return history

I have been handling with .npy files …I don’t knew where I was going wrong @ptrblck pls help me out

6 
# Data transforms (normalization & data augmentation)
stats = ((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
train_tfms = tt.Compose([
                         tt.RandomHorizontalFlip(), 
                         tt.ToPILImage(),
                         # tt.RandomResizedCrop(256, scale=(0.5,0.9), ratio=(1, 1)), 
                         tt.ColorJitter(brightness=0.1, contrast=0.1, saturation=0.1, hue=0.1),
                         tt.ToTensor(), 
                         tt.Normalize(*stats,inplace=True)])
valid_tfms = tt.Compose([tt.ToTensor(), tt.Normalize(*stats)])
7

how can I tackle this problem?

8

Check the inputs to your transformation and make sure to pass valid objects as PIL.Images or tensors instead of strings.

1 Like
9

— Epoch 0 — — Phase train —

TypeError Traceback (most recent call last)

in <cell line: 8>() 21 data_loader = test_loader 22 —> 23 for batch_i, (X, y) in enumerate(dataloaders[phase]): 24 image_sequences = Variable(X.to(device), requires_grad=True) 25 labels = Variable(y.to(device), requires_grad=False)

3 frames

/usr/local/lib/python3.10/dist-packages/torch/_utils.py in reraise(self) 692 # instantiate since we don’t know how to 693 raise RuntimeError(msg) from None → 694 raise exception 695 696

TypeError: Caught TypeError in DataLoader worker process 0. Original Traceback (most recent call last): File “/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/worker.py”, line 308, in _worker_loop data = fetcher.fetch(index) File “/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py”, line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File “/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py”, line 51, in data = [self.dataset[idx] for idx in possibly_batched_index] File “”, line 21, in getitem img1 = img[:,128i:128(i+1),:] TypeError: ‘NoneType’ object is not subscriptable

10

It seems img in __getitem__ is None so check the dataset implementation to see why that’s the case.