GPU utilization always at 40%

1

Whatever I do the GPU utilization is always at 40%, when training and when not training.

GPU: NVIDIA GeForceRTX 4060
PyTorch version: Version: 2.3.0+cu121

This should boost the GPU utilisation to about 90% but it doesnt, what could be the reason?:

import torch
import torch.nn as nn
from torch.utils.data import DataLoader

import torchvision.models as models
import torchvision.datasets as datasets
import torchvision.transforms as transforms


model = models.resnet50()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)

dataset = datasets.FakeData(
    size=1000,
    transform=transforms.ToTensor())
loader = DataLoader(
    dataset,
    num_workers=1,
    pin_memory=True
)

model.to('cuda')

if __name__ == '__main__':
    for data, target in loader:
        data = data.to('cuda', non_blocking=True)
        target = target.to('cuda', non_blocking=True)
        optimizer.zero_grad()
        output = model(data)
        loss = criterion(output, target)
        loss.backward()
        optimizer.step()
    
    print('Done')
2

That seems okay for the size of the batch/data and model you are using. Remember some stuff happens on the CPU so the GPU isn’t always betting utilized. For example, the loader operates in the CPU