I’m a beginner with pytorch.
I’m using pytorch to build a CNN for object detection.
I encounter something really weird when using DataLoader to feed data.
The size of my input images are [3, 640, 640].
Normally, with 2 GPUs of 12 GB, I can only feed about 8 images once a time. However, when I used DataLoader with batchsize = 8, I marked a very low usage of GPU memory.(About 20%) Then I increased batchsize = 40, I started to see a 90% usage.
To figure out what’s wrong. I run a sample example with resnet50 backbones.
import torch.utils.data
import torchvision
net = torchvision.models.resnet50(pretrained=True)
#net = torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))
net.cuda()
class RandomDataset(torch.utils.data.Dataset):
def __init__(self):
super(RandomDataset, self).__init__()
def __getitem__(self, image_id):
return torch.rand([3,640,640])
def __len__(self):
return 1000
ds = RandomDataset()
dl = torch.utils.data.DataLoader(ds, batch_size=15, shuffle=False, num_workers=8)
with torch.no_grad():
for batch_idx, inputs in enumerate(dl):
net.eval()
inputs = inputs.cuda()
out = net(inputs)
print((torch.cuda.max_memory_cached(0)+torch.cuda.max_memory_allocated(0))/1024/1024/1024)
So, 1 GPU can only feed 15 images normally. (If I don’t use dataloader, GPU reaches 100% usage) While, in DataLoader, it costs only 2.5Gb. It reaches 100% when I set batch_size = 100 which is totally not reasonable.
Can anyone help me with this? Do I make some mistakes somewhere? Or I misunderstand the meaning of batch_size?