GPU does not wake up

1

Dear all,

I am trying to activate the GPU (nvidia P4000) but it keeps at 5% usage whatever I am doing.

I have started the code with that :

use_cuda = torch.cuda.is_available()
device = torch.device(“cuda:0” if use_cuda else “cpu”)

and I get a correct answer from command ‘device’
==> device(type=‘cuda’, index=0)
From command ‘torch.cuda.get_device_name(0)’ I get
==> ‘Quadro P4000’

So everything seems OK.
Then I am loading all the tensors to the GPU :

MyLoss = torch.nn.MSELoss()
batch_size = 10

YCost = Y_tensor(‘this_forecast’,df,indY)
YCost = YCost.to(device)

model = torch.nn.Sequential(torch.nn.Linear(Xdim,Midsize),torch.nn.Tanh(),torch.nn.Linear(Midsize,Ydim))
model = model.to(device)
optim = torch.optim.SGD(params=model.parameters(),lr=EPS)

tensorX = torch.tensor(df[indX].values)
tensorX = tensorX.to(device)
tensorY = Y_tensor(‘last_forecast’,df,indY)
tensorY = tensorY.to(device)

train_tensor_data = torch.utils.data.TensorDataset(tensorX, tensorY)

indices_train, indices_valid = split_indice_block(0.7, 20, len(train_tensor_data))
train_data = torch.utils.data.Subset(train_tensor_data, indices_train)
valid_data = torch.utils.data.Subset(train_tensor_data, indices_valid)

train_loader = DataLoader(train_data,batch_size,shuffle=True)
valid_loader = DataLoader(valid_data, batch_size)

for epoch in range(NB_EPOCH):

  •    model.train()*

  •    for xbatch, ybatch in train_loader:*

  •        xbatch, ybatch = xbatch.float().to(device), ybatch.float().to(device)*

  •        outputs = model(xbatch).to(device)*

  •        loss = MyLoss(outputs,ybatch)*

  •        optim.zero_grad()*

  •        loss.backward()*

  •        optim.step()*

…

CPU is actually asleep at 1% usage.
But the GPU keeps loaded 5% only and running time is incredibly slow.

So what can possibly be wrong ?

Thanks !

2

Hi,

The most likely reason is that the cpu has nothing to do as the dataset is already in memory and so it only needs to start work on the GPU.
The GPU has very little to do as well as the network is small and the batch size is small as well.

Increasing the batch size should increase the gpu usage.
GPUs are built to do big tasks, not a lot of small ones. So if you have a lot of small tasks, you won’t be able to use a GPU efficiently.

3

Thanks for your reply !

It is true that everything is quiet ‘small’ on my context. But still take a lot of time on the CPU so I hoped naively that GPU could help…

Anyway, when I run the program on the CPU, I have a 80% utilization of the CPU and GPU is effectively at 0.
On GPU, CPU is almost at 0 and GPU at 5%.

So my conclusion is that you are right. My small business has nothing to do on the GPU. But still I am wondering why GPU does not help…