Hi,
My GPU Nvidia gtx1050 Ti
I am trying to train it on GPU but I only see CPU utilization 60-90 percent and GPU around 5 percent during training may be due to the copying of tensors to GPU I don’t know. But it just goes up 5 percent and comes down.
I tried increasing batch size to 64 or 128 based on some solutions online but it just gives me Cuda out of memory error. I have used like 2.3gb out of 4gb and says need 120 MB and has only 110 mb.---- I dont even understand how it works at this point
Then tried decreasing batch size to 16 then also got Cuda out of memory error saying need like 40 MB and has 16mb same 2.3 GB used out of 4gb
In the end it worked for 8 batch size but it only uses CPU no GPU
I used fastai with batch size 128 it works fine and uses gpu . I dont know where I did wrong. Any help appreciated. Below is my code wrote based on pytorch image classifier tutorial
Model Resnet pretrained True
has 205 labels with 117000 images approx as data for training.
Just using pretrained weights such that weights will modify accordingly without freezing weights and not training from scratch I assume is what the code does. Feel free to correct me if I did something wrong or better solution. In the end I am just noob in pytorch … My first code in pytorch
trainset… PIL Image format with rgb converted to prevent error of 4 channel and removed if not opened by PIL --invalid images: corrupted image
device = torch.device("cuda:0") #0 device is my nvidia gtx 1050 ti when printed
model.fc=nn.Linear(2048, 205)
from torchvision import transforms
t = transforms.Compose([
transforms.Resize(256), #[2]
transforms.CenterCrop(224), #[3]
transforms.ToTensor(), #[4]
transforms.Normalize( #[5]
mean=[0.485, 0.456, 0.406], #[6]
std=[0.229, 0.224, 0.225] ) #[7]
])
trainloader = torch.utils.data.DataLoader(trainset, batch_size=8,
shuffle=True)
if torch.cuda.is_available():
print('yes gpu')
torch.set_default_tensor_type('torch.cuda.FloatTensor')
model = model.cuda()
import torch.optim as optim
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
for epoch in range(6): # loop over the dataset multiple times
running_loss = 0.0
for i, data in enumerate(trainloader, 0):
# get the inputs; data is a list of [inputs, labels]
inputs, labels,img_name = data
inputs = inputs.to(device)
labels = labels.to(device)
# zero the parameter gradients
optimizer.zero_grad()
# forward + backward + optimize
outputs = model(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
# print statistics
running_loss += loss.item()
total=0
correct=0
if i % 2000 == 1999:
# print every 2000 mini-batches
print('[%d, %5d] loss: %.3f' %
(epoch + 1, i + 1, running_loss / 2000))
outputs = model(inputs)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print('Accuracy of the network on the batch size images: %d %%' % (
100 * correct / total)) #training accuraccy
running_loss = 0.0
print('Finished Training')