# No GPU utilization although CUDA seems to be activated

**URL:** <https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964>\
**Category:** vision\
**Created:** [March 26, 2019, 11:17pm UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964 "2019-03-26T23:17:30Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 26, 2019, 11:17pm UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/1 "2019-03-26T23:17:30Z")

</div>

Hi,

I have an Alienware laptop with GeForce GTX 980M , and I’m trying to run my first code in pytorch - using transfer learning with resnet.  
The thing is that I get no GPU utilization although all CUDA signs in python seems to be ok:

print(“torch.cuda.is\_available() =”, torch.cuda.is\_available())  
print(“torch.cuda.device\_count() =”, torch.cuda.device\_count())  
print(“torch.cuda.device(‘cuda’) =”, torch.cuda.device(‘cuda’))  
print(“torch.cuda.current\_device() =”, torch.cuda.current\_device())  
print(“torch.cuda.get\_device\_name(0) =”,torch.cuda.get\_device\_name(0))  
cudnn.benchmark = True  
#Additional Info when using cuda  
if device.type == ‘cuda’:  
print(torch.cuda.get\_device\_name(0))  
print(‘Memory Usage:’)  
print(‘Allocated:’, round(torch.cuda.memory\_allocated(0)/1024**3,1), ‘GB’)  
print('Cached: ', round(torch.cuda.memory\_cached(0)/1024**3,1), ‘GB’)

torch.cuda.is\_available() = True  
torch.cuda.device\_count() = 1  
torch.cuda.device(‘cuda’) = \<torch.cuda.device object at 0x0000021F70F15D68\>  
torch.cuda.current\_device() = 0  
torch.cuda.get\_device\_name(0) = GeForce GTX 980M  
GeForce GTX 980M  
Memory Usage:  
Allocated: 0.0 GB  
Cached: 0.0 GB

I tried googling it and also looked here but came up with nothing helpful.  
running nvidia-smi also shows that there is no GPU memory usage, and the following message:

```
    Process ID : 19792
        Type : C
        Name : C:\Users\itaim\AppData\Local\Programs\Python\Python36\python.exe
        Used GPU Memory : Not available in WDDM driver model

```

attaching task manager screenshot and nviidia-smi output:

 ![image](https://discuss.pytorch.org/uploads/default/original/2X/f/f15780e4b68d61551372b4e937e6a5fef819c220.png)

nvidia-smi -l

C:\Users\itaim\>nvidia-smi -l  
±----------------------------------------------------------------------------+  
| NVIDIA-SMI 419.35 Driver Version: 419.35 CUDA Version: 10.1 |  
|-------------------------------±---------------------±---------------------+  
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |  
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |  
|===============================+======================+======================|  
| 0 GeForce GTX 980M WDDM | 00000000:01:00.0 Off | N/A |  
| N/A 66C P0 69W / N/A | 917MiB / 8192MiB | 51% Default |  
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+  
| Processes: GPU Memory |  
| GPU PID Type Process name Usage |  
|=============================================================================|  
| 0 19792 C …cal\Programs\Python\Python36\python.exe N/A |  
±----------------------------------------------------------------------------+  
nvidia-smi -q

C:\Users\itaim\>nvidia-smi -q

==============NVSMI LOG==============

Timestamp : Wed Mar 27 01:09:08 2019  
Driver Version : 419.35  
CUDA Version : 10.1

Attached GPUs : 1  
GPU 00000000:01:00.0  
Product Name : GeForce GTX 980M  
Product Brand : GeForce  
Display Mode : Disabled  
Display Active : Disabled  
Persistence Mode : N/A  
Accounting Mode : Disabled  
Accounting Mode Buffer Size : 4000  
Driver Model  
Current : WDDM  
Pending : WDDM  
Serial Number : N/A  
GPU UUID : GPU-544e0a63-680d-08ca-c2da-1ef57adb2fff  
Minor Number : N/A  
VBIOS Version : 84.04.87.00.08  
MultiGPU Board : No  
Board ID : 0x100  
GPU Part Number : N/A  
Inforom Version  
Image Version : N/A  
OEM Object : N/A  
ECC Object : N/A  
Power Management Object : N/A  
GPU Operation Mode  
Current : N/A  
Pending : N/A  
GPU Virtualization Mode  
Virtualization mode : None  
IBMNPU  
Relaxed Ordering Mode : N/A  
PCI  
Bus : 0x01  
Device : 0x00  
Domain : 0x0000  
Device Id : 0x13D710DE  
Bus Id : 00000000:01:00.0  
Sub System Id : 0x07081028  
GPU Link Info  
PCIe Generation  
Max : 3  
Current : 3  
Link Width  
Max : 16x  
Current : 8x  
Bridge Chip  
Type : N/A  
Firmware : N/A  
Replays Since Reset : 0  
Replay Number Rollovers : 0  
Tx Throughput : 59000 KB/s  
Rx Throughput : 26000 KB/s  
Fan Speed : N/A  
Performance State : P0  
Clocks Throttle Reasons  
Idle : Not Active  
Applications Clocks Setting : Not Active  
SW Power Cap : Not Active  
HW Slowdown : Not Active  
HW Thermal Slowdown : N/A  
HW Power Brake Slowdown : N/A  
Sync Boost : Not Active  
SW Thermal Slowdown : Not Active  
Display Clock Setting : Not Active  
FB Memory Usage  
Total : 8192 MiB  
Used : 827 MiB  
Free : 7365 MiB  
BAR1 Memory Usage  
Total : 256 MiB  
Used : 229 MiB  
Free : 27 MiB  
Compute Mode : Default  
Utilization  
Gpu : 39 %  
Memory : 25 %  
Encoder : 0 %  
Decoder : 0 %  
Encoder Stats  
Active Sessions : 0  
Average FPS : 0  
Average Latency : 0  
FBC Stats  
Active Sessions : 0  
Average FPS : 0  
Average Latency : 0  
Ecc Mode  
Current : N/A  
Pending : N/A  
ECC Errors  
Volatile  
Single Bit  
Device Memory : N/A  
Register File : N/A  
L1 Cache : N/A  
L2 Cache : N/A  
Texture Memory : N/A  
Texture Shared : N/A  
CBU : N/A  
Total : N/A  
Double Bit  
Device Memory : N/A  
Register File : N/A  
L1 Cache : N/A  
L2 Cache : N/A  
Texture Memory : N/A  
Texture Shared : N/A  
CBU : N/A  
Total : N/A  
Aggregate  
Single Bit  
Device Memory : N/A  
Register File : N/A  
L1 Cache : N/A  
L2 Cache : N/A  
Texture Memory : N/A  
Texture Shared : N/A  
CBU : N/A  
Total : N/A  
Double Bit  
Device Memory : N/A  
Register File : N/A  
L1 Cache : N/A  
L2 Cache : N/A  
Texture Memory : N/A  
Texture Shared : N/A  
CBU : N/A  
Total : N/A  
Retired Pages  
Single Bit ECC : N/A  
Double Bit ECC : N/A  
Pending : N/A  
Temperature  
GPU Current Temp : 66 C  
GPU Shutdown Temp : 96 C  
GPU Slowdown Temp : 91 C  
GPU Max Operating Temp : 101 C  
Memory Current Temp : N/A  
Memory Max Operating Temp : N/A  
Power Readings  
Power Management : N/A  
Power Draw : 41.15 W  
Power Limit : N/A  
Default Power Limit : N/A  
Enforced Power Limit : N/A  
Min Power Limit : N/A  
Max Power Limit : N/A  
Clocks  
Graphics : 1126 MHz  
SM : 1126 MHz  
Memory : 2505 MHz  
Video : 1036 MHz  
Applications Clocks  
Graphics : 1038 MHz  
Memory : 2505 MHz  
Default Applications Clocks  
Graphics : 1038 MHz  
Memory : 2505 MHz  
Max Clocks  
Graphics : 1126 MHz  
SM : 1126 MHz  
Memory : 2505 MHz  
Video : 1036 MHz  
Max Customer Boost Clocks  
Graphics : N/A  
Clock Policy  
Auto Boost : N/A  
Auto Boost Default : N/A  
Processes  
Process ID : 20840  
Type : C  
Name : C:\Users\itaim\AppData\Local\Programs\Python\Python36\python.exe  
Used GPU Memory : Not available in WDDM driver model

and my code:

import torch  
import torchvision  
import torch.nn as nn  
import torch.nn.functional as F  
from torchvision import transforms, datasets, models  
from torch.utils.data import Dataset, DataLoader  
from torch.autograd import Variable  
import torch.optim as optim  
from torch.optim import lr\_scheduler  
from torch.utils.data.sampler import SubsetRandomSampler  
import torch.backends.cudnn as cudnn  
import matplotlib.pyplot as plt  
import numpy as np  
import torch.optim as optim  
import pandas as pd  
import os  
from PIL import Image  
import pickle  
import urllib  
from urllib.request import urlopen  
import time  
import copy

device = torch.device(“cuda:0” if torch.cuda.is\_available() else “cpu”)

print(“torch.cuda.is\_available() =”, torch.cuda.is\_available())  
print(“torch.cuda.device\_count() =”, torch.cuda.device\_count())  
print(“torch.cuda.device(‘cuda’) =”, torch.cuda.device(‘cuda’))  
print(“torch.cuda.current\_device() =”, torch.cuda.current\_device())  
print(“torch.cuda.get\_device\_name(0) =”,torch.cuda.get\_device\_name(0))

cudnn.benchmark = True

#Additional Info when using cuda  
if device.type == ‘cuda’:  
print(torch.cuda.get\_device\_name(0))  
print(‘Memory Usage:’)  
print(‘Allocated:’, round(torch.cuda.memory\_allocated(0)/1024**3,1), ‘GB’)  
print('Cached: ', round(torch.cuda.memory\_cached(0)/1024**3,1), ‘GB’)

class DogsTrainingDataset(Dataset):

```
def __init__ (self,text_file,root_dir):
    """
    Args:
        text_file(string): path to text file
        root_dir(string): directory with all train images
    """
    self.name_frame = pd.read_csv(text_file,sep=",",usecols=range(1))
    self.label_frame = pd.read_csv(text_file,sep=",",usecols=range(1,2))
    self.label_idx_frame = pd.read_csv(text_file,sep=",",usecols=range(2,3))
    self.root_dir = root_dir
    self.transform = transforms.Compose([
         transforms.Resize(224),
         transforms.CenterCrop(224),
         transforms.ToTensor(),
         transforms.Normalize( mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
         ])

def __len__ (self):
    return len(self.name_frame)

def __getitem__ (self, idx):
    img_name = os.path.join(self.root_dir, self.name_frame.iloc[idx, 0]) + '.jpg'
    label_str = self.label_frame.iloc[idx, 0]
    label = self.label_idx_frame.iloc[idx, 0]
    image = Image.open(img_name)
    image = self.transform(image)
    #labels = labels.reshape(-1, 2)
    return [image, label, label_str]

```

def imshow(img, title=None):  
inp = img.numpy().transpose((1, 2, 0))  
mean = np.array([0.485, 0.456, 0.406])  
std = np.array([0.229, 0.224, 0.225])  
inp = std \* inp + mean  
inp = np.clip(inp, 0, 1)  
plt.imshow(inp)  
if title is not None:  
plt.title(title)  
plt.pause(0.001) # pause a bit so that plots are updated

def train\_model(model, criterion, optimizer, scheduler, validation\_split, dataloaders, num\_epochs=25):  
since = time.time()

```
best_model_wts = copy.deepcopy(model.state_dict())
best_acc = 0.0
best_epoch_idx = 0
train_loss = []
train_acc = []
val_loss = []
val_acc = []

val_size = np.floor(len(dataloaders['train'].dataset) * validation_split)
train_size = len(dataloaders['train'].dataset) - val_size

for epoch in range(num_epochs):
    print('Epoch {}/{}'.format(epoch, num_epochs - 1))
    print('-' * 10)

    # Each epoch has a training and validation phase
    for phase in ['train', 'val']:
        if phase == 'train':
            dataset_size = train_size
            scheduler.step()
            model.train() # Set model to training mode
        else:
            dataset_size = val_size
            model.eval() # Set model to evaluate mode

        running_loss = 0.0
        running_corrects = 0

        # Iterate over data.
        for inputs, labels, labels_str in dataloaders[phase]:
            inputs = inputs.to(device)
            labels = labels.to(device)

            # zero the parameter gradients
            optimizer.zero_grad()

            # forward
            # track history if only in train
            with torch.set_grad_enabled(phase == 'train'):
                #if(phase == 'train'):
                # outputs,aux = model(inputs)
                #else:
                # outputs = model(inputs)
                outputs = model(inputs)
                _, preds = torch.max(outputs, 1)
                loss = criterion(outputs, labels)

                # backward + optimize only if in training phase
                if phase == 'train':
                    loss.backward()
                    optimizer.step()

            # statistics
            running_loss += loss.item() * inputs.size(0)
            running_corrects += torch.sum(preds == labels.data)
        epoch_loss = running_loss / dataset_size
        epoch_acc = running_corrects.double() / dataset_size
        
        print('{} Loss: {:.4f} Acc: {:.4f}'.format(
            phase, epoch_loss, epoch_acc))
        # save loss and accuracy
        if phase == 'train':
            train_loss.append(epoch_loss)
            train_acc.append(epoch_acc)
        else:
            val_loss.append(epoch_loss)
            val_acc.append(epoch_acc)
        # deep copy the model
        if phase == 'val' and epoch_acc > best_acc:
            best_acc = epoch_acc
            best_epoch_idx = epoch
            best_model_wts = copy.deepcopy(model.state_dict())

time_elapsed = time.time() - since
print('Training complete in {:.0f}m {:.0f}s'.format(
    time_elapsed // 60, time_elapsed % 60))
print('Best val Acc: {:4f}'.format(best_acc))

# plot loss and accuracy

plt.figure()
plt.xlabel('# of epochs')
plt.title('Accuracy')
plt.plot(range(num_epochs), train_acc, label='Train Accuracy')
plt.plot(range(num_epochs), val_acc, label='Validation Accuracy')
plt.legend()
plt.figure()
plt.xlabel('# of epochs')
plt.title('Loss')
plt.plot(range(num_epochs), train_loss, label='Train Loss')
plt.plot(range(num_epochs), val_loss, label='Validation Loss')
plt.legend()
print('best epoch index:',best_epoch_idx)
# load best model weights
model.load_state_dict(best_model_wts)    
return model

```

dogsTrainSet = DogsTrainingDataset(text\_file =‘C:\Users\itaim\Master\TeamLily\labels.csv’,  
root\_dir = ‘C:\Users\itaim\Master\TeamLily\train’)  
num\_epochs = 10  
batch\_size = 16  
validation\_split = .2  
shuffle\_dataset = True  
random\_seed= 42

# Creating data indices for training and validation splits:

dataset\_size = len(dogsTrainSet)  
indices = list(range(dataset\_size))  
split = int(np.floor(validation\_split \* dataset\_size))  
if shuffle\_dataset :  
np.random.seed(random\_seed)  
np.random.shuffle(indices)  
train\_indices, val\_indices = indices[split:], indices[:split]

# Creating PT data samplers and loaders:

train\_sampler = SubsetRandomSampler(train\_indices)  
val\_sampler = SubsetRandomSampler(val\_indices)

# handle data loaders

dogsTrainLoader = DataLoader(dogsTrainSet, batch\_size=16, sampler=train\_sampler, pin\_memory=True)  
dogsValLoader = DataLoader(dogsTrainSet, batch\_size=16, sampler=val\_sampler, pin\_memory=True)  
dataloaders = {‘train’: dogsTrainLoader , ‘val’: dogsValLoader}

# init model and its attributes

model = models.resnet50(pretrained=True)  
#model.aux\_logit=False

# option 1 - use the model as feature extractor (train only classifier)

for param in model.parameters():  
param.requires\_grad = False  
num\_ftrs = model.fc.in\_features  
model.fc = nn.Linear(num\_ftrs, 120)  
model = model.to(device)

criterion = nn.CrossEntropyLoss()

# choose SG with momentum

optimizer = optim.SGD(model.fc.parameters(), lr=0.001, momentum=0.9)

# Decay LR by a factor of 0.1 every 7 epochs

exp\_lr\_scheduler = lr\_scheduler.StepLR(optimizer, step\_size=7, gamma=0.1)

# train the model and evaluate on the validation set

print(‘#########################################’)  
print(‘running with SGD’)  
print(‘#########################################’)  
model\_trained1 = train\_model(model, criterion, optimizer,  
exp\_lr\_scheduler, validation\_split, dataloaders, num\_epochs=25)

print(‘#########################################’)  
print(‘running with Adam’)  
print(‘#########################################’)  
optimizer = optim.Adam(model.fc.parameters(), lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight\_decay=0, amsgrad=False)  
exp\_lr\_scheduler = lr\_scheduler.StepLR(optimizer, step\_size=7, gamma=0.1)  
model\_trained2 = train\_model(model, criterion, optimizer,  
exp\_lr\_scheduler, validation\_split, dataloaders, num\_epochs=25)

any suggestions?

Thanks in advance 🙂

---

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [March 27, 2019, 9:40am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/2 "2019-03-27T09:40:40Z")

</div>

Hi,

You can print the inputs before feeding them to the model to make sure they are on the gpu.  
It is possible that your model is so small that the gpu is barely used as well, you can try increasing the batch size to see if the gpu usage increases.

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 27, 2019, 4:39pm UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/3 "2019-03-27T16:39:52Z")

</div>

Hi,  
thanks for your response 🙂  
I printed the tensors and they’re on the GPU (device=cuda0).  
I also tried increasing the batch size to 32 and 64 and still nothing…

what about the alarming signs from nvidia-smi, such as the “N/A” in the GPU memory usage for the process, or this:

```
Processes
    Process ID : 15980
        Type : C
        Name : C:\Users\itaim\AppData\Local\Programs\Python\Python36\python.exe
        Used GPU Memory : Not available in WDDM driver model

```

what is this WDDM?

Thanks,  
Itai

---

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [March 27, 2019, 4:44pm UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/4 "2019-03-27T16:44:22Z")

</div>

I have to admit I don’t know how these things work on windows…  
Maybe @peterjc123 will know what’s happening here?

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 28, 2019, 8:46am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/5 "2019-03-28T08:46:35Z")

</div>

@peterjc123 have any idea??

---

<div class="post-metadata">

**Author:** ![peterjc123](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/peterjc123/32/2790_2.png) [@peterjc123](https://discuss.pytorch.org/u/peterjc123)\
**Post date:** [March 28, 2019, 8:52am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/6 "2019-03-28T08:52:27Z")

</div>

WDDM is a driver model for GPU under Windows. By using WDDM, you can use it as a device to render your graphics as well as do some math calculating. The alternative one is TCC. By using TCC, it can be only used to do some calculations so that if you don’t have any other GPUs, you can not even boot up your machine. But your ML process won’t get interrupted so it’s great for HPC. Unfortunately, it is only available for Tesla and Quadro cards. Here are the details for the TCC mode: [https://docs.nvidia.com/gameworks/content/developertools/desktop/nsight/tesla\_compute\_cluster.htm](https://docs.nvidia.com/gameworks/content/developertools/desktop/nsight/tesla_compute_cluster.htm).

From what you posted, I don’t think there’s anything wrong there since it says `Not available in WDDM driver model`.

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 28, 2019, 8:54am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/7 "2019-03-28T08:54:22Z")

</div>

Thanks for your response 🙂  
I understand… so my issues are not related to this WDDM thing.  
have any idea why I get no GPU usage?

---

<div class="post-metadata">

**Author:** ![peterjc123](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/peterjc123/32/2790_2.png) [@peterjc123](https://discuss.pytorch.org/u/peterjc123)\
**Post date:** [March 28, 2019, 8:57am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/8 "2019-03-28T08:57:45Z")

</div>

What do you mean by `get no GPU usage`? If you are referring to the GPU memory usage of a separate process, then it is clearly written there: `Not available in WDDM driver model`. However, the mem usage for the whole GPU should be available through `nvidia-smi`.

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 28, 2019, 9:01am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/9 "2019-03-28T09:01:35Z")

</div>

I’m running my pytorch code (you can take a look at the original post) and I get almost no GPU usage (~1%) when looking in the task manager (also attached above).  
All the CUDA signs from python seems to be ok so I’m not sure what’s going on here…

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 28, 2019, 9:02am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/10 "2019-03-28T09:02:34Z")

</div>

BTW when running “nvidia-smi -l” I see “N/A” in “GPU Memory Usage” of the python process

---

<div class="post-metadata">

**Author:** ![peterjc123](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/peterjc123/32/2790_2.png) [@peterjc123](https://discuss.pytorch.org/u/peterjc123)\
**Post date:** [March 28, 2019, 9:06am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/11 "2019-03-28T09:06:49Z")

</div>

@imesery The graph in Task Manager is confusing. Actually, it doesn’t show anything about gpu mem usage. And the python output is correct because when you call it, you didn’t store any tensors on GPU.

---

<div class="post-metadata">

**Author:** ![peterjc123](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/peterjc123/32/2790_2.png) [@peterjc123](https://discuss.pytorch.org/u/peterjc123)\
**Post date:** [March 28, 2019, 9:11am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/12 "2019-03-28T09:11:43Z")

</div>

You can simply execute the following piece of code to verify whether it’s working.

```python
>>> import torch
>>> a = torch.cuda.FloatTensor(10000)
>>> print("Allocated:", round(torch.cuda.memory_allocated(0)/10243,1), "GB")
Allocated: 3.9 GB
>>> b = torch.cuda.FloatTensor(20000)
>>> print("Allocated:", round(torch.cuda.memory_allocated(0)/10243,1), "GB")
Allocated: 11.8 GB

```

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 28, 2019, 9:13am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/13 "2019-03-28T09:13:17Z")

</div>

but I do store tensors on GPU, when printing the tensors it says they’re on “device=cuda0”.  
attaching another screenshot of task manager during the python run when GPU graph is enlarged.  
the CPU usage went up to ~55% when started running but the GPU usage is at 0.

 ![image](https://discuss.pytorch.org/uploads/default/original/2X/5/5408e8c3f388e3b61a0096b493c4c34a9f86867c.png)

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 28, 2019, 9:16am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/14 "2019-03-28T09:16:29Z")

</div>

import torch

a = torch.cuda.FloatTensor(10000)  
print(“Allocated:”, round(torch.cuda.memory\_allocated(0)/10243,1), “GB”)

b = torch.cuda.FloatTensor(20000)  
print(“Allocated:”, round(torch.cuda.memory\_allocated(0)/10243,1), “GB”)

Allocated: 22595.4 GB  
Allocated: 22595.4 GB

---

<div class="post-metadata">

**Author:** ![peterjc123](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/peterjc123/32/2790_2.png) [@peterjc123](https://discuss.pytorch.org/u/peterjc123)\
**Post date:** [March 28, 2019, 9:18am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/15 "2019-03-28T09:18:39Z")

</div>

Yes, it’s because it’s not large enough.  
Try this one:

```python
torch.rand(20000,20000).cuda()

```

It will allocate nearly 2GB.  
P.S. `print(“Allocated:”, round(torch.cuda.memory_allocated(0)/10243,1), “GB”)` this clause is not correct and I just copied it from your code to say that it’s not zero.

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 28, 2019, 9:20am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/16 "2019-03-28T09:20:56Z")

</div>

yes I fixed the clause to be 1024\*\*3.

torch.rand(20000,20000).cuda()  
print(“Allocated:”, round(torch.cuda.memory\_allocated(0)/1024\*\*3,1), “GB”)

Allocated: 0.2 GB

---

<div class="post-metadata">

**Author:** ![peterjc123](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/peterjc123/32/2790_2.png) [@peterjc123](https://discuss.pytorch.org/u/peterjc123)\
**Post date:** [March 28, 2019, 9:23am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/17 "2019-03-28T09:23:29Z")

</div>

You’ll need to save it into a variable, otherwise it will be released.

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 28, 2019, 9:23am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/18 "2019-03-28T09:23:45Z")

</div>

when trying to add this print inside my train\_model function (after each data fetch) I get 0.5GB, added it after this:  
inputs = inputs.to(device)  
labels = labels.to(device)

---

<div class="post-metadata">

**Author:** ![peterjc123](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/peterjc123/32/2790_2.png) [@peterjc123](https://discuss.pytorch.org/u/peterjc123)\
**Post date:** [March 28, 2019, 9:26am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/19 "2019-03-28T09:26:43Z")

</div>

I don’t know how large your data is so I don’t know whether it’s correct. But I think since it’s increasing, it should have no much trouble.

---

<div class="post-metadata">

**Author:** ![imesery](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/imesery/32/32735_2.png) [@imesery](https://discuss.pytorch.org/u/imesery)\
**Post date:** [March 28, 2019, 9:26am UTC](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964/20 "2019-03-28T09:26:48Z")

</div>

ran this again:  
c = torch.rand(20000,20000).cuda()  
print(“Allocated:”, round(torch.cuda.memory\_allocated(0)/1024\*\*3,1), “GB”)

and I get 1.5GB

[Next page](https://discuss.pytorch.org/t/no-gpu-utilization-although-cuda-seems-to-be-activated/40964.md?page=2)