Not enough memory while extracting Resnet features

vision
1

I am extracting Resnet features for videos(1000-2000 frames each). I am looping for each video, each frame to extract feature for that frame.
But after 10-12 videos I get the error:

Error : Torch: not enough memory: you tried to allocate 38GB. Buy new RAM!

Sample code for feature extraction of one frame

import torch
import torch.nn as nn
import torchvision.models as models
from torch.autograd import Variable

resnet152 = models.resnet152(pretrained=True)
modules=list(resnet152.children())[:-1]
resnet152=nn.Sequential(*modules)
for p in resnet152.parameters():
    p.requires_grad = False

img = torch.Tensor(3, 224, 224).normal_() # random image
img = img.unsqueeze(0)  # add batch dimension
img_var = Variable(img) # assign it to a variable
features_var = resnet152(img_var) # get the output from the last hidden layer of the pretrained resnet
features = features_var.data # get the tensor out of the variable

Solutions tried:

  1. ps -elf | grep python
    kill -9 [pid]

  2. sudo nvidia-smi --gpu-reset

Can anyone please tell how to handle this?

2

Based on the error message it seems you don’t have enough CPU RAM.
Could you observe the RAM with htop or any other tool while loading the features?
Are you trying to store all features of all videos in a list?

Also as a side note, if you just want to use the pretrained model to extract features, you should call model.eval() before using it and use

with torch.no_grad():
    image = ...
    features = resnet152(image)
    ...

to avoid computing intermediate results and save some GPU memory.