Running torch.nn.Sequential on CUDA breaks length of item vector

vision
1

Hi there,

I’m working on updating the wonderful library that Nilesh Verma wrote - DeepImageSearch (GitHUB). It is a library for finding similar images. It is really accurate in super simple to use.

Nilesh’s library uses CPU for loading the pre-trained model and removing the last layer.

The problem with CPU based process is that it is very slow - 4 it/s. So for 10.000 images processing takes an hour. And I have a 1 TB of images to process.

I wanted to transfer this process to GPU. So I updated this code from

        print("\033[91m Please Wait Model Is Loading or Downloading From Server!")
        base_model = timm.create_model(self.model_name, pretrained=self.pretrained)
        self.model = torch.nn.Sequential(*list(base_model.children())[:-1])

to

 # Load the pre-trained model and remove the last layer
        print("\033[91m Please Wait Model Is Loading or Downloading From Server!")
        base_model = timm.create_model(self.model_name, pretrained=self.pretrained)
        self.model = torch.nn.Sequential(*list(base_model.children())[:-1]).cuda(0)

which immediately causes Exception:

Metadata and Features are already present, Do you want Extract Again? Enter yes or no
yes
  1%|          | 6/1000 [00:00<00:17, 57.16it/s]/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/PIL/Image.py:992: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images
  warnings.warn(
100%|██████████| 1000/1000 [00:22<00:00, 43.62it/s]
Traceback (most recent call last):
  File "/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/pandas/core/indexes/range.py", line 345, in get_loc
    return self._range.index(new_key)
ValueError: 0 is not in range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/stoneball/PycharmProjects/DeepImageSearch_GPU_V2/DeepImageSearch_GPU_V2.py", line 11, in <module>
    st.run_index()
  File "/home/stoneball/PycharmProjects/DeepImageSearch_GPU_V2/DeepImageSearch.py", line 168, in run_index
    self._start_indexing(data)
  File "/home/stoneball/PycharmProjects/DeepImageSearch_GPU_V2/DeepImageSearch.py", line 140, in _start_indexing
    d = len(image_data['features'][0])  # Length of item vector that will be indexed
  File "/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/pandas/core/series.py", line 1007, in __getitem__
    return self._get_value(key)
  File "/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/pandas/core/series.py", line 1116, in _get_value
    loc = self.index.get_loc(label)
  File "/home/stoneball/DeepImageSearch_GPU_V2/lib/python3.10/site-packages/pandas/core/indexes/range.py", line 347, in get_loc
    raise KeyError(key) from err
KeyError: 0
 Image Meta Information Saved: [metadata-files/vgg19/image_data_features.pkl]

From my understanding using CUDA changes the length of the vector. So from info I found on this forum I went for adding a Flatten layer:

 # Load the pre-trained model and remove the last layer
        print("\033[91m Please Wait Model Is Loading or Downloading From Server!")
        base_model = timm.create_model(self.model_name, pretrained=self.pretrained)
        self.model = torch.nn.Sequential(*list(base_model.children())[:-1]).cuda(0)
        self.model = torch.nn.Flatten()
        self.model.eval()

The code runs successfully, using CUDA speeds up the process 50x on GTX 1080 ti - BUT i break the the accuracy of the model. Similar images are all over the place.

What am I doing wrong?

Thank you for any tips.

2

The error is raised from pandas and I’m unsure how it could be related to the usage of your GPU as it seems to be part of the data loading and processing. Could you check why the indexing fails with the invalid key?

3

Thank you for input @ptrblck. I found that pandas dataframe is empty because of this code:

    def _extract(self, img):
        # Resize and convert the image
        img = img.resize((224, 224))
        img = img.convert('RGB')

        # Preprocess the image
        preprocess = transforms.Compose([
            transforms.ToTensor(),
            transforms.Normalize(mean=[0.485, 0.456, 0.406],std=[0.229,0.224, 0.225]),
        ])
        x = preprocess(img)
        x = Variable(torch.unsqueeze(x, dim=0).float(), requires_grad=False)

        # Extract features
        feature = self.model(x)
        feature = feature.data.numpy().flatten()
        return feature / np.linalg.norm(feature)

The output of this code is input into pandas data frame.

So my question is: if I run this code in CPU image features are extracted properly - how can I run this code on GPU?

4

You would have to move the model and input data to the GPU first and should be able to execute the same code. It’s still unclear how an empty result can be returned if the same code is executed so check which object is set to None.

5

I’m trying to wrap my head around :slight_smile:

So I understand - model is loaded to GPU.

I understand I have to load tensor data also to GPU - which I tried:

    def _extract(self, img):
        # Resize and convert the image
        img = img.resize((224, 224))
        img = img.convert('RGB')

        # Preprocess the image
        preprocess = transforms.Compose([
            transforms.ToTensor(),
            transforms.Normalize(mean=[0.485, 0.456, 0.406],std=[0.229,0.224, 0.225]),
        ])

        x = preprocess(img)
        **x = x.cuda(0)**
        x = Variable(torch.unsqueeze(x, dim=0).float(), requires_grad=False)


        # Extract features
        feature = self.model(x)
        feature = feature.data.numpy().flatten()
        return feature / np.linalg.norm(feature)

and than I get Exception:

Exception: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

I understand I have to copy the tensor to CPU and back to GPU. How can I do that?

Thank you @ptrblck for helping out! ML is a totally different beast to tame, than anything else I did in Python.

6

The new error is most likely raised in:

feature = feature.data.numpy().flatten()

and you would need to detach the tensor and move the the CPU before applying numpy operations on it:

feature = feature.detach().cpu().numpy().flatten()

Also, don’t use the deprecated Variable and .data attribute.

Note however, that using numpy operations (and thus detaching the tensor) will not allow you to backpropagate through these operations anymore in case that’s needed.