Numpy not Available

1

Hello All,
I am trying to use TPUs on a Kaggle Notebook and installing Pytorch XLA as follows -

!pip3 install mkl

!curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py

!python3 pytorch-xla-env-setup.py --version nightly --apt-packages libomp5 libopenblas-dev

However once Pytorch XLA is installed, It does not let me use numpy functions like numpy.uint8 or for that matter even torch.from_numpy() does not work. The error always says - Numpy not Available
The whole stack is as follows -

RuntimeError                              Traceback (most recent call last)
<ipython-input-1-abfcbbc939b0> in <module>
   1026                                  segmentation_Maps='/kaggle/input/pascal-voc/VOC2012/SegmentationClass/') 
   1027 dataloader = DataLoader(dataset, batch_size=5)
-> 1028 for _, data in enumerate(dataloader):
   1029     i = data['image']
   1030     gt = data['ground_truth']

/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py in __next__(self)
    519             if self._sampler_iter is None:
    520                 self._reset()
--> 521             data = self._next_data()
    522             self._num_yielded += 1
    523             if self._dataset_kind == _DatasetKind.Iterable and \

/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py in _next_data(self)
    559     def _next_data(self):
    560         index = self._next_index()  # may raise StopIteration
--> 561         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    562         if self._pin_memory:
    563             data = _utils.pin_memory.pin_memory(data)

/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py in fetch(self, possibly_batched_index)
     42     def fetch(self, possibly_batched_index):
     43         if self.auto_collation:
---> 44             data = [self.dataset[idx] for idx in possibly_batched_index]
     45         else:
     46             data = self.dataset[possibly_batched_index]

/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py in <listcomp>(.0)
     42     def fetch(self, possibly_batched_index):
     43         if self.auto_collation:
---> 44             data = [self.dataset[idx] for idx in possibly_batched_index]
     45         else:
     46             data = self.dataset[possibly_batched_index]

<ipython-input-1-abfcbbc939b0> in __getitem__(self, item)
    939         print(mask.shape)
    940         image = Image.fromarray(np.uint8(image)).convert('RGB')
--> 941         mask = torch.from_numpy(np.uint8(mask))
    942 
    943         image = self.transforms(image)

RuntimeError: Numpy is not available

I have no clue what exactly is going on here. Does anybody know how to solve it ?
PS - Pytorch XLA updates PyTorch to a nightly 1.9 version, so maybe that causes an issue ?
TIA

2

Ok. I’ve had this one before. I think (I’m only hypothesizing) that you have to refer to Numpy as np forever after. I think. You’ve referred to it as from_numpy. Maybe I’m just running the wrong track and the error is something else.

If this doesn’t work, I’m super sorry, I’m under 15 years of age.

3

Are you getting the same error after installing numpy via pip install numpy?

1 Like
4

@ptrblck
Sorry. I didn’t read and undetstnad the question fully. Yes, have you checked your Numpy installation?

5

Thank you for replying

Yep, I have tried installing NumPy both before and after installing PyTorch XLA.

I believe the error rises from a NumPy version mismatch.

I had missed a warning which essentially said - Compiled with numpy version 0xe but found 0xd.

The problem is that pip install --upgrade numpy does not work because Kaggle loads a few modules (like Numpy and Pandas) . So one has to reload the module.
I did that too but it didn’t work.

If someone has successfully done it please let me know.

PS - Sorry this was not a pytorch specific question. I had’nt thought of numpy mismatch and environment pre-loading

1 Like
6

Do you need to update numpy or generally reinstall it? Based on the warning it seems that your PyTorch setup already ships with the right numpy version (the binaries would use numpy 1.15, if I’m not mistaken).

7

Honestly, I have no clue, based on the error message, I tried updating Numpy, and reloading the module, and even uninstalling and installing it again. but to no avail.
If I understand the error message properly, pytorch XLA was compiled with version 0xe but the numpy version on system is 0xd, correct ?.
Also, pytorch XLA updates the pytorch installation to 1.9, so maybe that’ why?(I am literally guessing now)

8

Use numpy directly to convert the image to tensor instead of ToTensor(). The following worked for me.
img=torch.tensor(np.array(img,dtype=np.float64))/255.0

1 Like
9

Hi, installing 1.7 version instead of nightly helped me solve this issue you can try that probably

!python3 pytorch-xla-env-setup.py --version 1.7 --apt-packages libomp5 libopenblas-dev
10

Thanks a lot, I will try it out

11

Install the latest numpy 1.21 solved my problem.

2 Likes
12

Was getting this error after installing torch, everything was installed properly but wouldn’t work until I restarted VSCode.

1 Like
13

Damn, this simple trick solved my issues haha!

1 Like
14

I restarted VS Code and my code started running! xd