Fix random result in lineaIndex RuntimeError

1

Hi,

if I don’t fix random, the following code in yolov3 could be run correctly.
if I fix random with the following code, I experience the following error:


import random
import numpy as np
torch.manual_seed(5)
random.seed(5)
np.random.seed(5)
torch.backends.cudnn.benchmark = False
torch.use_deterministic_algorithms(True)

# t shaple: 75X80
#n=75
#i=0
#tcls shape: list with len 3, each shaple: 75, 55, 22

t[range(n), tcls[i]] = self.cp

Exception has occurred: RuntimeError
linearIndex.numel()*sliceSize*nElemBefore == value.numel()INTERNAL ASSERT FAILED at "/pytorch/aten/src/ATen/native/cuda/Indexing.cu":253, please report a bug to PyTorch. number of flattened indices did not match number of elements in the value tensor751
  File "/trainer/yolov3_train/utils/loss.py", line 142, in __call__
    t[range(n), tcls[i]] = self.cp
  File "/trainer/trainer_yolo.py", line 558, in train
    loss, loss_items = compute_loss(pred, targets.to(device))  # loss scaled by batch_size
  File "/trainer/trainer_yolo.py", line 128, in trainer_yolov3
    self.train(hyp, opt, device, tb_writer=None)
  File "/runner_full.py", line 325, in compressor_runner
    results = trainer.trainer_yolov3(None, None, 99)
  File "runner_full.py", line 405, in <module>
    runner(ectp)
2

Could you update PyTorch to the latest nightly release (or 1.10.1) as this was a known and already fixed issue?

3

@ptrblck ,
Thank you!
In order to use 1.10.1, I have to upgrade cuda.
I will try it.

4

Note that the pip wheels and conda binaries ship with their own CUDA runtime (as specified during the installation) so you would only need a compatible NVIDIA driver (unless you want to build PyTorch or a custom CUDA extension from source which would then use your local CUDA toolkit installation).