Prediction error : RuntimeError: CUDA error: an illegal memory access was encountered

I encountered this error when computing MSE using a trained model. The related codes are as follows:

for j in range(5):

    a = torch.randn(40, 8)
    pred = net(a)
    loss = ((pred - y_train)**2).mean()

    train_and_eval(trainloader, testloader, lr, optimizer, loss_func, net, y_test, Epoch = 1)

It seems that when I use pred, it will throw an error(even when I just print it). How can I fix this?

When I run the code in debug mode, I found that when j is 0, pred is a normal tensor, but when j is 1, pred turned to become torch.Tensor object at ..., what does it imply?

Could you run your code with CUDA_LAUNCH_BLOCKING=1 python script.py args and post the stack trace here, please?

I tried and the results were the same as before

Could you then post the stack trace, please?

When I set pred to cpu the problem was solved. Thanks for the help anyway.