I’m training a deep learning model that need more than 8GB of GPU RAM. And I wonder can that trained model be used to predict on 2GB of RAM GPU like GTX 1030, so I can run test while training?
You could try to perform the inference on your second GPU and see, if if has enough memory:
device = 'cuda:1' # assuming your GTX1030 is cuda:1 model = model.to(device) with torch.no_grad(): for data, target in val_loader: data = data.to(device) target = target.to(device) output = model(data) ...
If you want to run the prediction simultaneously while training, you could save the current model checkpoints, load them in the prediction script and run the inference part.