Problems using DataParallel

Seungjun99 · May 9, 2024, 7:25am

Hi, I just started studying pytorch recently, and want to train a model with multi GPUs, so tried an example using DataParallel.

But after the model is assigned to GPUs, the training does not proceed.

I have 8 GTX 1080Ti GPUs.

This is the example I tried.
Example

github.com

chi0tzp/pytorch-dataparallel-example/blob/master/main.py

import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable
from torch.utils.data import Dataset, DataLoader


class RandomDataset(Dataset):
    def __init__(self, size, length):
        self.len = length
        self.data = torch.randn(length, size)

    def __getitem__(self, index):
        return self.data[index]

    def __len__(self):
        return self.len


class Model(nn.Module):

This file has been truncated. show original

I got Let’s use 8 GPUs! message but no more output.

python test.py
Let's use 8 GPUs!

This is my torch version.

pip show torch
Name: torch
Version: 2.2.2+cu118

Is there anything else I need to configure to use DataParallel?