Set random seed for a given network

Hello! I tried for a while to initialize my network using a fixed random seed, but it doesn’t seem to work (if I run the code multiple times and print the parameters, they are different every time). Here is what I have so far:

torch.cuda.manual_seed_all(0)
class Encoder(nn.Module):
    def __init__(self):
        super().__init__()

        N = 32
        self.encoder = nn.Sequential(
            nn.Conv2d(3, N, 4, 2, 1),
            nn.ReLU(True),
            nn.Conv2d(N, N, 4, 2, 1),
            nn.ReLU(True),
            nn.Conv2d(N, 2*N, 4, 2, 1),
            nn.ReLU(True),
            nn.Conv2d(2*N, 2*N, 4, 2, 1),
            nn.ReLU(True),
            nn.Conv2d(2*N, 8*N, 4, 1),
            nn.ReLU(True),
            View((-1, 8*N*1*1)),
            nn.Linear(8*N, 4),
        )

    def forward(self, x):
        z = self.encoder(x)
        return z

model_E = Encoder().cuda()

I tired to place torch.cuda.manual_seed_all(0) at different places in my code, but it seems like no matter where I put it, it doesn’t make a difference. How should I do it the right way? Thank you!

Add this at the starting of the first file you call (eg: main.py)

SEED = 1234

random.seed(SEED)
np.random.seed(SEED)
torch.manual_seed(SEED)
torch.backends.cudnn.deterministic = True