What accounts for getting different testing accuracy every code run?

1

Hello,

I am applying leave-one-out cross validation with CNN. My problem is that I keep getting different accuracy every time I run the code, although I reinitialize the network in the beginning of each fold by the same values and I do not shuffle the training set as shown in the following snippets.

    classname = m.__class__.__name__
    if classname.find('Conv2d') != -1:
        m.weight.data.fill_(0)
        m.bias.data.fill_(0)
    elif classname.find('BatchNorm2d') != -1:
        m.weight.data.fill_(0.0)
        m.bias.data.fill_(0)

and

    train_loader = DataLoader(dataset=train_dataset, batch_size=batch_size,shuffle=False)

Also, I want to know what makes the network performance stable. In fact, why they created initialization with random values if this might lead to different results.

Thanks in advance,

2

Since you model is initialized with random values, try to set the seed, so that the pseudorandom number generator generates the same “random” values for every run.

np.random.seed(SEED) # if numpy is used
torch.manual_seed(SEED)
if use_cuda:
    torch.cuda.manual_seed(SEED)

Also, some differences might occur when you are using CUDA, since some operations are not deterministic as fasr as I know.

2 Likes