How to initialize a CNN with weights from flat numpy array

1

Hi

Does anybody have a way to go back from flattening all the weights and putting them into a numpy vector (as done in the code snippet below). I trained the network for 3 days, before my allotted time expired, causing the program to cancel before converging. I have the checkpoints saved as numpy vectors. The model architecture is Resnet18 (for CIFAR10).

gg = np.array()
for name, weight in network.named_parameters():
    gg.extend(weight.cpu().detach().numpy().flatten())
np.save("name" + str(q)+str(n), gg)

I suppose it is possible to manually split the numpy vector and convert it into a tensor, but any help is appreciated.

2

Yes, manually unwrapping the data should work. Something like this might help:

# flatten
gg = []
for name, weight in network.named_parameters():
    gg.extend(weight.cpu().detach().numpy().flatten())
    
# restore
sd = {}
i = 0
for name, weight in network.named_parameters():
    param_len = weight.nelement()
    param = torch.tensor(gg[i:i+param_len])
    param = param.view_as(weight)
    sd[name] = param
    i += param_len
    
for (name1), (name2, param2) in zip(sd, network.named_parameters()):
    param1 = sd[name1]
    print('name {} equal: {}'.format(name1, name1==name2))
    print('param equal {}'.format((param1 == param2).all()))
1 Like