by this i mean, after each update of the weights, i’d like to store the values in a dictionary or list. this is for a particular layer and not for all the network
Do you want to overwrite the old values after each update?
Would this work?
model = nn.Sequential(
nn.Linear(20, 10),
nn.ReLU(),
nn.Linear(10, 2)
)
optimizer = optim.SGD(model.parameters(), lr=1e-3)
x = torch.randn(1, 20)
# Store just the first layer
weights = dict(model[0].named_parameters())
for epoch in range(10):
output = model(x)
output.mean().backward()
optimizer.step()
# Update weights
weights = dict(model[0].named_parameters())
no id actually like to store all the updates, in a dictionary so that i can see how the weights change as training proceeds
Then you could try the following:
model = nn.Sequential(
nn.Linear(20, 10),
nn.ReLU(),
nn.Linear(10, 2)
)
optimizer = optim.SGD(model.parameters(), lr=1e-3)
x = torch.randn(1, 20)
# Store just the first layer
weights = {k: v.clone() for k, v in model[0].named_parameters()}
updates = {k: v.clone() for k, v in model[0].named_parameters()}
for epoch in range(10):
optimizer.zero_grad()
output = model(x)
output.mean().backward()
optimizer.step()
new_weights = {k: v.clone() for k, v in model[0].named_parameters()}
updates = {k: new_weights[k] - weights[k] for k in weights}
weights = new_weights
print(updates)
1 Like
thank you so much! that should do it
just for future reference, and for whoever else looks over this question. I decided to use a pickel file to store the weights as the network was being trained. It was simpler, since i needed to load the weights in another file that I was using. but thank you very much for your help