Is there a way to extract weight update matrix using PyTorch api?

1

During training we are update weights inside the model.
Say we take conv layers.
How do I access the weight update matrix?

Is there an api for this sort of thing or do I need to save weights of interest myself and then find the difference of those?

Thank you

1 Like
2

Hi,

If you’re using the builtin optimizers from torch.optim, then not really. Because each optimizer is free to make the update as it sees fit. And some of them (life LBFGS) actually does multiple updates.

If you’re doing the update yourself by hand, then I guess you can do this by making your update code return what you want.

2 Likes
3

I see, yeah I was asking in scope of torch optimizers. Thank you