Hi,

How would I calculate the Fisher information matrix for a single layer in the network i.e just one nn.Linear

Thanks for your help

Hi,

How would I calculate the Fisher information matrix for a single layer in the network i.e just one nn.Linear

Thanks for your help

Please take a look at the wiseodd/natural-gradients repository. It has a couple of examples including a 2-layer NN which you can simplify to a single-layer NN.