Weight/neuron based learning rate

[mxahan (Zahid Hasan)]

@razor

am trying to have individual learning rates for each neuron, that is, the learning rate becomes a matrix. How would anyone implement that for linear layers only?
Thanks!

You can do that by defining different optimizers for each of the neurons. This link may help you.