How to set different learning rates for different part of the same tensor?

4

No, you could also use different (sub-)tensors and stack/concatenate them in the actual forward pass as explained e.g. here.

1 Like