Query on ExpandedWeights

Opacus
1

Hi,

I saw that Opacus supports ExpandedWeights that can potentially improve the latency of per-sample gradient computation over the GradSampleModule (“hooks” approach) (ref).

The README file mentions that there may be unexpected errors when using ExpandedWeights and that it is in beta.

What is the current state of ExpandedWeights, is it out of beta and in stable mode now? If not, is there a timeline for it? Additionally, I saw that it is not supported for RNN, LSTMs and a few other ops, is there a plan to enable ExpandedWeights over these?

Thanks,
Pooja

2

Hi,
We are still in the beta mode. Right now our priority is on other modes (“hook”, and “functorch”) and there is no recent plan to support new functions for EW.
Thanks,
Huanyu