How to use make_private() with a model that registers a buffer?

Specifically, I’m trying to use Opacus with the the transformers implementation of the LayoutLMv3 model. When run a forward pass, I receive the following error message:

NotImplementedError: [NotImplementedError("Model contains a trainable layer that Opacus doesn't currently support(model:LayoutLMv3Model....

Upon looking at the source of this error, it comes from this check:

Indeed, the LayoutLMv3 model does register a buffer:

What is the recommended way to use Opacus in this situation?

I have not looked at your model clearly. But if you are sure that the buffer you have will not lead to a privacy leakage (unlike batch normalization), feel free to just comment it out in the code.