I was wondering if it is safe to save the HF transformer models using private method model._module.save_pretrained() to save the model and load the model as it is usually done using the HF library since the wrapper GradSampleModule() around the model doesn’t have any method like save_pretrained
model, optimizer, dataloader = privacy_engine.make_private( ... module=model, ... optimizer=optimizer, ... data_loader=dataloader, ... noise_multiplier=1.0, ... max_grad_norm=1.0, ... ) —Training— model._module.save_pretrained(Path)
and loading the model for inference using
from transformers import xyz model = xyz.from_pretrained(Path)
I’m not sure if it’s safe becasue I’m getting very good accuracy and privacy with the model saved and loaded using the above snippet.