Dear all,
I am applying OPACUS according to related tutorial:
self.privacy_engine = PrivacyEngine()
self.net, self.optimizer, self.train_loader = self.privacy_engine.make_private( module=self.net, optimizer=self.optimizer, data_loader=self.trainloader, max_grad_norm=1., noise_multiplier=60,)
as you can see the noise mult is quite large and the clipping norm “reasonable” for an entry test,
I checked epsilon and I got quite a low value as expected, The optimizer is SGD.
What I find unexpected is that no drop occurs for some datasets I am applying OPACUS to.
Is that possible and if it is the case, how can I manage it or there is something wrong I cannot see.
Thanks a lot for any feedback you will provide!