I.e. I have defined a generic DPMixin class to apply to various models I have. For some models, I want to privatise the model’s optimizer (model.optim) when model.train is called, for others, e.g. a GAN, I want to privatise model.discriminator.optim, in the latter case I can simply call self.discriminator = self.make_private(100, self.discriminator) and it works fine. I would like to also be able to call self.make_private(100) in the case where I want the top level “model” to be privatised. I have pasted my current attempt at this below, but it doesn’t seem to work for the latter case.
def make_private(self, num_epochs, module: Optional[nn.Module] = None) -> GradSampleModule:
"""
Make the passed module (or the full model if a module is not passed), and its associated optimizer and data loader private.
Args:
num_epochs: The number of epochs to train for, used to calculate the privacy budget.
module: The module to make private.
Returns:
The privatised module.
"""
module = module or self
self.privacy_engine = PrivacyEngine(secure_mode=self.secure_mode)
with warnings.catch_warnings():
warnings.filterwarnings("ignore", message="invalid value encountered in log")
warnings.filterwarnings("ignore", message="Optimal order is the largest alpha")
module, module.optim, self.data_loader = self.privacy_engine.make_private_with_epsilon(
module=module,
optimizer=module.optim,
data_loader=self.data_loader,
epochs=num_epochs,
target_epsilon=self.target_epsilon,
target_delta=self.target_delta,
max_grad_norm=self.max_grad_norm,
)
print(
"Using sigma={} and C={} to target (ε, δ) = ({}, {})-differential privacy.".format(
module.optim.noise_multiplier, self.max_grad_norm, self.target_epsilon, self.target_delta
)
)
self.get_epsilon = self.privacy_engine.accountant.get_epsilon
return module
In the latter case, I get this error when I go to calculate epsilon:
File "/Users/harrisonwilde/Library/Mobile Documents/com~apple~CloudDocs/PhD/NHS/NHSSynth/src/nhssynth/modules/model/models/model.py", line 234, in _generate_metric_str
val = self.get_epsilon(self.target_delta)
File "/Users/harrisonwilde/.venv/nhssynth/lib/python3.10/site-packages/opacus/accountants/prv.py", line 97, in get_epsilon
dprv = self._get_dprv(eps_error=eps_error, delta_error=delta_error)
File "/Users/harrisonwilde/.venv/nhssynth/lib/python3.10/site-packages/opacus/accountants/prv.py", line 114, in _get_dprv
domain = self._get_domain(
File "/Users/harrisonwilde/.venv/nhssynth/lib/python3.10/site-packages/opacus/accountants/prv.py", line 150, in _get_domain
return Domain.create_aligned(-L, L, mesh_size)
File "/Users/harrisonwilde/.venv/nhssynth/lib/python3.10/site-packages/opacus/accountants/analysis/prv/domain.py", line 31, in create_aligned
size = int(np.round((t_max - t_min) / dt)) + 1
ValueError: cannot convert float NaN to integer
Any help would be much appreciated!