New version opacus without dataloader

Opacus
1

Hi!

I’m using Opacus to train my model. The version before 1.0.0 works fine without dataloader. However, it seems that the new version privacy engine requires one at initialization. Is there any way to avoid this?

Thx!

2

Hi!
Thanks for using opacus.

There’s certainly a way to use opacus without DataLoader - please refer to our Migration Guide, section “No DataLoader”

Feel free to ask any further questions here

1 Like
3

thx for reply!
However, as I’m following the guide, something seems to get wrong:

optimizer.attach_step_hook(…

However, there’s an error claim that SGD has no such attribute. I use SGD from torch.optim.SGD. Any idea about the situation?

4

Whoops, thanks for pointing this out - that’s actually a mistake on migration guide. You need to do

dp_optimizer.attach_step_hook(...

See PR #307

5

Thx, the guide works fine for the problem. However, there’s a new one. I attached the privacyengine to the optim, but when I ran the code, it reported a CUDA out of memory.

RuntimeError: CUDA out of memory. Tried to allocate 6.10 GiB (GPU 0; 23.70 GiB total capacity; 13.27 GiB already allocated; 3.20 GiB free; 18.51 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

So I then check the my graphics using nvidia-smi. I got 2 rtx3090 on the server, both available with 24gb memory. Then I ran the code with older opacus, basically the same but the way to attach privacyengine. The older one works.

Trace back reported something about backward() having trouble. Any idea?

6

One potential reason this could be happening is poisson sampling. If you’re using default setup with Opacus 1.0, then your dataloader is using a non-standard sampler (UniformWithReplacementSampler ). (it was available before, but had to be manually enabled).

With this sampler batches are of variable size: the average size is still the same, but some batches are larger than average. This leads to an increased memory requirement, as your memory is limited by the peak batch size, not average.
We have a utility to handle this: BatchMemoryManager (see example here)

If it’s not that, then it’s interesting and we’ll need to double check and see what could’ve caused it.