# Opacus **URL:** https://discuss.pytorch.org/c/opacus/29.md?page=2 [Latest](https://discuss.pytorch.org/latest.md) · [Categories](https://discuss.pytorch.org/categories.md) **Page:** 3 --- ## [Opacus' Problem with Batch Norm vs TFP](https://discuss.pytorch.org/t/opacus-problem-with-batch-norm-vs-tfp/169172)
**Author:** [@ege\_b](https://discuss.pytorch.org/u/ege_b)\ **Replies:** 3\ **Last updated:** [January 30, 2023, 9:32am UTC](https://discuss.pytorch.org/t/opacus-problem-with-batch-norm-vs-tfp/169172 "2023-01-30T09:32:59Z")
Hi Opacus Team, I’ve been wandering around this topic for a while now and could not find a really pleasing answer: As far as I’ve understood, the privacy accounting mechanism of Opacus and TF Privacy are based on the s… --- ## [Optimizer step failing with opacus](https://discuss.pytorch.org/t/optimizer-step-failing-with-opacus/171167)
**Author:** [@hungryc8rpillar](https://discuss.pytorch.org/u/hungryc8rpillar)\ **Replies:** 2\ **Last updated:** [January 28, 2023, 8:59am UTC](https://discuss.pytorch.org/t/optimizer-step-failing-with-opacus/171167 "2023-01-28T08:59:27Z")
“Per sample gradient is not initialized. Not updated in backward pass?” Hi I get the above error message at optimizer.step() when training with opacus. I have already read other forum entries with the same or similar er… --- ## [Not sure if this layer is DP-friendly](https://discuss.pytorch.org/t/not-sure-if-this-layer-is-dp-friendly/169152)
**Author:** [@ege\_b](https://discuss.pytorch.org/u/ege_b)\ **Replies:** 1\ **Last updated:** [January 2, 2023, 2:31pm UTC](https://discuss.pytorch.org/t/not-sure-if-this-layer-is-dp-friendly/169152 "2023-01-02T14:31:41Z")
Hi, I do have an implementation of PG-GANs in hand and the discriminator of this model has a so-called Minibatch-STD layer. Opacus seems to validate the layer w/o any problems. However, what this layer actually does is t… --- ## [Text Classification tutorial without frozen layers](https://discuss.pytorch.org/t/text-classification-tutorial-without-frozen-layers/159167)
**Author:** [@long21wt](https://discuss.pytorch.org/u/long21wt)\ **Replies:** 5\ **Last updated:** [January 2, 2023, 2:27pm UTC](https://discuss.pytorch.org/t/text-classification-tutorial-without-frozen-layers/159167 "2023-01-02T14:27:08Z")
Hi, I tried the tutorial notebook on Text Classification. It works well. However, I don’t understand if I don’t freeze any layer, there will be a problem in training step. More specifically: /usr/local/lib/python3.7/di… --- ## [AttributeError in DataLoader using Opacus](https://discuss.pytorch.org/t/attributeerror-in-dataloader-using-opacus/167396)
**Author:** [@mmisiur](https://discuss.pytorch.org/u/mmisiur)\ **Replies:** 3\ **Last updated:** [December 7, 2022, 10:34am UTC](https://discuss.pytorch.org/t/attributeerror-in-dataloader-using-opacus/167396 "2022-12-07T10:34:58Z")
Hello, I was excited to learn about Opacus so now I’m trying to apply it to my training. I’m using DataLoader with my custom dataset MyDataset(Dataset). During calling dataloader to get data I’m getting an error: File … --- ## [No module named 'privacy\_lint' ! How can i fixe this error!](https://discuss.pytorch.org/t/no-module-named-privacy-lint-how-can-i-fixe-this-error/166137)
**Author:** [@Rafika\_Benledghem](https://discuss.pytorch.org/u/Rafika_Benledghem)\ **Replies:** 3\ **Last updated:** [November 24, 2022, 12:15am UTC](https://discuss.pytorch.org/t/no-module-named-privacy-lint-how-can-i-fixe-this-error/166137 "2022-11-24T00:15:33Z")
I try to run this code from github examples cifar10\_attack.py but it doesn’t work !!! --- ## [How to implement other DP methods?](https://discuss.pytorch.org/t/how-to-implement-other-dp-methods/165017)
**Author:** [@Frankozay](https://discuss.pytorch.org/u/Frankozay)\ **Replies:** 2\ **Last updated:** [November 8, 2022, 11:11am UTC](https://discuss.pytorch.org/t/how-to-implement-other-dp-methods/165017 "2022-11-08T11:11:33Z")
Hello everyone, I’m a beginner in differential privacy, I think that use Opacus is easy to implement DP-SGD, my question may be silly but I wonder if there is a way to use DP independently? I don’t know if I’ve express… --- ## [High CPU usage normal?](https://discuss.pytorch.org/t/high-cpu-usage-normal/162774)
**Author:** [@FMorsbach](https://discuss.pytorch.org/u/FMorsbach)\ **Replies:** 3\ **Last updated:** [November 8, 2022, 9:30am UTC](https://discuss.pytorch.org/t/high-cpu-usage-normal/162774 "2022-11-08T09:30:28Z")
Hi there, I have a question regarding the CPU usage when training with Opacus. I noticed that when training with Opacus the CPU usage explodes compared to non-private training. I am wondering whether this is expected or… --- ## [DPOptimizer: filter parameters to add noise to](https://discuss.pytorch.org/t/dpoptimizer-filter-parameters-to-add-noise-to/164027)
**Author:** [@Gianmarco\_Aversano](https://discuss.pytorch.org/u/Gianmarco_Aversano)\ **Replies:** 5\ **Last updated:** [October 27, 2022, 2:53pm UTC](https://discuss.pytorch.org/t/dpoptimizer-filter-parameters-to-add-noise-to/164027 "2022-10-27T14:53:50Z")
Hello I modified Opacus source code to create a modified version of DPOptimizer that will add noise only to some specified parameter groups of the the underlying optimizer. I did this because I was having countless erro… --- ## [Opacus framework for Multilabel multiclass classification problem](https://discuss.pytorch.org/t/opacus-framework-for-multilabel-multiclass-classification-problem/164118)
**Author:** [@sabeekakhan7](https://discuss.pytorch.org/u/sabeekakhan7)\ **Replies:** 1\ **Last updated:** [October 24, 2022, 11:52am UTC](https://discuss.pytorch.org/t/opacus-framework-for-multilabel-multiclass-classification-problem/164118 "2022-10-24T11:52:45Z")
Hello, I am stuck with something I was hoping to get some help with. I am trying to train a differentially private multilabel multiclass model on the NIH Chest X-Ray dataset using the Opacus framework. I have tried diff… --- ## [Error when trying Federated Learning with Opacus](https://discuss.pytorch.org/t/error-when-trying-federated-learning-with-opacus/153049)
**Author:** [@general](https://discuss.pytorch.org/u/general)\ **Replies:** 4\ **Last updated:** [October 11, 2022, 2:57pm UTC](https://discuss.pytorch.org/t/error-when-trying-federated-learning-with-opacus/153049 "2022-10-11T14:57:59Z")
Hi, I have a simple federated learning setup, where I train a Densenet-121 on subsets of the CIFAR-10 dataset. By setting a variable to False or True, I can control whether it does not use Opacus/Differential privacy or… --- ## [What the warning is?](https://discuss.pytorch.org/t/what-the-warning-is/163268)
**Author:** [@JeffffFu](https://discuss.pytorch.org/u/JeffffFu)\ **Replies:** 2\ **Last updated:** [October 11, 2022, 9:44am UTC](https://discuss.pytorch.org/t/what-the-warning-is/163268 "2022-10-11T09:44:50Z")
10/11/2022 17:28:42 - Despite set\_to\_none is set to False, opacus will set p.grad\_sample and p.summed\_grad to None due to non-trivial gradient accumulation behaviour I ran almost the same two pieces of code, why does t… --- ## [Layers whose .grad exists but grad\_sample is None](https://discuss.pytorch.org/t/layers-whose-grad-exists-but-grad-sample-is-none/163100)
**Author:** [@Anirban\_Nath](https://discuss.pytorch.org/u/Anirban_Nath)\ **Replies:** 1\ **Last updated:** [October 10, 2022, 8:49am UTC](https://discuss.pytorch.org/t/layers-whose-grad-exists-but-grad-sample-is-none/163100 "2022-10-10T08:49:59Z")
I have a few layers in my model whose .grad exists but .grad\_sample is None. How is .grad\_sample different from .grad and how can one be None and the other not? --- ## [Some linear layers have a valid grad\_sample, others don't](https://discuss.pytorch.org/t/some-linear-layers-have-a-valid-grad-sample-others-dont/162212)
**Author:** [@Anirban\_Nath](https://discuss.pytorch.org/u/Anirban_Nath)\ **Replies:** 4\ **Last updated:** [October 6, 2022, 6:02pm UTC](https://discuss.pytorch.org/t/some-linear-layers-have-a-valid-grad-sample-others-dont/162212 "2022-10-06T18:02:52Z")
I have two linear layers defined in the model as such: - self.feature\_reduction = nn.Linear(in\_features = 512, out\_features = 256) self.class\_embed = nn.Linear(in\_features = self.hidden\_dim, out\_features = 2) They both… --- ## [Arbitrary batch sizes after passing dataloader through privacy engine](https://discuss.pytorch.org/t/arbitrary-batch-sizes-after-passing-dataloader-through-privacy-engine/161632)
**Author:** [@Anirban\_Nath](https://discuss.pytorch.org/u/Anirban_Nath)\ **Replies:** 4\ **Last updated:** [October 3, 2022, 7:10am UTC](https://discuss.pytorch.org/t/arbitrary-batch-sizes-after-passing-dataloader-through-privacy-engine/161632 "2022-10-03T07:10:55Z")
In my code, I have defined privacy engine as follows: - unet, optimizer, trainloader = privacy\_engine.make\_private\_with\_epsilon( module = unet, data\_loader = trainloader, optimizer = optimizer, … --- ## [Why the DPRNN and DPGRU layers are not registered in the ModuleValidator](https://discuss.pytorch.org/t/why-the-dprnn-and-dpgru-layers-are-not-registered-in-the-modulevalidator/160138)
**Author:** [@xichens](https://discuss.pytorch.org/u/xichens)\ **Replies:** 1\ **Last updated:** [August 30, 2022, 12:40pm UTC](https://discuss.pytorch.org/t/why-the-dprnn-and-dpgru-layers-are-not-registered-in-the-modulevalidator/160138 "2022-08-30T12:40:38Z")
I see the drop-in replacements DPRNN, DPGRU for the corresponding PyTorch modules are defined in the same file as the DPLSTM, but currently only the DPLSTM is registered in the ModuleValidator, not the other two. So Modu… --- ## [What is the Privacy Accounting usage?](https://discuss.pytorch.org/t/what-is-the-privacy-accounting-usage/159018)
**Author:** [@Leonmac](https://discuss.pytorch.org/u/Leonmac)\ **Replies:** 5\ **Last updated:** [August 26, 2022, 4:51pm UTC](https://discuss.pytorch.org/t/what-is-the-privacy-accounting-usage/159018 "2022-08-26T16:51:02Z")
Hello Opacus team, Can someone explain the usage of the " Privacy Accounting" and, is there any application documentation, or code example regarding it? --- ## [The Opacus example, train batch size vs sampling rate](https://discuss.pytorch.org/t/the-opacus-example-train-batch-size-vs-sampling-rate/158771)
**Author:** [@Leonmac](https://discuss.pytorch.org/u/Leonmac)\ **Replies:** 4\ **Last updated:** [August 26, 2022, 2:27pm UTC](https://discuss.pytorch.org/t/the-opacus-example-train-batch-size-vs-sampling-rate/158771 "2022-08-26T14:27:40Z")
Hi opacus team: I am doing a test by using this example project: One specific thing I noticed is the train\_batch\_size is defined as here: batch\_size=int(args.sample\_rate \* len(train\_dataset)) Therefore a train datalo… --- ## [Parameter selection recommendations for the DPSGD algorithm](https://discuss.pytorch.org/t/parameter-selection-recommendations-for-the-dpsgd-algorithm/159529)
**Author:** [@JeffffFu](https://discuss.pytorch.org/u/JeffffFu)\ **Replies:** 1\ **Last updated:** [August 24, 2022, 4:54pm UTC](https://discuss.pytorch.org/t/parameter-selection-recommendations-for-the-dpsgd-algorithm/159529 "2022-08-24T16:54:15Z")
Any DPSGD algorithm in FashionMnist and cifar10 parameter selection suggestions? Such as sigma, C, learning rete. --- ## [Restoring the original model](https://discuss.pytorch.org/t/restoring-the-original-model/155484)
**Author:** [@luc12](https://discuss.pytorch.org/u/luc12)\ **Replies:** 2\ **Last updated:** [August 24, 2022, 2:04pm UTC](https://discuss.pytorch.org/t/restoring-the-original-model/155484 "2022-08-24T14:04:55Z")
Hi everyone, I’m using Opacus and I have a very specific question Basically, I want to alternate the training using differential privacy with the training without DP: I wrap the model with make\_private and I train th… --- ## [The samping of RDP](https://discuss.pytorch.org/t/the-samping-of-rdp/158322)
**Author:** [@JeffffFu](https://discuss.pytorch.org/u/JeffffFu)\ **Replies:** 7\ **Last updated:** [August 24, 2022, 1:46am UTC](https://discuss.pytorch.org/t/the-samping-of-rdp/158322 "2022-08-24T01:46:25Z")
1、the uniform samping with Replacement ，uniform samping with Replacement and the Poisson samping . What is the relationship between the three of them? 2、the paper《RDP of the sampled Gussian Mechanism》 use Poisson sampi… --- ## [Virtual batch sizing with multi GPU training](https://discuss.pytorch.org/t/virtual-batch-sizing-with-multi-gpu-training/158614)
**Author:** [@timudk](https://discuss.pytorch.org/u/timudk)\ **Replies:** 1\ **Last updated:** [August 9, 2022, 9:32pm UTC](https://discuss.pytorch.org/t/virtual-batch-sizing-with-multi-gpu-training/158614 "2022-08-09T21:32:00Z")
Hi there, I was wondering if virtual batch sizing (as explained, for example, in Opacus · Train PyTorch models with Differential Privacy) is supposed to work with multi-GPU training? For me, virtual batch sizing work we… --- ## [AttributeError: 'GradSampleModule' object has no attribute for method](https://discuss.pytorch.org/t/attributeerror-gradsamplemodule-object-has-no-attribute-for-method/157135)
**Author:** [@Leonmac](https://discuss.pytorch.org/u/Leonmac)\ **Replies:** 10\ **Last updated:** [August 4, 2022, 3:14am UTC](https://discuss.pytorch.org/t/attributeerror-gradsamplemodule-object-has-no-attribute-for-method/157135 "2022-08-04T03:14:24Z")
Hello, I am using the flower as an FL framework and I am trying to put DP support by using Opacus. Here is the problem I meet: I am using a very common MNIST model and inherited to get a new class: # https://github.… --- ## [Convert\_batchnorm\_modules does not exist](https://discuss.pytorch.org/t/convert-batchnorm-modules-does-not-exist/157156)
**Author:** [@Zebang\_Shen](https://discuss.pytorch.org/u/Zebang_Shen)\ **Replies:** 2\ **Last updated:** [July 21, 2022, 11:53pm UTC](https://discuss.pytorch.org/t/convert-batchnorm-modules-does-not-exist/157156 "2022-07-21T23:53:19Z")
I am trying to use Opacus to implement DP-SGD, but I cannot find the function “convert\_batchnorm\_modules” anywhere in the package. Cannot find answer from google results neither. Can someone please help me with this? T… --- ## [I want to know about the opacus version, why there is a problem with using the official example, For example the link https://blog.openmined.org/differentially-private-deep-learning-using-opacus-in-20-lines-of-code/ ](https://discuss.pytorch.org/t/i-want-to-know-about-the-opacus-version-why-there-is-a-problem-with-using-the-official-example-for-example-the-link-https-blog-openmined-org-differentially-private-deep-learning-using-opacus-in-20-lines-of-code/156936)
**Author:** [@jiuyu\_yang](https://discuss.pytorch.org/u/jiuyu_yang)\ **Replies:** 3\ **Last updated:** [July 21, 2022, 1:29am UTC](https://discuss.pytorch.org/t/i-want-to-know-about-the-opacus-version-why-there-is-a-problem-with-using-the-official-example-for-example-the-link-https-blog-openmined-org-differentially-private-deep-learning-using-opacus-in-20-lines-of-code/156936 "2022-07-21T01:29:25Z")
Let’s say this is an example of a runtime errorUploading: 1.png…I would like to know, what is the correct version of Opacus, or is it for other reasons, thank you --- ## [Using nn.Parameter in Opacus](https://discuss.pytorch.org/t/using-nn-parameter-in-opacus/126569)
**Author:** [@ChrisWaites](https://discuss.pytorch.org/u/ChrisWaites)\ **Replies:** 3\ **Last updated:** [July 6, 2022, 9:04pm UTC](https://discuss.pytorch.org/t/using-nn-parameter-in-opacus/126569 "2022-07-06T21:04:41Z")
Hey there, we are trying to train a private version of a particular model which uses nn.Parameter and getting the error torchdp.dp\_model\_inspector.IncompatibleModuleException. In particular, the parameters are defined f… --- ## [ModuleValidator.fix does not immediately change the model - how to deal with it?](https://discuss.pytorch.org/t/modulevalidator-fix-does-not-immediately-change-the-model-how-to-deal-with-it/152783)
**Author:** [@general](https://discuss.pytorch.org/u/general)\ **Replies:** 6\ **Last updated:** [May 31, 2022, 8:32pm UTC](https://discuss.pytorch.org/t/modulevalidator-fix-does-not-immediately-change-the-model-how-to-deal-with-it/152783 "2022-05-31T20:32:27Z")
Hi, I would like to use opacus with a DenseNet121. Since this neural network is by default not compatible with opacus, I need to use ModuleValidator.fix(model) before training the model. Using the debugger, I noticed t… --- ## [Gradients before clip are much lager than the clip bound](https://discuss.pytorch.org/t/gradients-before-clip-are-much-lager-than-the-clip-bound/152253)
**Author:** [@Zark](https://discuss.pytorch.org/u/Zark)\ **Replies:** 5\ **Last updated:** [May 26, 2022, 10:35am UTC](https://discuss.pytorch.org/t/gradients-before-clip-are-much-lager-than-the-clip-bound/152253 "2022-05-26T10:35:33Z")
Hi, I’m using Opacus to implement DPSGD in my program. One thing I noticed is that the gradients to be clipped are much larger than the max\_grad\_norm, as the screenshot from the console output. The code is from the opa… --- ## [About dcgan.py results](https://discuss.pytorch.org/t/about-dcgan-py-results/144824)
**Author:** [@chensquan](https://discuss.pytorch.org/u/chensquan)\ **Replies:** 2\ **Last updated:** [May 26, 2022, 9:16am UTC](https://discuss.pytorch.org/t/about-dcgan-py-results/144824 "2022-05-26T09:16:15Z")
Hi！ When I run the exmaple of the dcgan.py. the results is so poor.But when I set the parameter sigma=0.01, the generator image could be recognized，but the epsilon is too large to achieve the desire effect. However， when… --- ## ['GradSampleModule' object has no attribute](https://discuss.pytorch.org/t/gradsamplemodule-object-has-no-attribute/151318)
**Author:** [@donarni](https://discuss.pytorch.org/u/donarni)\ **Replies:** 2\ **Last updated:** [May 10, 2022, 4:32pm UTC](https://discuss.pytorch.org/t/gradsamplemodule-object-has-no-attribute/151318 "2022-05-10T16:32:58Z")
Hello Opacus Team. After wrapping a nn.Module with GradSampleModule, how can I access the methods (e.g., def init\_weights (self) :) of the model? Specifically, I want to use Opacus without PrivacyEngine to privatize th… [Previous page](https://discuss.pytorch.org/c/opacus/29.md?page=1) [Next page](https://discuss.pytorch.org/c/opacus/29.md?page=3)