1、the uniform samping with Replacement ，uniform samping with Replacement and the Poisson samping . What is the relationship between the three of them?

2、the paper《RDP of the sampled Gussian Mechanism》 use Poisson samping?

3、Opacus provide uniform samping with Replacement and Poisson samping methods, do they both correspond to the same privacy calculation formula？

1、the uniform samping with Replacement ，uniform samping without Replacement and the Poisson samping . What is the relationship between the three of them?

Hi,

Re 1:

We say Uniform sampling to mean Poisson sampling: Each sample is selected with a probability equal to `sample_rate`

The with replacement is due to the fact that during a batch construction, we consider the whole dataset (all the samples), and sample each data point with the probability `sample_rate`

Re 2: Yes

Re 3: Yes, because the two are one and the same.

Thank you for your answer, I have two more questions as follows.

1, the uniform sampling with replacement , the uniform sampling without replacement and the Poisson sampling are differential. In uniform sampling, sample rate is the number of samples sampled In uniform sampling, the sample rate is the ratio of the number of samples sampled to the total number of samples, so the number of samples sampled is fixed and the same for each sample. In poisson samping, the sample rate is the probability of each sample being sampled, which means that the number of samples sampled is not necessarily the same each time. Am I understanding this correctly?

2. In GDP privacy calculation, uniform sampling and poisson sampling have different privacy budgets for the same parameters, why are the privacy losses calculated the same for both in RDP?

3、what is the differential of UniformWithReplacementSampler and DistributedUniformWithReplacementSampler?

there are somebody can answer my question? thanks

Hey JeffffFu,

Thanks for your interest! To answer your questions:

- That’s correct, with Poisson sampling your batch size may vary (hence we only set the
*expected*batch size). - Poisson sampling is necessary to obtain privacy budget computations under the paper you mentioned.
- The
`UniformWithReplacementSampler`

is to be used when you train on a single process (generally a single GPU), the`DistributedUniformWithReplacementSampler`

is when you train in a distributed way, generally on multiple GPUs.

Hope this helps,

Pierre

thank you very much!