Is there any literature/documentation on the necessity of cryptographically-secure RNG in Opacus? As far as I can tell it’s only used for noise generation so would like to understand the underlying risks/compromises. It looks to me like TF-Privacy isn’t using a CSPRNG for its noise generation so am curious about the reasoning for the divergence.
Excellent question! There are two separate uses of CSPRNG in Opacus - for batch composition (shuffling or sampling) and for noise generation. If batches are formed using an insecure RNG, the algorithm is still differentially private, but we don’t get to claim amplification by subsampling.
It looks to me like TF-Privacy isn’t using a CSPRNG for its noise generation so am curious about the reasoning for the divergence.
It’s a pragmatic choice. Basically, TF does not natively support CSPRNG; Pytorch’s decision to implement it is in no small part due to its commitment to supporting Opacus.
I’ll add parenthetically, that, strictly speaking, DP is an information-theoretic notion. As such, it cannot be satisfied with “just” cryptographically secure PRNG. For purists (or complexity theorists) out there, Opacus meets the definition of Computational Differential Privacy. In practice, all randomness available on computers is computational, and I don’t believe drawing the distinction is productive at this point of technology development.