In the case of multiple networks, when torch.backends.cudnn.benchmark = True, will the benchmark be repeated?

1

When I have two networks that need to be trained alternately, for example, GAN network as a typical case (generator A, discriminator B).
I train A, B alternately, for example, A->B->A->B…. At this point, will the benchmark be run every time, or only the first time A is executed and the first time B is executed?

In fact, I’d rather know what conditions would trigger the benchmark, or re-benchmark.
Thanks!

2

The benchmark will be executed once per workload (input size, data layout, dtype, compute capability, etc.) and the selected algorithm will be cached.

3

Thank you very much for your answer, I understand.
The benchmark is independent of other networks.