When I inferred from the Inception_v3 model, I found on the profiler that kernel functions of volta_gcgemm_32, void fft1d_r2c_32, void fft1d_c2r_32 work in streams that I did not specify.
How is this possible?
These seem to be cudnn kernels, which are called internally in cudnn and might use different streams.
Thanks for replying.
But, I didn’t find this phenomenon in other kinds of models. So only the Inception_v3 model uses cudnnn?
Is there a way to make those kernels run only on the stream I specified?
It might be kernel-dependent, but if I’m right in my guess, I think you should also see it in more models.
Are you seeing the same kernel name in other models and if so, are these executed in a different way?
The cudnn kernel works on other models, but the kernel with the same name as the Inception does not work.
If your prediction is correct, what should I do to operate those kernels only on the specified stream?
What do you mean by “work”? Is the same kernel called in other models?
If so, how are you profiling it to determine the stream usage of this kernel?
The same kernel as Inception_v3 (ex. void fft1d_r2c_32) is not called from other models.
However, cudnn kernels with different names (ex. volta_scudnn_128x128_relu_interior_v1) are called.
And the kernels called from other models run only on the specified stream.
In both cases, the cudnn kernel are called, why does inception’s cudnn kernel run on unspecified streams but other models’s cudnn kernel run on only specified streams?