Support for persistent rnn kernels

1

Is there a plan to support persistent RNN kernels:

http://proceedings.mlr.press/v48/diamos16.pdf
https://svail.github.io/persistent_rnns/

It seems that CuDNN already supports that.

Thanks,

Yusuf

1 Like
2

Hi @yusuf_isik,

I am also looking for the same.

Do you know if any resource where CuDNN Persistent RNN implementation can be obtained from. Or could you point to file where persistent RNN is implemented in CuDNN. Thanks

3

You can login to Nvidia Developer, and look at the developer guide for Cudnn:

https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html

All the functions for recurrent networks in cudnn requires a cudnnRNNDescriptor_t instance as input. When you initialize an RNN descriptor object with cudnnSetRNNDescriptor() function, you define the algorithm with cudnnRNNAlgo_t value, which can be CUDNN_RNN_ALGO_STANDARD, CUDNN_RNN_ALGO_PERSIST_STATIC, or CUDNN_RNN_ALGO_PERSIST_DYNAMIC.

You can refer to the cudnn developer guide for information.

1 Like