https://pytorch.org/docs/stable/_modules/torch/nn/modules/rnn.html
It says that “Second bias vector is included for CuDNN compatibility. Only one bias vector is needed in standard definition.”
https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html#cudnnRNNMode_t