Cannot make LSTM deterministic

The LSTM documentation suggests that it’s possible to make the LSTM deterministic if the right environment variable is set. I found that not to be the case. Minimal reproduction is included below. If this is intended behavior then the documentation should be updated so as not to mislead users.

lstm_test.py:

import torch
import torch.nn as nn

torch.use_deterministic_algorithms(True)
seed = 1
model= nn.LSTM(4, 4, 2, batch_first = True, dropout=0.5, bidirectional=True).cuda()

for i in range(3):
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)
    model.train()
    data = torch.randn(4, 4)
    if i > 0:
        print(torch.equal(data, pre_data))
    pre_data = data
    data = data.cuda()
    out, _ = model(data)
    loss = out.sum()
    print(i, loss.item())

Calling in bash results in the following:

> CUBLAS_WORKSPACE_CONFIG=:4096:2 python lstm_test.py
0 -3.0934510231018066
True
1 -2.6078720092773438
True
2 -2.4402637481689453

My env:
Ubuntu 18.04.6 LTS
torch.version = 1.12.1
torch.version.cuda = 11.3
sys.version = 3.9.13