Cannot get the value of hidden weights of RNN

I declare my RNN as

self.rnn = torch.nn.RNN(input_size=encoding_dim, hidden_size=1, num_layers=1, nonlinearity='relu')


# [[Parameter containing:
tensor([[-0.8099, -0.9543,  0.1117,  0.6221,  0.5034, -0.6766, -0.3360, -0.1700,
         -0.9361, -0.3428]], requires_grad=True), Parameter containing:
tensor([[-0.1929]], requires_grad=True), Parameter containing:
tensor([0.7881], requires_grad=True), Parameter containing:
tensor([0.4320], requires_grad=True)]]

# {RuntimeError}Could not run 'aten::values' with arguments from the 'CPU' backend. 'aten::values' is only available for these backends: [SparseCPU, Autograd, Profiler, Tracer].

Clearly I see the value of the weights, but cannot access to it. Documentation says I need to specify requires_grad=True, but that does not work.

Is there a more elegant and usable way than self.rnn.all_weights[0][0][0]?