Hello all,

I am pretty new to PyTorch, and am trying to implement a prediction code starting with just an Elman Network (RNN module with 1 layer)

With the code I have written (given below), my loss function (MSE) does not decrease and keeps giving random values for all epochs.

x is the input, which is a pattern in the form of 2-D co-ordinates, and y is the prediction (output) I would like to make.

#Just parsing an input file

xa = pd.read_csv(‘two_circ.csv’)

xb = np.zeros((len(xa),2))

for i in range(len(xa)):

xb[i] = xa.values[i] #Input layer

x = torch.from_numpy(xb)[:,None,:].float()

#Build output matrix for prediction

ya = np.roll(xb,1,axis=0)

ya[0] = ya[1] #Output layer

y = torch.from_numpy(ya)[:,None,:].float()

n_inputs, n_neurons, n_outputs, batch_size, n_epochs = 2, 5, 2, len(xb), 300

rnn = nn.RNN(n_inputs, n_neurons)

loss = nn.MSELoss() #Selected loss function

optimizer = optim.Adam(rnn.parameters(), lr = 1e-1)

for i in range(n_epochs):

h0 = torch.randn(1, 1, n_neurons)

```
xnew = x[i][:,None,:]
hidden, h0 = rnn(xnew, h0)
hid = torch.squeeze(hidden,1) #Remove one (the second) dimension
o = nn.Linear(n_neurons,n_outputs) #Apply linear function to the hidden layer
out = o(hid) #Gets a tensor with 2 outputs
a = nn.Tanh()
output = a(out)
error = loss(output, y[i])
optimizer.zero_grad()
error.backward(retain_graph=True)
optimizer.step()
print('Loss for Epoch '+str(i)+' is '+str(error))
```

I am specifically confused regarding the whole deal with Variable with the autograd function(which if I understand correctly, I don’t think is needed with the RNN module).

Thank you in advance