Hi everyone, I need some suggestion to solve my problem. I have a signal dataset of shape (ns, nl, nc). where ns is the number of samples, nl is the length of the signal and nc is the number of channels. eg:(200, 5000, 8). I’m trying to understand how to train an rnn on this datatset. Assuming a many to one rnn, How do I train the data. Should I just loop through epochs and loop through each time step of signal (size = 1, 8) and train the model? Any suggestions or Ideas are appreciated. Also, It does not have to be an many to one RNN. I’m just confused as to how to train. Thanks in advance.
You can do it either way. But I think passing in all of the time steps at once and letting c++ handle it is the optimized method.
batch_size = 200
seq_len = 5000
features = 8
data = torch.rand((batch_size, seq_len, features))
model = nn.RNN(features, 128, batch_first = True)
outputs, hidden = model(data)
print(outputs.size())
The above iterates through seq_len under the hood. And then you’ll calculate the loss based on the targets for every time step in that sequence.
Thanks for the reply, I’m a little confused here. The output that I get is of shape batch_size, length of signal, hidden. Does that mean I’m getting outputs at each time step ? If so, shouldn’t i be getting one activation at the last time step for each channel. thats is 200, 8, 128 ?
Ideally, you want a corresponding target for each time step. You want to get the loss for each timestep output of the model and use that to adjust the weights. But if you don’t have targets for all of the timesteps, then just pass in what you have into the loss function:
# this example assumes we only have latter targets
loss = loss_function(outputs[:,-targets.size(1):,:], targets)
I’m not clear your meaning on “activation” here. The output of the RNN will simply be [batch_size, seq_len, hidden_dim]. You could run a linear layer on those RNN outputs in order to change the hidden dim to your target size.
2 Likes
Thanks for the reply. It makes sense now. I meant the hidden states. apologies for the ambiguity. Thanks again
Is there a particular reason why you’re using RNN here?
No reason, I’ve tried resnet 50 with 1d convs. RNNs are known for sequential data. So I just wanted to see if it helps me. But the performance with RNN is not as good as Resnet. You got any suggestion that can help me. Thanks
Transformers have worked well for sequential data perfoming better than RNNs so maybe you can look into implementing that for your application.
Thanks. That is the next thing on my list and a Chrononet.