Produce output of given size

1

Here inputs are sequence of indices with shape (35, 8, 1) and targets of size (35).
input_size = 1
output_size = 28
hidden_size = 64
rnn = CharRNN(input_size, hidden_size, output_size, num_layers=5)

class CharRNN(nn.Module):
def init(self, input_size, hidden_size, output_size, num_layers=1):
super(CharRNN, self).init()
self.hidden_size = hidden_size
self.rnn = nn.RNN(input_size, hidden_size, num_layers, batch_first=True)
self.fc = nn.Linear(hidden_size, output_size)

def forward(self, x, h_state):
x, h_state = self.rnn(x, h_state)
print(x.size())
out = self.fc(x)
return out, h_state
Here I am getting output of size (35, 8, 28) but I want it to be of size (35, 28) as I am trying to generate 28 probabilities for each of 35 sequences and getting the index having maximum value from those 28 probabilities for each sequence. Can anyone please help me?

2

Hi Akhil!

As I understand it, you have a batch size of 35, a sequence length of 8, and
an input_size (the number of ā€œfeaturesā€ that each ā€œitemā€ in a sequence has)
of 1.

The output x of self.rnn() still has the sequence-length dimension of 8 (as
well as the batch dimension of 35).

self.fc is a Linear. Linear is applied just to the final dimension of its input,
with any preceding dimensions being passed through unchanged. So the output
of self.fc also has the sequence-length dimension (as well as the batch
dimension).

This is why your final output has the unwanted sequence-length dimension of 8.

How best to fix this depends on the logic of your use case. But two possibilities
could be:

Pass just the last ā€œitemā€ in the sequence output by self.rnn to self.fc:

out = self.fc (x[:, -1, :])

Or flatten() the sequence-length and hidden_size dimensions of the output
of self.fc together and pass that to a Linear with a larger in_features:

self.fc = nn.Linear (8 * hidden_size, output_size)
...
out = self.fc (x.flatten (1, 2))

Both of these will give you out with shape [batch_size, output_size], which
equals [35, 28].

Best.

K. Frank

1 Like
3

Hey KFrank!

First of all thank you for such a well explained answer. Yes, I have also made the same changes and it is working now without any error.
Also you have explained me why self.fc has all the preceding dimensions.
I come to know another method for doing the same which is using ā€˜flattenā€™.
Thanks

Best.
Akhil