I have an RCC which outputs tensorrs of size 516x24 that I’d like to get to size 4x6. I have a single linear layer that reduces it to 516x6, but I don’t know how to reduce the 1st dim of 516. Any suggestions? Thanks!
The first dimension is just the batch size. So you can just lower your batch size if you want to get the output to be smaller.
My batch_size is 4. Input X is (batch_size, 129, 64). RNN outputs (batch_size, 129, 516). Then I pass the output through a .contiguous.view(-1, 516), which gives me a size of (4, 516), which I pass through a linear layer to get size of (516, 6). However, I need it to be (4, 6) at the end.
Can you show me your code?
I think I figured it out, by changing the input size of my linear layer to be large enough. I don’t know if this is the best approach though. Here’s my model code:
def __init__(self, input_size, output_size, hidden_dim, n_layers):
super(Net, self).__init__()
self.input_size = input_size
self.output_size = output_size
self.hidden_dim = hidden_dim
self.n_layers = n_layers
self.rnn = torch.nn.RNN(self.input_size, self.hidden_dim, self.n_layers, batch_first=True)
# 64 is the size of my middle (2nd) dimension of X
self.fc = torch.nn.Linear(64 * hidden_dim, self.output_size)
def forward(self, X):
batch_size = X.size(0) # 4
hidden = self.init_hidden(batch_size)
out, hidden = self.rnn(X, hidden)
out = out.contiguous().view(batch_size, -1)
out = self.fc(out)
return out, hidden
def init_hidden(self, batch_size):
return torch.zeros(self.n_layers, batch_size, self.hidden_dim)```That looks good. One question however, why are you returning the hidden layer if you just reinitialize it every forward pass? Doesn’t that defeat the purpose of returning it.
I was following the tutorial below, which does that. I think it’s just initialising the very first hidden state for each batch, isn’t it? I don’t use the return value though. Does it serve any use outside of training the model?
No it doesn’t look like it does. It does not matter much though I am glad you fixed the issue.