I’m feeding one-hot vectors representing words in sentences to an LSTM. Can those individual vectors be bundled up into a 3D tensor which I then pass into the LSTM as a batch?
In traditional sense of RNN in NLP, you pass each sentence as a data in a batch. Also I’m not sure how you can bundle 1D vectors into 3D tensors.
I would bundle the individual vectors (1x60) into a Variable. So if the number of letters for a sentence was 200, my input would look like:
input (batch_first) = 200 x 1 x 60
nn.LSTM( input , (h_0, c_0), batch_first = true)
Does that make sense?
From your code it seems that you don’t want each letter to depend on previous letter. Then you should use normal MLP, which does the same thing as is better suited and optimized for such task.
I do actually! Trying to recreate word formation in certain kinds of news
articles. Just curious, what about my approach suggested letter order did
not matter?
You should bundle them in the num_seq dimension then (dim 1). Since your dim 1 now has size 1, it indicates that each sequence is of length 1. RNNs do not preserve information across sequences.