Are features within an LSTM connected

1

with an LSTM layer with multiple features, do the features within the LSTM have any effect on one another? Is there some level of a fully connected layer for the features of each time slice?

lstm = torch.nn.LSTM( 2, hidden_size=10, num_layers=1, batch_first=True)
features = torch.rand( (1,10,2))

'''
tensor([[[0.3317, 0.1547],
         [0.1294, 0.4251],
         [0.3386, 0.1474],
         [0.9933, 0.0323],
         [0.1047, 0.8910],
         [0.2937, 0.5866],
         [0.7244, 0.8024],
         [0.6509, 0.6176],
         [0.2874, 0.3786],
         [0.1911, 0.8797]]])
'''

With the above example would feature column 1 have any effect or interaction with feature column 2 for each time step? Or are they treated separately within the LSTM such that each feature series is independent of one another?

2

1000022804
10000228041240×1155 90.1 KB

Do you see those Xs inside circles? Those are matrix multiplications (matmul). When you matmul(or matrix-vector product) two matrices(or in this case, a vector and a matrix), all parts of the matrix(or vector) interact. So, yes, the input features interact with one another before the output.

1 Like