How to work with 3D Torch in a linear forward pass

1

Hi everyone!

I saw several topics on how to feed a 3D tensor into a Neural Network but it’s not quite clear to me what’s happening.
I’m working with stock data and for each individual stock I have a 2 dimension torch containing several information (in columns) for each day. So, for each stock I have a matrix with features for column and days for rows and my NN works fine for a single stock. However, I’d like to apply the same operations for several stocks, thus yielding a 3d torch with the third dimension referring to each individual asset.
If I reshape the data into a 2d torch (stacking the features) I think i’ll lose information when trying to predict a single outcome with the trained NN. I’ve also found some topics explaining how to pass a 3d tensor to a Linear pass, but I understand they will make the Linear regression layer by layer, whereas I’d like to minimize the error for the whole 3d tensor.

Do you have any thoughts on that that might help me out?

thanks!

2

Would you like to use the same linear layers for both inputs?
If so, you could just create a tensor of the shape [batch_dim, nb_stocks, in_features] and the layers will be applied on each nb_stock dimension.

3

I tried that, but it seems the output converges on the out layer. It is, instead of the output adjusting to the stock series, it returns a horizontal line. I’m not sure what the problem might be

4

Could you try to fit a very small data sample (e.g. just one or five samples) and see if your model correctly overfits on this small sample?
If that’s not possible, there might be other problems in your code.
However, if that works, I would try to scale up the problem using a bit more samples and see, when the model training breaks. Maybe your model’s architecture is currently not suitable for the problem or your model has just not enough capacity.