Greetings to everyone!
I have a dataset with 4 modalities and I want to feed each modality to a LSTM or RNN and then combining the results of all the 4 networks in order to obtain a global result.
I wrote a couple of examples with RNN and softmax at the end, but I don’t know how to deal with 4 networks at once.
Does anyone knows a simple example of combining 2 LSTM’s or RNN’s by means of late fusion in PyTorch?
Should I feed all the 4 networks to the same loss function?
Thanks a lot!