Building the network in pytorch

I am trying to implement the following model in torch. I have found source code for Tensorflow but not Torch. If there is any source that I can read to help me build the model in Torch that would be great.

basically, I need to know:

  1. how to feed two inputs
  2. how to specify connections similar to the image

Thank you!

Hi, could you explain little bit more what do you want to do? view1 and 2 are your inputs (just by arrow direction…)?

Sure! thanks:-)

View1 and View2 are the two inputs. Both inputs are fully connected to one of the middle layers ( the right one), and to separate neurons in the other latent space ( the left one, you can see that view 1 is connected to the right neuron in the left box, and view 2 is connected to the left neuron in the left box)

Maybe @ptrblck may help you in a better way but I think what you can do is building up the whole model as a single nn.Linear layer and to set gradients equal to 0 for the corresponding neurons in the backward pass (obviously initializing that weight as zero too)

It’s a little bit mess because weights are passed as a package, so you cannot disable requires grad in a subgraph.

Another option could be to initialize corresponding weights to 0 and detaching specific neurons during the forward pass.

As you can see it’s a mess.