If i have a network of input 13 and output dim 4. I would like to pass the output of dim 4 back into another network with input dim of 6. If I pad the output of 4 dim with 2 extra dim of 0s, use the output of the 2nd Net to construct a loss, but back prop only on the parameters of the 1st network, would the zero padding cause any problems with the back-prop?
If no problems there, what’s a good way of padding the output of the 1st network? Thank you.