Skip layer in pyTorch

1

Thank you all for the help in this fabulous forum.

I was a Torch user, and new to pytorch. Right now I want to do something like “skip connection”.

Below is the code I used in torch. How can I make a “skip connection” in pytorch

main = nn.Sequential()
...
local conc = nn.ConcatTable()
local conv = nn.Sequential()
conv:add(SpatialConvolution(...))
conc:add(nn.Identity())
conc:add(conv)
main:add(conc)
main:add(nn.CAddTable())
3 Likes
2

Pytorch is very similar to nngraph in LuaTorch, except that you dont have Cadd, Cmul or any of the table layers. Its the normal +, * operator.

Assuming proper padding for compatible sizes -

input = Variable(torch.Tensor(...))
conv_out =self.conv(input)
out = conv_out + input
2 Likes
3

A skip connection just requires passing the return value from one layer to one further down:

Models will typically add or concatenate the skip connection:

4 Likes