Keras to Pytorch Convolutional Neural Network

1

I am trying to convert this keras model to pytorch.

Build VT-CNN2 Neural Net model using Keras primitives –

- Reshape [N,2,128] to [N,1,2,128] on input

- Pass through 2 2DConv/ReLu layers

- Pass through 2 Dense layers (ReLu and Softmax)

- Perform categorical cross entropy optimization

dr = 0.5 # dropout rate (%)
model = models.Sequential()
model.add(Reshape([1]+in_shp, input_shape=in_shp))
model.add(ZeroPadding2D((0, 2)))
model.add(Convolution2D(256, 1, 3, border_mode=‘valid’, activation=“relu”, name=“conv1”, init=‘glorot_uniform’))
model.add(Dropout(dr))
model.add(ZeroPadding2D((0, 2)))
model.add(Convolution2D(80, 2, 3, border_mode=“valid”, activation=“relu”, name=“conv2”, init=‘glorot_uniform’))
model.add(Dropout(dr))
model.add(Flatten())
model.add(Dense(256, activation=‘relu’, init=‘he_normal’, name=“dense1”))
model.add(Dropout(dr))
model.add(Dense( len(classes), init=‘he_normal’, name=“dense2” ))
model.add(Activation(‘softmax’))
model.add(Reshape([len(classes)]))
model.compile(loss=‘categorical_crossentropy’, optimizer=‘adam’)
model.summary()

Layer (type) Output Shape Param # Connected to

reshape_1 (Reshape) (None, 1, 2, 128) 0 reshape_input_1[0][0]

zeropadding2d_1 (ZeroPadding2D) (None, 1, 2, 132) 0 reshape_1[0][0]

conv1 (Convolution2D) (None, 256, 2, 130) 1024 zeropadding2d_1[0][0]

dropout_1 (Dropout) (None, 256, 2, 130) 0 conv1[0][0]

zeropadding2d_2 (ZeroPadding2D) (None, 256, 2, 134) 0 dropout_1[0][0]

conv2 (Convolution2D) (None, 80, 1, 132) 122960 zeropadding2d_2[0][0]

dropout_2 (Dropout) (None, 80, 1, 132) 0 conv2[0][0]

flatten_1 (Flatten) (None, 10560) 0 dropout_2[0][0]

dense1 (Dense) (None, 256) 2703616 flatten_1[0][0]

dropout_3 (Dropout) (None, 256) 0 dense1[0][0]

dense2 (Dense) (None, 11) 2827 dropout_3[0][0]

activation_1 (Activation) (None, 11) 0 dense2[0][0]

reshape_2 (Reshape) (None, 11) 0 activation_1[0][0]

Here is my Pytorch code: The input has already been converted to [N,1,2,128] , The number of classes is 11. Think the issue is with the forward pass. Thanks :slight_smile:

#Input [N , 1 , 2, 128]
class ConvNet(nn.Module):
def init(self,dr=0.5):
super(ConvNet,self).init()
self.pad = nn.ConstantPad2d((2,2,0,0),0)
self.conv1 = nn.Conv2d(1,256,kernel_size=(1,3)) #(Input_channels, Output_channels, Kernel)
self.drop = nn.Dropout(dr)
self.zeropad2 = nn.ConstantPad2d((2,2,0,0),0)
self.conv2 = nn.Conv2d(256,80,kernel_size=(2,3))
self.flatten = nn.Flatten()
self.Dense = nn.Linear(10560,128) # at the point will get flatten, the number of neurons required will be 10560
self.Dense2 = nn.Linear(256,11)
self.softmax = nn.Softmax()

def forward(self, x):
    x = self.pad(x) #pad
    x = F.relu(self.conv1(x)) #conv1
    x = self.drop(x) #dropout
    x = self.zeropad2(x) #zeropadding2
    x = F.relu(self.conv2(x)) #conv2
    x = x.view(x.size(0),-1) #flatten
    x = self.relu.Dense(x) #Linear
    x = x.view(x.shape[0],-1) #Linear
    x = self.relu.Dense2(x)
    x = F.softmax(x) #softmax
    return x
2

I don’t know which arguments Convolution2D expect, but based on:

# Keras
model.add(Convolution2D(256, 1, 3, border_mode=‘valid’, activation=“relu”, name=“conv1”, init=‘glorot_uniform’))

# PyTorch
self.conv1 = nn.Conv2d(1,256,kernel_size=(1,3)) #(Input_channels, Output_channels, Kernel)

it seems as if you assume the 2. and 3. arguments in Keras are defining the kernel size?
Could you double check it, as I would expect to see a tuple argument instead of separate inputs.

Also,

self.Dense = nn.Linear(10560,128)
self.Dense2 = nn.Linear(256,11)

won’t work since Dense returns 128 features while Dense2 expects 256.
You wouldn’t need to flatte nthe activation again after the first linear layer as you’ve already flattened it after conv2.
Also, remove the softmax layer in PyTorch as nn.CrossEntropyLoss expects raw logits.

PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier.