Novice here. I am trying to implement convnet on top of huggingface bert-base-uncased, and I am using a tensorflow example. The original model looks something like:
tf_roberta_model (TFRobertaMode ((None, 96, 768), (N 124645632 input_1
dropout_38 (Dropout) (None, 96, 768) 0 tf_roberta_model
conv1d (Conv1D) (None, 96, 128) 196736 dropout_38
I have a couple questions:
- how can an input shape of (None, 96, 768) after being applied: tf.keras.layers.Conv1D(128, 2,padding=‘same’)(x1) output a shape of (None, 96, 128) ?
- I’m struggling to implement the same as above in pytorch, and so far I got it to run (albeit does no good) as follows: nn.Conv1d(in_channels=128, out_channels=1,kernel_size=2). What am I doing wrong?
Thanks in advance!!!