Pytorch equivalent of this Tensorflow Neural Network

vision
1

I am trying to build a Pytorch model for a sentence classification task using the Clinical BERT, which can be accessed via the transformers module. Can you please tell me what is the equivalent of this Tensorflow model? I am very new to Pytorch but I am familiar with the basics. Thanks!

bert_model = TFAutoModel.from_pretrained("distilbert-base-uncased")
    bert_model.trainable = False
    
    token_ids = Input(shape=(maxlen,), dtype=tf.int32,
                                      name="token_ids")
    attention_masks = Input(shape=(maxlen,), dtype=tf.int32,
                                            name="attention_masks")
    bert_output = bert_model(token_ids,attention_mask=attention_masks)

    output = Dense(2,activation="softmax")(bert_output[0][:,0])

    model = Model(inputs=[token_ids,attention_masks],outputs=output)

    # compile
    model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])

    # train
    model.fit([tokenized_train["input_ids"],tokenized_train["attention_mask"]],
              train_y, batch_size=25, epochs=3)

    # evaluate
    score = model.evaluate([tokenized_test["input_ids"],tokenized_test["attention_mask"]],test_y,verbose=0)
    
    print("Accuracy on test data:",score[1])

    return model, [tokenized_train["input_ids"],tokenized_train["attention_mask"]], test_y