I am trying to implement my own model based on RoBERTa. My Python code looks like below.
class My_RoBERTa(nn.Module): def __init__(self, some_config, some_version): super().__init__() self.roberta = RobertaModel.from_pretrained(some_version) def forward(self, input_ids, attn_masks, labels=None): print(1) print(input_ids.size()) print('* *') print(attn_masks.size()) print('* *') outputs = self.roberta(input_ids, attn_masks) print(3) print(4) print(5)
If I instantiate this model and pass 3 examples with a length = 25, then my expectation of output is below
1 torch.Size([3,25]) * * torch.Size([3,25]) * * 3 4 5
However, when I run it, I actually get something like below.
1 torch.Size([3,25]) * * torch.Size([3,25]) * * 1 torch.Size([3,25]) * * torch.Size([3,25]) * * 3 3 4 4 5 5
I think this is some kind of overwriting issue caused by the “forward” method, but I do not know how to resolve this problem. Could anyone please help me out?
Thank you so much in advance!