RuntimeError: size mismatch, m1: [1 x 30], m2: [20 x 128] at C:\w\1\s\windows pytorch aten src TH generic THTensorMath.cpp:752

nlp
1

Hey, This is my part of code

#cnt dimension[17152, 1]
class NGram(nn.Module):
    def __init__(self,vocab_size,embedding_dim,context_size):
        super(NGram,self).__init__()
        self.embeddings=nn.Embedding(vocab_size,embedding_dim)
        self.linear1=nn.Linear(context_size*embedding_dim,128)
        self.linear2=nn.Linear(128,vocab_size)
    def forward(self,inputs):
        embeds=self.embeddings(inputs)
        embeds=embeds.view(1,-1)
        out=self.linear1(embeds)
        
        out=out.F.relu(out)
        #dimension=(1,128)
        
        out=self.linear2(out)
        #dimension=(1,vocab_size)
        
        log_probs=F.log_softmax(out,dim=1)
        return log_probs
    def extract(self,inputs):
        embeds=self.embeddings(inputs)
        return embeds
=====
target=[]
for i in trigrams:
    target.append(i[2])
losses=[]
criterion=nn.NLLLoss()
model=NGram(len(vocab),10,2)
optimizer=optim.SGD(model.parameters(),lr=0.001)
for epoch in range(20):
    total_loss=torch.Tensor([0])
    for context in trigrams[:17152]:
        context_idxs=[word_to_indx[w][0] for w in context]
        context_var=Variable(torch.LongTensor(context_idxs))
        optimizer.zero_grad()
        log_probs=model(context_var)
        loss=criterion(log_probs,Variable(torch.LongTensor([word_to_idx[target]])))
        loss.backward()
        optimizer.step()
        total_loss+=loss.data
    losses.append(total_loss)
    print("{}epochs loss->{:.2f}".format(epoch,total_loss.numpy()[0]))
===============

RuntimeError                              Traceback (most recent call last)
<ipython-input-20-03b51c2b5546> in <module>
     12         context_var=Variable(torch.LongTensor(context_idxs))
     13         optimizer.zero_grad()
---> 14         log_probs=model(context_var)
     15         loss=criterion(log_probs,Variable(torch.LongTensor([word_to_idx[target]])))
     16         loss.backward()

c:\python36\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

<ipython-input-18-7e8cf4a210e6> in forward(self, inputs)
      8         embeds=self.embeddings(inputs)
      9         embeds=embeds.view(1,-1)
---> 10         out=self.linear1(embeds)
     11 
     12         out=out.F.relu(out)

c:\python36\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

c:\python36\lib\site-packages\torch\nn\modules\linear.py in forward(self, input)
     85 
     86     def forward(self, input):
---> 87         return F.linear(input, self.weight, self.bias)
     88 
     89     def extra_repr(self):

c:\python36\lib\site-packages\torch\nn\functional.py in linear(input, weight, bias)
   1367     if input.dim() == 2 and bias is not None:
   1368         # fused op is marginally faster
-> 1369         ret = torch.addmm(bias, input, weight.t())
   1370     else:
   1371         output = input.matmul(weight.t())

RuntimeError: size mismatch, m1: [1 x 30], m2: [20 x 128] at C:\w\1\s\windows\pytorch\aten\src\TH/generic/THTensorMath.cpp:752

the error: RuntimeError: size mismatch, m1: [1 x 30], m2: [20 x 128] at C:\w\1\s\windows\pytorch\aten\src\TH/generic/THTensorMath.cpp:752
does my linear network have wrong?

2

Your code and comments are not really helpful, so I can only make some half-informed guesses.

I assume the shape of inputs is (1, 3) since it comes from a array called trigrams. After embeds=self.embeddings(inputs) the shape of embeds is (1, 3, 10) with 10 being the embed_dim. After embeds=embeds.view(1,-1) the shape of embeds is (1, 30).

However, self.linear1=nn.Linear(context_size*embedding_dim,128), given your parameter values means self.linear1=nn.Linear(2*10,128). Here you have to mismatch between the tensor of size 30 from embeds and the expected size of 20 from self.linear1.

If my assumptions are correct, model=NGram(len(vocab),10,3) should help.

3

Your right !!, Dimension will drive every crazy