Learner pytorch: Extending pytorch

Hi all,
I have a query about the extending nn as in the link below

If I am not wrong, the example in the above link is an example of Linear module. From my basic understanding, linear module just takes in data and multiplies with weight and adds bias ( y = w*x +c)

I have been trying to understand this extension using a fictitious example:
data = [(1,1), (3,9), (2,8), (4,64), (5,125), (6,216)]

from torch.autograd import Variable
for epoch in range(100):
    for i, data2 in enumerate(data):
        X, Y = iter(data2)  #iterating thru data and passing one X,Y pair each time
                            # X is optimizable and requires_grad=True
        X, Y = Variable(torch.FloatTensor([X]), requires_grad=True), Variable(torch.FloatTensor([Y]), requires_grad=False)
        outputs = net(X)
        loss = criterion(outputs, Y)
        if (i % 10 == 0):  
            print("Epoch {} - loss: {}".format(epoch, loss.data[0]))

I got the following error:

Error here:  ----> output = input.mm(weight.t())
RuntimeError: matrices expected, got 1D, 2D tensors at /Users/administrator/nightlies/pytorch-1.0.0/wheel_build_dirs/conda_3.6/conda/conda-bld/pytorch_1544137972173/work/aten/src/TH/generic/THTensorMath.cpp:935

I replaced mm with matmul (as some posts suggested mm is same as mamul, but I dont think so, as mm and not matmul supports broadcasting so they are not same (please, correct me if I am worng)

After changing mm to matmul I still encounter error:
Error: RuntimeError: t() expects a 2D tensor, but self is 1D

While the following works: (although no clue why?).

m = LinearFunction(1,1)
input = torch.randn(10, 2)
weight = torch.randn(1, 2)
X=LinearFunction.apply(input, weight)

In my understanding Linear follows a staright line equation.
One more thing why m=LinearFunction(1,1) with input as 10,2 still works?
May be I am missing something simple understanding, and would appreciate any help/pointers.

It is as the error says, the weight attribute in the forward step should be a 2D tensor to take its transpose, but the weight size for your model is 1D.

In the second example the weight vector is indeed 2D and thus you can take its transpose.

1 Like

Thanks a lot for your reply