Cannot handle multiple inputs to forward

1

I am having problems with this:

# hidden features
n_hidden = 256
# input activation factor 
n_fac = 42 
# batch size
bs = 512

class Model007(nn.Module):
    def __init__(self, vocab_size, n_fac):
        super().__init__()
        self.l1 = nn.Embedding(vocab_size, n_fac)
        self.l2 = nn.Linear(n_fac, n_hidden)
        
        
    def forward(self, c1, c2, c3):
        print("c1", c1)
        print("c2", c2)
        print("c3", c3)
        
        in1 = torch.relu(self.l2(self.l1(c1)))
        in2 = torch.relu(self.l2(self.l1(c2)))
        in3 = torch.relu(self.l2(self.l1(c3)))
                
        return in1 + in2 + in3
    
x1 = np.array([13,3,28,24,33,2,3,62,47,58])
x2 = np.array([15,21,30,27,17,3,3,54,54,44])
x3 = np.array([32,2,27,19,2,3,32,3,60,47])
x = np.stack([x1,x2,x3], axis=1)
y = np.array([3,28,24,33,2,3,62,47,58,54])
# converting to tensor
X = torch.from_numpy(x).cuda()
Y = torch.from_numpy(y).cuda()
print(X)
print(Y)
print("...")



ds = utils.TensorDataset(X, Y) 
dl = utils.DataLoader(ds, batch_size=2, shuffle=False) 
print(ds.tensors)
it = iter(dl)

# mb mini bach, yt is target
mb, yt = next(it) 
print(mb)

m = Model007(vocab_size, n_fac).cuda()

y_hat = m(mb)

print(y_hat)

The error I am getting is like this:

TypeError: forward() missing 2 required positional arguments: 'c2' and 'c3'
2

Hi,

You should never call .forward() directly, but call the module on the inputs: m(mb).

3

OK, I will improve this in code now.

4

So your forward function takes 3 things as input def forward(self, c1, c2, c3): but only one is given y_hat = m(mb).

5

Yes, this is the case. I watched some video from 2018 where just one parameter x provided to forward was enough if it has len(x) = 3.

6

I don’t think so. Wasn’t he doing m(*mb) ? Notice the *.

7

Ah yes. image
This was the case indeed.

8

@albanD
Any idea how to tweak code to make *mb to be list of three parameters like in previous versions of PyTorch.

m = Model007(vocab_size, n_fac).cuda()
ds = utils.TensorDataset(X, Y) 
dl = utils.DataLoader(ds, batch_size=2, shuffle=False) 
it = iter(dl)
*mb, yt = next(it)
y_hat = m(*mb)
9

What *x, y = some_tuple does in python 3+ (I don’t think it works for 2.x) is store the last element of the tuple in y and the rest in x. If if next(it) returns 4 elements in your code sample, mb will be a tuple with 3 element after doing *mb, yt = next(it).

10

Confirmed, * notation doesn’t work in Python 2. I was using Python3 :wink:

I learned about this relatively new Python feature two days ago, I admit:
*a,b = [1,2,3]. This would set a=[1,2] and b=3.

I tried both:
ds = utils.TensorDataset(X1, X2, X3, Y) # three inputs and output Y
as well as: ds = utils.TensorDataset(X, Y) # X is stack of x1, x2, x3

In the second case I just had to unbind mini-batch to take the columns:
torch.unbind(mb, dim=-1)
Each column in mini-batch represents the new input to my mini RNN I was making.

The number of rows in mini-batch is equal to the batch_size, or less if no enough data. You helped. For instance, if I had mini batch like this:

tensor([[13, 15, 32],
        [ 3, 21,  2]], device='cuda:0')
torch.Size([2, 3])

After tup = torch.unbind(mb, dim=1) the returned tuple I expected was:

(tensor([13, 3], device='cuda:0'), tensor([15, 21], device='cuda:0'), tensor([32, 2], device='cuda:0')) <class 'tuple'>

Each of these tensors represent three inputs to forward() method.
Strange, I got the same output for dim=-1.