Why can't I have out_features = 1 in a Linear layer?

1

I cannot create and use a torch.nn.Linear layer with a single output (i.e., out_features = 1).

The simplest example I can reproduce is

t = torch.tensor([
    [1.0, 2.0],
    [3.0, 4.0],
    [5.0, 6.0],
])
lin = torch.nn.Linear(2, 1)
lin(t)

which gives

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/workspaces/scratch/python/tsp.ipynb Cell 9 line 7
      1 t = torch.tensor([
      2     [1.0, 2.0],
      3     [3.0, 4.0],
      4     [5.0, 6.0],
      5 ])
      6 lin = torch.nn.Linear(2, 1)
----> 7 lin(t)

File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py:114, in Linear.forward(self, input)
    113 def forward(self, input: Tensor) -> Tensor:
--> 114     return F.linear(input, self.weight, self.bias)

RuntimeError: could not create a primitive descriptor for a matmul primitive

This is unexpected since it seems like all other values for out_features work fine. For example

t = torch.tensor([
    [1.0, 2.0],
    [3.0, 4.0],
    [5.0, 6.0],
])
lin = torch.nn.Linear(2, 3)
lin(t)

gives

tensor([[-1.7300, -1.4911, -0.7885],
        [-2.9954, -2.1098, -2.1714],
        [-4.2608, -2.7285, -3.5543]], grad_fn=<AddmmBackward0>)

How do I get a 1D output (for each example) from a Linear layer?

2

Your code runs fine for me. No errors. Perhaps you could report an issue on the PyTorch github and include your version, OS, etc.

3

Same here, the example code runs fine, both on CPU or GPU (PyTorch version 2.0.1+cu117 on Ubuntu 22.04)

4

Thanks for testing this!

It seems like this is a bug that affects ARM and was already handled in nn.Linear forward error on AArch64 if the out_features equals to 1 · Issue #110149 · pytorch/pytorch · GitHub. Apologies for not checking there first – I didn’t realize it was a bug and assumed I was misusing something.

Thanks again.