Triangular_solve transpose keyword argument

1

The keyword argument transpose of the triangular_solve function is confusing me. In the document, it says:

transpose (bool, optional) – whether A should be transposed before being sent into the solver.

In the code below, x_1 makes sense to me, but transpose doesn’t seem to be doing anything in x_2.

import torch

u = torch.tensor([
    [2.0, 0.0],
    [4.0, 3.0],
])

x = torch.tensor([
    [2.0],
    [3.0],
])

b = u.T @ x

x_1 = torch.triangular_solve(b, u.T)
print(x_1.solution)

x_2 = torch.triangular_solve(b, u, transpose=True)
print(x_2.solution)

x_3 = torch.triangular_solve(b, u, transpose=False)
print(x_3.solution)

Output:

tensor([[2.],
        [3.]])
tensor([[8.],
        [3.]])
tensor([[8.],
        [3.]])

torch==1.10.0 and numpy==1.21.3

2

I think you want upper=False when you pass in u (as opposed to u.T).

Best regards

Thomas

3

Thank you for looking into this, Thomas.

  1. Does that mean upper=False transposes the matrix?
  2. What is transpose for then?
4

First, upper=False/True will decide whether PyTorch will look at the upper or lower “half” (including diagonal) of the matrix. You pass in a lower lower triangular matrix, you want upper=False because otherwise it’ll look like a diagonal matrix to PyTorch.

After this, PyTorch will consider transpose=False/True and transpose it. But if, as in the example you posted, PyTorch gets a diagonal matrix (because the upper half is only the entries in the diagonal), you’ll get the same result whether you transpose or not. If you pass in a matrix that is not diagonal, transpose will make a difference, just as you expect.

Best regards

Thomas

5

Thank you for the detailed response, Thomas.

While I don’t think that the function arguments and their behaviors are intuitive or well-explained in the documentation, I now understand what is going on underneath the hood thanks to your explanation.

6

Yeah, I stared down the documentation, implementation, and tests for quite a while before I got it.
I even found a bug or two in the test coverage before noticing that it is the upper not matching the input. :slight_smile:
If you have ideas how to improve the documentation, I think it might well be worth it.
It’s a bit tricky with all the options.