In this issue @ezyang references an implementation of convolutions that uses the Toeplitz matrix.
I have a state_dict (and also a nn.Module class) from a network and explicitly need these Toeplitz matrices for further calculations but I admittedly have not a strong grasp on the things going on in ATen and how I could use that directly in Python. Is there a way to do this?
Hi McLawrence,
Please refer to the nn.Unfold module. This would enable you to generate the toeplitz matrices(column matrix).
Hope this helps!
@Mazhar_Shaikh Using unfold, I can create a matrix from the input and do a matrix multiplication with the kernel vector.
However, what I need would be a matrix from the kernel, not the input. Which seems not possible with unfold (as the kernel is smaller than the input). Do you know of an alternative?
I’m not sure if this could be useful. However, since I recently needed a Toeplitz matrix in custom layer and this was the only resource I could find, I prepared a function that returns the Toeplitz matrix for real values using the Unfold module.
Here’s the function:
def pytorch_toeplitz(V):
'''
It creates the Toeplitz matrix for each row in V.
INPUT:
V: torch tensor (batch_size x d)
OUTPUT:
T: (batch_size x d x d)
EXAMPLE:
V = torch.tensor([[1, 0.5, 0.1], [1.3, 0.9, -0.1]])
T = pytorch_toeplitz(V)
print(T.shape)
print(T[0])
print(T[1])
'''
d = V.shape[1]
A = V.unsqueeze(1).unsqueeze(2)
A_nofirst_flipped = torch.flip(A[:, :, :, 1:], dims=[3])
A_concat = torch.concatenate([A_nofirst_flipped, A], dim=3)
unfold = torch.nn.Unfold(kernel_size=(1, d))
T = unfold(A_concat)
T = torch.flip(T, dims=[2])
return T