How to compute covariance with torch.cov?

1

I noticed that the numpy and torch interface are not the same:

https://numpy.org/doc/stable/reference/generated/numpy.cov.html

https://pytorch.org/docs/stable/generated/torch.cov.html

torch.cov(input, *, correction=1, fweights=None, aweights=None) → Tensor

numpy.cov(m, y=None, rowvar=True, bias=False, ddof=None, fweights=None, aweights=None, *, dtype=None)[source]

Which means I can’t easily compute the covariance of two batches…since torch.conv does not take y. Is this a mistake?

2

Per the numpy docs, passing y separately should be equivalent to stacking it to “m” or (cat’ing in torch parlance):
numpy.cov — NumPy v1.21 Manual

>>> import torch
>>> x = torch.randn(128,128,dtype=torch.double)
>>> y = torch.randn(128,128,dtype=torch.double)
>>> X = torch.cat((x, y), axis=0)
>>> import numpy as np
>>> np_res = np.cov(x.numpy(), y.numpy())
>>> res = torch.cov(X)
>>> (res.shape, torch.tensor(np_res).shape)
(torch.Size([256, 256]), torch.Size([256, 256]))
>>> torch.allclose(res, torch.tensor(np_res))
True
>>>
3

I just did this:

def cov(x: Tensor, y: Optional[Tensor] = None) -> Tensor:
    """
    Compute covariance of input

    :param x: [M, D]
    :param y: [M, D]
    :return:
    """
    if y is not None:
        y = x
    else:
        assert x.size(0) == y.size(0)
    # - center first
    x = center(x, dim=0)
    y = center(y, dim=0)
    # - conv = E[XY] is outer product of X^T Y or X Y^T depending on shapes
    sigma_xy: Tensor = x.T @ y
    return sigma_xy

should be the same no?