# Inconsistency between PyTorch and TensorFlow's variance function's results and how PyTorch implements it using the summation function?

I’m working on a vision model and it requires using variance function.

## Inconsistency between PyTorch and TensorFlow variance function outputs

For some reasons I am testing the TensorFlow and PyTorch’s variance functions and I find that the results for same input values are different. And it sounds weird because variance is a parameterless function, hence, there are no values to set that might affect the output value.

``````# TensorFlow Example
import tensorflow as tf
x = tf.constant([[1., 2.], [3., 4.]])

print(tf.math.reduce_variance(x))
print(tf.math.reduce_variance(x, 0))
print(tf.math.reduce_variance(x, 1))

# Outputs
# tf.Tensor(1.25, shape=(), dtype=float32)
# tf.Tensor([1. 1.], shape=(2,), dtype=float32)
# tf.Tensor([0.25 0.25], shape=(2,), dtype=float32)
``````
``````# PyTorch Example
import torch as t
x = t.tensor([[1., 2.], [3., 4.]])

print(t.var(x))
print(t.var(x, 0))
print(t.var(x, 1))

# Outputs
# tensor(1.6667)
# tensor([2., 2.])
# tensor([0.5000, 0.5000])
``````

## Implementation help with PyTorch’s `var()` using `sum()` function

Since my model is trained in PyTorch I want to know how to write the `var()` function using `sum()` function. The implementation I tried is as follows but the result doesn’t match with that of PyTorch’s official implementation:

``````import torch as t
_ = t.manual_seed(0)

def var(x, dim):
x_mean = t.mean(x)
sq_devs = t.square(x - x_mean)
return t.mean(sq_devs, dim=dim)

x = t.randn(5, 4, 3)
x_var1 = t.var(x, dim=0)
x_var2 = var(x, dim=0)

print("x:", x)
print("x_var1:", x_var1)
print("x_var2:", x_var2)
``````

Output

``````x: tensor([[[-1.1258, -1.1524, -0.2506],
[-0.4339,  0.8487,  0.6920],
[-0.3160, -2.1152,  0.3223],
[-1.2633,  0.3500,  0.3081]],

[[ 0.1198,  1.2377,  1.1168],
[-0.2473, -1.3527, -1.6959],
[ 0.5667,  0.7935,  0.5988],
[-1.5551, -0.3414,  1.8530]],

[[ 0.7502, -0.5855, -0.1734],
[ 0.1835,  1.3894,  1.5863],
[ 0.9463, -0.8437, -0.6136],
[ 0.0316, -0.4927,  0.2484]],

[[ 0.4397,  0.1124,  0.6408],
[ 0.4412, -0.1023,  0.7924],
[-0.2897,  0.0525,  0.5943],
[ 1.5419,  0.5073, -0.5910]],

[[-0.5692,  0.9200,  1.1108],
[ 1.2899, -1.4959, -0.1938],
[ 0.4455,  1.3253, -1.6293],
[-0.5497, -0.4798, -0.4997]]])
x_var1: tensor([[0.5831, 1.0012, 0.4474],
[0.4593, 1.6567, 1.5645],
[0.3082, 1.8627, 0.9353],
[1.5127, 0.2319, 0.9604]])
x_var2: tensor([[0.4835, 0.8038, 0.5476],
[0.4048, 1.3637, 1.2851],
[0.2938, 1.5346, 0.7877],
[1.3801, 0.2065, 0.8126]])
``````

I’d really appreciate anyone willing to help me here.

Rahul Bhalley

Hi,

I think what you want is to pass `unbiased=False` to the `var` function in pytorch: torch.var — PyTorch 1.8.1 documentation
By default, pytorch compute the unbiased variance but I think Tensorflow computes the biased version.

1 Like

Thank you! It helps.

Hey @albanD! How can I implement the `var()` function using PyTorch’s `sum()` function?

I found the solution myself.

Following is an unbiased estimator implementation of variance using summation function, if anyone needs that:

``````def var(input, dim, keepdim=False):
input_means = t.mean(input, dim=dim, keepdim=True)
difference = input - input_means
squared_deviations = t.square(difference)
return t.mean(squared_deviations, dim=dim, keepdim=keepdim)
``````