evgenizer
(ESorkin)
December 28, 2018, 7:25pm
#1
I am confused: can you help me understand why does torch.norm(x) depend on the size of x? Thanks!

```
N=10000
x=torch.ones(N)
n1=torch.norm(x)
n2=numpy.sqrt(N)
```

```
[output]: n1=tensor(100.), n2=100.
```

```
N=40000
x=torch.ones(N)
n1=torch.norm(x)
n2=numpy.sqrt(N)
```

```
[output]: n1=tensor(266.0605) n2=200.
```

smth
December 29, 2018, 1:37am
#2
We calculate the Vector 2-norm by default. And that is a function of the total sum, by definition:

http://mathworld.wolfram.com/L2-Norm.html

However, once you hit floating point precision limits, you get â€śinaccurateâ€ť resultsâ€¦

smth
December 29, 2018, 1:40am
#3
Iâ€™m taking a look at what these accuracy limits are, wrt floating point, and if that might be affecting things.

evgenizer
(ESorkin)
December 29, 2018, 1:40am
#4
I know that, but for x=torch.ones(40000), the answer should be 200, not 266.0605 as I get with torch.norm(x)

smth
December 29, 2018, 1:43am
#5
Iâ€™m sorry, I posted my answer as I was typing it, by mistake.

I looked into this further, it looks like a bug that we introduced in 1.0.0, Iâ€™m taking a further look and filing an issue.

It is only affected on the CPU, and produces the correct result on GPU.

evgenizer
(ESorkin)
December 29, 2018, 1:45am
#6
Got it, I think gpu is affected as well, though.

smth
December 29, 2018, 1:47am
#7
I checked the GPU implementation via:

```
>>> import torch
>>> x = torch.ones(40000)
>>> torch.norm(x)
tensor(266.0605)
>>> torch.norm(x.to(dtype=torch.float32, device='cuda'))
tensor(200., device='cuda:0')
```

Seems to work fine.
Are you seeing incorrect result on GPU as well?

evgenizer
(ESorkin)
December 29, 2018, 1:50am
#8
Double checked it â€“ no, you are right, GPU result is correct. Thanks!

smth
December 29, 2018, 1:57am
#9
I filed an issue at https://github.com/pytorch/pytorch/issues/15602
It will for sure be fixed in our next minor release on Jan 15th, and will be fixed in our nightlies much sooner than that, Iâ€™m having it looked at with high priority.

Really sorry for the bug!

1 Like

smth
January 16, 2019, 11:23pm
#10
just fyi, this is fixed now via https://github.com/pytorch/pytorch/pull/15885 and will go into the 1.0.1 release in the time window of < 1 week from now.

1 Like