Track_running_stats removed?

JerryBai · February 27, 2018, 12:32am

Is the 'track_running_stats" attr removed from init() of nn.BatchNorm1d() and nn.BatchNorm2d() in latest Pytorch?

jpeg729 · February 27, 2018, 1:04am

Nope.

pytorch/pytorch/blob/master/torch/nn/modules/batchnorm.py#L12


from .module import Module
from torch.nn.parameter import Parameter
from .. import functional as F




# TODO: check contiguous in THNN
# TODO: use separate backend functions?
class _BatchNorm(Module):


def __init__(self, num_features, eps=1e-5, momentum=0.1, affine=True,
             track_running_stats=True):
    super(_BatchNorm, self).__init__()
    self.num_features = num_features
    self.eps = eps
    self.momentum = momentum
    self.affine = affine
    self.track_running_stats = track_running_stats
    if self.affine:
        self.weight = Parameter(torch.Tensor(num_features))
        self.bias = Parameter(torch.Tensor(num_features))
    else:

JerryBai · February 27, 2018, 1:53am

I have also noticed this, but when I tried the following simple test, I got an error:

import torch
import torch.nn as nn
module = nn.BatchNorm2d(3, track_running_stats=False)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in ()
** 1 import torch**
** 2 import torch.nn as nn**
----> 3 module = nn.BatchNorm2d(3, track_running_stats=False)

TypeError: init() got an unexpected keyword argument ‘track_running_stats’

jpeg729 · February 27, 2018, 2:00am

My mistake, that link was for the source code of the master branch. You must be running the latest pytorch release, i.e. version 0.3.1.

Make sure you check the docs for the proper release version, in this case http://pytorch.org/docs/0.3.1/nn.html#torch.nn.BatchNorm1d
rather than the docs for the master branch.

The track_running_stats argument was not removed from __init__() of nn.BatchNorm*d. It was added after the last release.

JerryBai · February 27, 2018, 2:10am

I see, thanks a lot!