Negative value of running_var in batchnorm layer

MinLiAmoy · May 21, 2018, 11:18am

I try to train a YOLO object detector by pytorch and implement the model on FPGA. And When I check the value of running_var in batchnorm layer, there were some negative values.
I thought running_var was supposed to be positive. Did I miss something? Or Could anyone tell me the formula to calculate the batchnorm output with variables weight, bias, running_var, running_mean?

Thanks,
Min

Stone · June 8, 2018, 2:48am

Hi, I came across this problem too. This problem is caused by the 32/64 bit compiler problem. The metadata of Yolo’s weight file is composed of 3 int and 1 size_t, unfortunately the size_t depends on your compiler and machine. For the official weight file, size_t==32bit, for your own weight file, it varies.

In brief, you may want to escape the first 5 int32 instead of 4 int32.

e.g.

github.com

marvis/pytorch-yolo2/blob/6c7c1561b42804f4d50d34e0df220c913711f064/darknet.py#L247


            loss.coord_scale = float(block['coord_scale'])
            out_filters.append(prev_filters)
            models.append(loss)
        else:
            print('unknown type %s' % (block['type']))


    return models


def load_weights(self, weightfile):
    fp = open(weightfile, 'rb')
    header = np.fromfile(fp, count=4, dtype=np.int32)
    self.header = torch.from_numpy(header)
    self.seen = self.header[3]
    buf = np.fromfile(fp, dtype = np.float32)
    fp.close()


    start = 0
    ind = -2
    for block in self.blocks:
        if start >= buf.size:
            break

you may try count=5 in this case.