Use of BatchNormalisation variables

C++
1

Hi all,
due to IT reasons, I cannot run the inference on my dataset using pytorch (long story).
I have created a C++11 code that loads weights and biases from a pytorch training and uses them to run the inference.
So far, the C++ inference works fine.

However, the model I’m currently using for my dataset relies on Batch Normalisation techniques (BatchNorm1D); I managed to extract the following normalisation variables from my training:

  • normalise.weight (vector)
  • normalise.bias (vector)
  • normalise.running_mean (vector)
  • normalise.running_var (vector)
  • normalise.num_batches_tracked (int number)

I have some troubles understanding how to use these values to normalise the inputs during the inference process. To be clear, I need to understand the underlying vector/matrix algebra of the normalization process.

I know that, given X as input:

X_norm = (X - mean)/sqrt{var + 1e-5}
out = gamma * X_norm + beta

but it is not clear to me which of the previous values are gamma and beta. My guess would be:
gamma = normalise.weights
and
beta = normalise.bias
I had a look at the original Batch Normalization paper, but that focuses more on the normalisation in batches during the training.

Any idea?
Thanks

2

You are correct in your assumption.
gamma corresponds to the weights and beta to the bias.

Are you now allowed to use PyTorch in Python or PyTorch in general for inference?
If it’s all about Python, we also have a C++ API, which could make your life a bit easier.

3

Thank you for your answer @ptrblck!

I tried using the C++ API for inference but it is not a feasible option due to technical issues with compilation.
I need to implement the inference process on a code based on CERN ROOT; something fails during the linking when trying to link both the Torch libraries (either pre-built of from source) and CERN ROOT and I’m therefore unable to use torch to implement the inference process on our code (you can find additional details here: Link to ROOT forum).

After a couple months of work I managed to compile CERN ROOT + Torch under gcc 4.8.5, which seems to be the only version of gcc that supports this combination of libraries; unfortunately, gcc 4.8.5 is not supported by our IT department.

4

If you have such restrictions, would it be possible to isolate your code in a docker container and install all libs there?
This wouldn’t touch the bare metal, so that you won’t mess up the server install.

5

That’s a possibility; to be honest, I’m not really experienced with docker containers and the code will have to run on a distributed computing system. I guess this would make such an implementation way more complex