Ensuring same predict results cross Python and C++

C++
1

I have trained my model in Python, now I’m trying to export it to our producing env that must run model in c++.

I tried a few of ways(trace/script/onnx) to export the model, and basically got simimlar results.
For example, I did things in Python as following:

torch.set_grad_enabled( False )
model.eval()
model.freeze()
model.to_onnx( "path/to/file.onnx", input_sample=features, dynamo=True, export_params=True )

And then do inference in C++ according to the ONNX Tutorials.

The strange thing is that I got 43% results exactly same with the ones from Python.
Because it’s a multi-task classification(6 label variables, and each var should be classified into 3 classes), I think that this ratio isn’t probabally a random results.

To avoid directly comaring float numbers, I didn’t comare the logits or probability values, instead apply an ArgMax on both of Python and C++ sides before I compare the integer class indexes.
I’m a fresh man with DL, how can I trouble shot and where should I begin at?
Thanks!!!

1 Like
2

It’s hard to impossible to make 2 different implementations output the same floating point number (bit-wise equality), and if one implementation outputs [0.499, 0.501] for a binary classification and the other [0.501, 0.499] (just as an example, it can be in the logits scale as well), you will get different output labels. So I would start by comparing the logits, and not the labels, to see if it is a matter of floating point different implementation, or something else.

3

Thanks for your help!

I changed my comparing codes according to your advice,
and almost all result probabilities matched when I give a very large tolerance(0.01) to math.isclose().

To my supprise, the deviation of float point(FP32) computing is so large.