Global_mean_pool gives different results every time with same input

1

I use global_mean_pool in torch geometric “x = global_mean_pool(x, batch)” to average node features into graph level features, however, I found that this operator gives me slightly different results every time.
I print the input x and calculated the mean feature in excel, compare to torch output and then I also checked with "x=global_mean_pool(x, None).
Excel calculation:
|0.2111766|0.0019979|0.2978659|0.1123972|0.1230657|
x=global_mean_pool(x,data.batch)
|0.2111767|0.0019979|0.2978655|0.1123973|0.1230659|
x=global_mean_pool(x,None)
|0.2111766|0.0019979|0.2978659|0.1123972|0.1230657|

So most of the output from x=global_mean_pool(x,data.batch) are wrong/off, and everytime I run the same it gives a different solution. Other options are consistent. Why is this operation random? BTW I have fixed all random seed, numpy random seed, torch and torch cuda random seeds.

2

The outputs might be expected if non-deterministic algorithms are used. If so, none of the outputs are wrong as they should show the same error to a wider dtype.

3

Thanks a lot for your reply! I can see a few operators listed on this page that are non-deterministic: torch.use_deterministic_algorithms — PyTorch 2.2 documentation

However, I can’t find any documentation that says global_mean_pool in torch geometric is deterministic or not, or does it have a deterministic implementation. Could you please help point me to a documentation page?

Why defining None instead of batch gives a consistent result?

I also hit an error if the optional “batch” parameter is not defined. The documentation says definition of ”batch” is optional:

https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.nn.pool.global_mean_pool.html

4

I’m not familiar enough with PyG so you might need to ask the authors how the algorithms are implemented and if a deterministic version exists.

1 Like