Different accuracy when I don't use Distributed Mode

I am currently running a code based on a paper I found on GitHub that corresponds to a paper published in ICLR 2023. I am using a single GPU (no distributed mode). However, the results are significantly different from what was reported in the paper. Is it expected for the results to differ? For instance, the result on a single GPU for the DTD dataset is 50.1%, whereas in the paper, it is reported as 54.1% using Vit-B/16.