[Tensorboard] How to overwrite checkpoint of same global step?

If you resume training from a specific epoch (same global step) and run tensorboard, tensorboard (add_scalar) will end up plotting 2 points for that particular global step.

Example, I want trying to test whether changing the learning rate halfway through training will improve/worsen the accuracy:

Screenshot from 2021-04-05 17-51-45

The plots for same time step are plotted twice (I resumed from +15 epochs behind the latest epoch).

Search up the web, cannot find any commands that can ask tensorboard to just overwrite the previous saved checkpoint for the new one.