Gradient accumulation gives a worse result then without it

I have an object detection model.
When I use gradient accumulation I get a worse loss/accuracy then without it, systematically.