“torch::NoGradGuard no_grad;” is used to prevent gradient calculation from building up memory during inference.
I am running a large model where the increase of memory is such that the memory crash unless NoGradGuard is used. I built two docker images on the same docker host server where both images are identical (same install, same make file, same cpp files…) except for the container os, i.e. one image was built on ubuntu 16.04, the other on ubuntu 18.04. Memory runs wild in the ubuntu 18.04 container while it is under control in ubuntu 16.04 container. Analysis showed that “torch::NoGradGuard no_grad;” prevents any memory increase in 16.04 but does not has any impact in 18.04.
Any explanation would be welcome?
Could you provide a minimal code snippet to reproduce this issue, please?
Sorry. I have not managed yet to produce simple code to reproduce the issue as it involves a lot of install manual work at present. I will keep trying.