Hi,
The autograd engine only records the “basic” operations done on Tensors. So any logic that you add around it will not impact the whole autograd engine.
The overhead you will have is the overhead of running more python code. Depending on the size of your net, that can be completely negigeable (big cnns) or significant (very very small nets). But the autograd engine will not be impacted by these changes.