Hi,
recently I found that JIT does not create DifferentiableGraph nodes corresponding to Conv and BN operations.
Both ops are not registered in symbolic_script.h and considered to be “non-differentiable” in the sense that their gradients are always computed by Autograd engine, which loses the opportunity of making bigger DifferentiableGraphs.
What’s the reason behind such non-optimal behavior of JIT?