[jit] Shouldn't I use torch.randn_like when using torch.jit.tracing?

jit
1

When I implement VAE in pytorch, I need to use torch.randn_like() like this:

"""
:param mean: torch.Tensor
"""
eps = torch.randn_like(mean)

However, when I use torch.jit.tracing to convert my VAE model into the graph representation, I face this warning message:

/*/lib/python3.6/site-packages/torch/jit/__init__.py:644: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
Not within tolerance rtol=1e-05 atol=1e-05 at input[8, 15] (-0.3760705292224884 vs. 5.01010274887085) and 199 other locations (100.00%)
  _check_trace([example_inputs], func, executor_options, module, check_tolerance, _force_outplace)
/*/lib/python3.6/site-packages/torch/jit/__init__.py:644: TracerWarning: Trace had nondeterministic nodes. Nodes:
	%eps : Float(10, 13) = aten::randn_like(%mean, %20, %21, %22), scope: Decoder
This may cause errors in trace checking. To disable trace checking, pass check_trace=False to torch.jit.trace()
  _check_trace([example_inputs], func, executor_options, module, check_tolerance, _force_outplace)
/*/lib/python3.6/site-packages/torch/jit/__init__.py:644: TracerWarning: Output nr 1. of the traced function does not match the corresponding output of the Python function. Detailed error:
Not within tolerance rtol=1e-05 atol=1e-05 at input[3, 8] (-1.5253040790557861 vs. 3.154987096786499) and 129 other locations (100.00%)
  _check_trace([example_inputs], func, executor_options, module, check_tolerance, _force_outplace)

What should I do?
What is trace checking?
What will passing check_trace=False to torch.jit.tracing() cause?

2

Trace checking compares the results of the traced function to the actual function to make sure they are the same. It’s just a sanity check, but non-deterministic ops like randn_like() will cause trace checking to fail. This failure is expected, and if you really want to use non-determinism in your model then you can disable the trace checking as the warning says.

3

@Michael_Suo
Thank you for your reply!
I understand what happened.
I’ll try it again with check_trace=False.