Loss oscillating and not minimizing

That’s strange as the target value does not need to be attached to a computation graph and can be static. Are you sure this was causing the issue?