HAOCHENYE
August 19, 2023, 6:08pm
1
In this function
// nodes get suppressed in some situations, see "suppress gradient
// accumulation" below. Note that only variables which have `requires_grad =
// True` can have gradient accumulators.
if (const auto& gradient = self.grad_fn()) {
return Edge(gradient, self.output_nr());
} else {
return Edge(grad_accumulator(self), 0);
}
}
void set_gradient_edge(const Variable& self, Edge edge) {
auto* meta = materialize_autograd_meta(self);
meta->grad_fn_ = std::move(edge.function);
meta->output_nr_ = edge.input_nr;
// For views, make sure this new grad_fn_ is not overwritten unless it is
// necessary in the VariableHooks::grad_fn below. This logic is only relevant
// for custom autograd Functions for which multiple operations can happen on a
// given Tensor before its gradient edge is set when exiting the custom
// Function.
auto diff_view_meta = get_view_autograd_meta(self);
if (diff_view_meta && diff_view_meta->has_bw_view()) {
I’ve observed that the is_view_ attribute is being verified in this section of the code:
// NB: could return nullptr
TORCH_CHECK(
self.defined(), "cannot call get_autograd_meta() on undefined tensor");
return static_cast<AutogradMeta*>(
self.unsafeGetTensorImpl()->autograd_meta());
}
DifferentiableViewMeta* get_view_autograd_meta(const at::TensorBase& self) {
// NB: return nullptr if self is not a view
AutogradMeta* meta = get_autograd_meta(self);
if (meta && meta->is_view_) {
return static_cast<DifferentiableViewMeta*>(meta);
} else {
return nullptr;
}
}
} // namespace impl
using at::Tensor;
Furthermore, the diff_view_meta->has_bw_view() function is being checked in the subsequent code. My understanding is that both is_view_ and backward_info_ are initialized when the view series of functions is invoked. I’m curious about the circumstances under which is_view_ might be True, while backward_info_ remains uninitialized, and yet has_bw_view() returns False.