We trace out the backwards pass “ahead of time” (hence AOTAutograd). Although we are not actually able to execute the backwards pass until later (when we have gradOut), we know the shape of gradOut, and thus can trace it out “ahead of time”.
You can read more about it here: AOTAutograd - Google Slides