Get gradient wrt to input

Figured it out here: Strange behavior of Variable.cuda() and Variable.grad