# Gradient w.r.t each sample

**URL:** <https://discuss.pytorch.org/t/gradient-w-r-t-each-sample/1433>\
**Category:** Uncategorized\
**Created:** [March 28, 2017, 2:10pm UTC](https://discuss.pytorch.org/t/gradient-w-r-t-each-sample/1433 "2017-03-28T14:10:39Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![smth](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/smth/32/13_2.png) [@smth](https://discuss.pytorch.org/u/smth)\
**Post date:** [March 28, 2017, 3:12pm UTC](https://discuss.pytorch.org/t/gradient-w-r-t-each-sample/1433/2 "2017-03-28T15:12:00Z")

</div>

there is no way to access the gradients wrt weight for each individual sample. Gradients wrt weights in pytorch are always accumulated (even over the mini-batch). If you want gradients wrt each sample, you will have to run each sample individually through the network. This is because both the THNN backend and CuDNN dont support individual sample gradients wrt weight.

---

_[View the full topic](https://discuss.pytorch.org/t/gradient-w-r-t-each-sample/1433)._