# Efficiently computing gradients for each example in a large dataset

**URL:** <https://discuss.pytorch.org/t/efficiently-computing-gradients-for-each-example-in-a-large-dataset/52689>\
**Category:** autograd\
**Created:** [August 6, 2019, 3:42pm UTC](https://discuss.pytorch.org/t/efficiently-computing-gradients-for-each-example-in-a-large-dataset/52689 "2019-08-06T15:42:15Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![alirs](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/f475e1/32.png) [@alirs](https://discuss.pytorch.org/u/alirs)\
**Post date:** [August 6, 2019, 3:42pm UTC](https://discuss.pytorch.org/t/efficiently-computing-gradients-for-each-example-in-a-large-dataset/52689/1 "2019-08-06T15:42:15Z")

</div>

Given a trained model (M), I’m interested in computing the utility of new (unseen) examples in a pool (for an active learning task). For this, I need to compute the magnitude of the gradient when M is trained on each new example. In code, it is something like:

```auto
losses, grads = [], []
for i in range(X_pool.shape[0]):
    pred = model(X_pool[i:i+1])
    loss = loss_func(pred, y_pool[i:i+1])

    model.zero_grad()
    loss.backward()

    losses.append(loss)
    grads.append(layer.weight.grad.norm())

```

However, this is quite slow when there is a large pool of examples, especially since this will be an inner loop in my scenario. How can this code be improved for efficiency?

I appreciate any suggestions or pointers. Thanks!

---

<div class="post-metadata">

**Author:** ![Vania\_Todorova](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vania_todorova/32/13798_2.png) [@Vania\_Todorova](https://discuss.pytorch.org/u/Vania_Todorova)\
**Post date:** [August 7, 2019, 7:23pm UTC](https://discuss.pytorch.org/t/efficiently-computing-gradients-for-each-example-in-a-large-dataset/52689/2 "2019-08-07T19:23:56Z")

</div>

did you ever figure out nything ? i m doing a linear regression with 1 layer but its diverging … any advice?