# Checking Whether Network is Dead, Pytorch?

**URL:** <https://discuss.pytorch.org/t/checking-whether-network-is-dead-pytorch/21266>\
**Category:** Uncategorized\
**Created:** [July 17, 2018, 9:07am UTC](https://discuss.pytorch.org/t/checking-whether-network-is-dead-pytorch/21266 "2018-07-17T09:07:32Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [July 17, 2018, 9:36am UTC](https://discuss.pytorch.org/t/checking-whether-network-is-dead-pytorch/21266/2 "2018-07-17T09:36:36Z")

</div>

By dead ReLUs you mean a negative input and thus a zero result?  
If so, you could count the zeros in the output activation.  
Here is a toy example for a simple model:

```python
model = nn.Sequential(
    nn.Linear(10, 10),
    nn.ReLU()
)

x = torch.randn(10, 10)
output = model(x)
(output == 0).sum(1).float() / output.size(1)

```

Fore more complex models, you could use a forward hook to get the intermediate activations.

---

_[View the full topic](https://discuss.pytorch.org/t/checking-whether-network-is-dead-pytorch/21266)._