# Dataloaders and Cuda management

**URL:** <https://discuss.pytorch.org/t/dataloaders-and-cuda-management/53671>\
**Category:** Uncategorized\
**Created:** [August 18, 2019, 6:16pm UTC](https://discuss.pytorch.org/t/dataloaders-and-cuda-management/53671 "2019-08-18T18:16:00Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 19, 2019, 9:57am UTC](https://discuss.pytorch.org/t/dataloaders-and-cuda-management/53671/4 "2019-08-19T09:57:38Z")

</div>

> [@ilyes](#):
>
> Which is weird! it’s just a one element tensor! Can you please explain why this huge difference between cache and allocated memory.

If I’m not mistaken, `cudaMalloc` rounds to 2MB blocks in newer GPUs, which is exactly `(2097152 / 1024**2)`, which would explain the minimal cache size.

> [@ilyes](#):
>
> One last question: Now as everything is working well, I re-checked my GPU performance and found that utilization is between `3% and 6%` Even when I increase the batchsize to 256 the range is `5% to 20%` . Why the GPU is not fully used ?

You might have a data loading bottleneck.  
Profile your data loading overhead using the code from the [ImageNet example](https://github.com/pytorch/examples/blob/6f62fcd361868406a95d71af9349f9a3d03a2c52/imagenet/main.py#L275). If you see some overhead in loading the data, check [this post](https://discuss.pytorch.org/t/how-to-prefetch-data-when-processing-with-gpu/548/19) from @rwightman, where he explains some possible workarounds.

---

_[View the full topic](https://discuss.pytorch.org/t/dataloaders-and-cuda-management/53671)._