# Efficiency of predictions using GPU vs CPU

**URL:** <https://discuss.pytorch.org/t/efficiency-of-predictions-using-gpu-vs-cpu/131945>\
**Category:** Uncategorized\
**Created:** [September 14, 2021, 2:38pm UTC](https://discuss.pytorch.org/t/efficiency-of-predictions-using-gpu-vs-cpu/131945 "2021-09-14T14:38:28Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![andreys42](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/andreys42/32/35546_2.png) [@andreys42](https://discuss.pytorch.org/u/andreys42)\
**Post date:** [September 14, 2021, 2:38pm UTC](https://discuss.pytorch.org/t/efficiency-of-predictions-using-gpu-vs-cpu/131945/1 "2021-09-14T14:38:28Z")

</div>

I have Inference routine consist of prediction using GPU:

```auto
for i in data: 
  pred_subcube= model(data[i].cuda())

```

Where data[i] is item (Tensor of shape 5x1x128x128x128, where 5 is batchsize, 1 channels, and 128x128x128 is shape of my 3D image) of DataLoader class, model is my U-Net like 3D CNN model.  
And then I would like to set ‘pred’ as part of numpy array :

`resulting_cube[i] = pred_subcube.detach().cpu().numpy()`

Is it efficient way in my case to predict using GPU and then sent it to CPU to fill my “resulting\_cube” array? Isn’t it more efficient to predict using CPU to not waste time to detach.

---

<div class="post-metadata">

**Author:** ![gphilip](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/858c86/32.png) [@gphilip](https://discuss.pytorch.org/u/gphilip)\
**Post date:** [September 14, 2021, 2:50pm UTC](https://discuss.pytorch.org/t/efficiency-of-predictions-using-gpu-vs-cpu/131945/2 "2021-09-14T14:50:31Z")

</div>

I am not an expert on this so take it for what it is worth, but my hunch is that if the model is bigger than some small limit then the time for detaching the results will be negligible compared to the time for computing them on the CPU.

---

<div class="post-metadata">

**Author:** ![andreys42](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/andreys42/32/35546_2.png) [@andreys42](https://discuss.pytorch.org/u/andreys42)\
**Post date:** [September 14, 2021, 3:03pm UTC](https://discuss.pytorch.org/t/efficiency-of-predictions-using-gpu-vs-cpu/131945/3 "2021-09-14T15:03:53Z")

</div>

that’s exactly the idea of my question) how deep have to be my model to use it with GPU and then detach

---

<div class="post-metadata">

**Author:** ![gphilip](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/858c86/32.png) [@gphilip](https://discuss.pytorch.org/u/gphilip)\
**Post date:** [September 14, 2021, 3:19pm UTC](https://discuss.pytorch.org/t/efficiency-of-predictions-using-gpu-vs-cpu/131945/4 "2021-09-14T15:19:43Z")

</div>

The one sure way to figure this out is to run some timed tests in either mode and compare. I am sorry, but I don’t know enough to make a good guess.

---

<div class="post-metadata">

**Author:** ![andreys42](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/andreys42/32/35546_2.png) [@andreys42](https://discuss.pytorch.org/u/andreys42)\
**Post date:** [September 15, 2021, 5:25am UTC](https://discuss.pytorch.org/t/efficiency-of-predictions-using-gpu-vs-cpu/131945/5 "2021-09-15T05:25:52Z")

</div>

yeah, thats nice suggestion, but I thought there are some thoughts from experts who already tested thouse routines