# Synchronize() is called, but I have never called that function

**URL:** <https://discuss.pytorch.org/t/synchronize-is-called-but-i-have-never-called-that-function/117133>\
**Category:** C++\
**Created:** [April 5, 2021, 4:20pm UTC](https://discuss.pytorch.org/t/synchronize-is-called-but-i-have-never-called-that-function/117133 "2021-04-05T16:20:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![111480](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/111480/32/35526_2.png) [@111480](https://discuss.pytorch.org/u/111480)\
**Post date:** [April 5, 2021, 4:20pm UTC](https://discuss.pytorch.org/t/synchronize-is-called-but-i-have-never-called-that-function/117133/1 "2021-04-05T16:20:03Z")

</div>

When I created a .densenet201pt from Torchvison using Torchscript and ran it on c++, I profiled it with nvprof and found that cudaStreamSyncronize() and cudaDeviceSyncronize() were called.  
But I didn’t explicitly call those functions.  
Where are these functions called?

 ![nvprof](https://discuss.pytorch.org/uploads/default/original/3X/6/f/6f30cbe7ddceb2537006b4fecb93a20262bcbeab.jpeg)

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [April 6, 2021, 8:20am UTC](https://discuss.pytorch.org/t/synchronize-is-called-but-i-have-never-called-that-function/117133/2 "2021-04-06T08:20:53Z")

</div>

These methods might be called internally, e.g. if you transfer device tensors to the CPU, as the GPU would need to be synchronized before the values can be read. [This section](https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html#avoid-unnecessary-cpu-gpu-synchronization) of the performance guide explains it in a bit more detail.