# Resetting cache in benchmark

**URL:** <https://discuss.pytorch.org/t/resetting-cache-in-benchmark/197218>\
**Category:** torch.\_dynamo\
**Created:** [February 16, 2024, 5:01pm UTC](https://discuss.pytorch.org/t/resetting-cache-in-benchmark/197218 "2024-02-16T17:01:28Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![AidanGoldfarb](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aidangoldfarb/32/67581_2.png) [@AidanGoldfarb](https://discuss.pytorch.org/u/AidanGoldfarb)\
**Post date:** [February 16, 2024, 5:01pm UTC](https://discuss.pytorch.org/t/resetting-cache-in-benchmark/197218/1 "2024-02-16T17:01:28Z")

</div>

I am attempting to benchmark some things with torch.compile, including the overhead of compilation in different modes. I am calling `dynamo.reset()` before each call to torch.compile, however this does not seem to be clearing the cache.

Below is the code I am executing, `mul` is a simple function, `timeit` simply passes arguments to its argument and calls the function, timing it . Whichever mode is called first has overhead of ~1 sec, the following three have orders of magnitude less.

Please let me know if I am clearing the cache incorrectly, thank you.

```auto
dynamo.reset()
def_cmptm, mul_def = timeit( torch.compile, args=mul, kwargs={"mode": "default",} )

dynamo.reset()
max_cmptm, mul_max = timeit( torch.compile, args=mul, kwargs={"mode": "max-autotune"} )

dynamo.reset()
red_cmptm, mul_red = timeit( torch.compile, args=mul, kwargs={"mode": "reduce-overhead"} )

dynamo.reset()
max_nc_cmptm, mul_max_nc = timeit( torch.compile, args=mul, kwargs={"mode": "max-autotune-no-cudagraphs"} )

```

---

<div class="post-metadata">

**Author:** ![AidanGoldfarb](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aidangoldfarb/32/67581_2.png) [@AidanGoldfarb](https://discuss.pytorch.org/u/AidanGoldfarb)\
**Post date:** [February 19, 2024, 3:56pm UTC](https://discuss.pytorch.org/t/resetting-cache-in-benchmark/197218/2 "2024-02-19T15:56:58Z")

</div>

It appears the first run is simply the compiler warming up. Inserting a dummy compilation solves this. Interestingly, `dynamo.reset()` still has no effect on compile times. This surprises me, as I would think some information about a previously compiled function would be cached.

---

<div class="post-metadata">

**Author:** ![bhack](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bhack/32/62677_2.png) [@bhack](https://discuss.pytorch.org/u/bhack)\
**Post date:** [March 9, 2024, 1:20am UTC](https://discuss.pytorch.org/t/resetting-cache-in-benchmark/197218/3 "2024-03-09T01:20:40Z")

</div>

But it seems that the team is using `dynamo.reset()` to clear the cache:

> <https://github.com/pytorch/FBGEMM/pull/1992>
>
> Summary: Preparing for reducing cache size work.
> 
> Differential Revision: D489405…44

---

<div class="post-metadata">

**Author:** ![AidanGoldfarb](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aidangoldfarb/32/67581_2.png) [@AidanGoldfarb](https://discuss.pytorch.org/u/AidanGoldfarb)\
**Post date:** [March 10, 2024, 4:22pm UTC](https://discuss.pytorch.org/t/resetting-cache-in-benchmark/197218/4 "2024-03-10T16:22:16Z")

</div>

Perhaps; perhaps it will be documented someday :-). I figured it may have to do with the recompilation cache.

---

<div class="post-metadata">

**Author:** ![nd21](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nd21/32/74879_2.png) [@nd21](https://discuss.pytorch.org/u/nd21)\
**Post date:** [April 23, 2025, 3:08am UTC](https://discuss.pytorch.org/t/resetting-cache-in-benchmark/197218/5 "2025-04-23T03:08:52Z")

</div>

I am facing a similar issue. My model’s peak memory usage for the first run is higher than the second run. And second run onwards the peak memory is somewhat the same. I am wondering what may be the reason for this?