# Why is torch.jit.script slower? **URL:** \ **Category:** jit\ **Created:** [May 3, 2021, 4:32pm UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131 "2021-05-03T16:32:02Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![111480](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/111480/32/35526_2.png) [@111480](https://discuss.pytorch.org/u/111480)\ **Post date:** [May 3, 2021, 4:32pm UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131/1 "2021-05-03T16:32:02Z")
I compared the performance(speed) of Torchvision’s Squeezenet original model with `torch.jit.script(model)` which I expected to speed up because Torchscript was asynchronous, but the original model was faster. What’s the reason? Which one did I miss? ![image](https://discuss.pytorch.org/uploads/default/original/3X/4/1/41722d8ff4bab290f121fa326bc6a86433b64bbd.png) scripted model : 0.2353s original model : 0.1644s ---
**Author:** ![googlebot](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/ba8739/32.png) [@googlebot](https://discuss.pytorch.org/u/googlebot)\ **Post date:** [May 4, 2021, 12:40pm UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131/2 "2021-05-04T12:40:48Z")
it is not asynchronous (beyond cuda kernel launches, which is not related to jit), just python-less execution mode with optimizations one thing I’ve seen, is that some jitted operations incorrectly enable requires\_grad but simpler explanation is that you’re not measuring it right - time the THIRD call of compiled model (actually, from your screenshot it seems you’re compiling twice [i.e. two model objects], which is also incorrect). reason is that profiling mode executor creates optimized bytecode on second call. ---
**Author:** ![111480](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/111480/32/35526_2.png) [@111480](https://discuss.pytorch.org/u/111480)\ **Post date:** [May 4, 2021, 2:42pm UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131/3 "2021-05-04T14:42:56Z")
Thanks for replying. The time was measured separately between `torch.jit.script` and the original model. like this, ```auto torch.jit.script(models.squeezenet1_0(pretrained=True).cuda().eval()) torch.jit.script(models.squeezenet1_0(pretrained=True).cuda().eval()) #models.squeezenet1_0(pretrained=True).cuda().eval() #models.squeezenet1_0(pretrained=True).cuda().eval() ``` Then result is ![image](https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee34ec665fdfee6138fb5a5d11cfacf39f13c89c.png) ```auto #torch.jit.script(models.squeezenet1_0(pretrained=True).cuda().eval()) #torch.jit.script(models.squeezenet1_0(pretrained=True).cuda().eval()) models.squeezenet1_0(pretrained=True).cuda().eval() models.squeezenet1_0(pretrained=True).cuda().eval() ``` ![image](https://discuss.pytorch.org/uploads/default/original/3X/6/d/6db01c642701f7c4be723ce92456dff790ca3ba6.png) Why exactly is Torchscript slower than pytorch? ---
**Author:** ![googlebot](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/ba8739/32.png) [@googlebot](https://discuss.pytorch.org/u/googlebot)\ **Post date:** [May 4, 2021, 2:59pm UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131/4 "2021-05-04T14:59:04Z")
as I said, do something like: net = jit.script(models.squeezenet1\_0(pretrained=True).cuda().eval()) net(x); net(x) and only then measure time (e.g. %timeit net(x)) ---
**Author:** ![111480](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/111480/32/35526_2.png) [@111480](https://discuss.pytorch.org/u/111480)\ **Post date:** [May 4, 2021, 3:24pm UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131/5 "2021-05-04T15:24:03Z")
As you say, Torchscript is faster. Is this the result of different ways of optimizing it? I want a detailed explanation. ---
**Author:** ![googlebot](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/ba8739/32.png) [@googlebot](https://discuss.pytorch.org/u/googlebot)\ **Post date:** [May 4, 2021, 4:20pm UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131/6 "2021-05-04T16:20:30Z")
In a nutshell, 1. “compilation” analyzes whole functions, with knowledge about variable types - some optimizations are done at this level (e.g. dead code elimination) 2. python bytecode interpreter is not used to execute generated code - more specialized executor for statically typed code supposedly works faster 3. fusion optimizations further compile specialized cuda kernels, so e.g. a.mul(b).add(c) is computed in one go 4. some patterns have specialized optimizations, e.g. conv+batchnorm ---
**Author:** ![vince62s](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vince62s/32/3661_2.png) [@vince62s](https://discuss.pytorch.org/u/vince62s)\ **Post date:** [October 12, 2022, 6:50pm UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131/7 "2022-10-12T18:50:58Z")
Hi I am hijacking this thread for a similar question. I ran the code of this link: [pytorch-jit.ipynb · GitHub](https://gist.github.com/ResidentMario/1a4f6473828048990e26d12d58d7a227) exact same time with and without, not seeing the perf difference as per the link. any clue? (running on pytorch 1.12.1 cuda 11.3) ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [October 13, 2022, 8:12am UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131/8 "2022-10-13T08:12:11Z")
Your notebook seems to show a ~2x speedup so I’m unsure how to interpret your post. Did you see any other results when properly profiling with syncs? ---
**Author:** ![vince62s](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vince62s/32/3661_2.png) [@vince62s](https://discuss.pytorch.org/u/vince62s)\ **Post date:** [October 13, 2022, 8:43am UTC](https://discuss.pytorch.org/t/why-is-torch-jit-script-slower/120131/9 "2022-10-13T08:43:16Z")
this is not “my” notebook. this is someone else’s noebook that I tried on my machine, hence not replicating the speedup.