# torch.compile **URL:** https://discuss.pytorch.org/c/compile/41.md?page=1 [Latest](https://discuss.pytorch.org/latest.md) · [Categories](https://discuss.pytorch.org/categories.md) **Page:** 2 --- ## [Torch.compile - getting \`nans\` with the latest version of torch](https://discuss.pytorch.org/t/torch-compile-getting-nans-with-the-latest-version-of-torch/223989)
**Author:** [@vgoklani](https://discuss.pytorch.org/u/vgoklani)\ **Replies:** 3\ **Last updated:** [January 6, 2026, 7:39pm UTC](https://discuss.pytorch.org/t/torch-compile-getting-nans-with-the-latest-version-of-torch/223989 "2026-01-06T19:39:06Z")
When running our training code on 4x RTX Blackwell 6000 cards using: nvcr.io/nvidia/pytorch:25.09-py3, our training is stable, but when upgrading to nvcr.io/nvidia/pytorch:25.10-py3 our loss quickly goes to NaN using the… --- ## [Torch.compile results in loss nan while it converges without torch.compile on mac cpu and mps](https://discuss.pytorch.org/t/torch-compile-results-in-loss-nan-while-it-converges-without-torch-compile-on-mac-cpu-and-mps/224290)
**Author:** [@Arpan\_Kumar\_Nandi](https://discuss.pytorch.org/u/Arpan_Kumar_Nandi)\ **Replies:** 0\ **Last updated:** [January 6, 2026, 3:02am UTC](https://discuss.pytorch.org/t/torch-compile-results-in-loss-nan-while-it-converges-without-torch-compile-on-mac-cpu-and-mps/224290 "2026-01-06T03:02:35Z")
import os import math import time import inspect from dataclasses import dataclass import torch import torch.nn as nn from torch.nn import functional as F # --------------------------------------------------------------… --- ## [How to verify that default\_decompositions successfully reduce operators to the Core ATen IR set?](https://discuss.pytorch.org/t/how-to-verify-that-default-decompositions-successfully-reduce-operators-to-the-core-aten-ir-set/224262)
**Author:** [@Tongkai](https://discuss.pytorch.org/u/Tongkai)\ **Replies:** 0\ **Last updated:** [December 30, 2025, 5:11pm UTC](https://discuss.pytorch.org/t/how-to-verify-that-default-decompositions-successfully-reduce-operators-to-the-core-aten-ir-set/224262 "2025-12-30T17:11:01Z")
Is there a way to test if all ops in default\_decompositions can be fully decomposed into the Core ATen IR (~180 ops) using ep.run\_decompositions, as specified in the Export IR documentation (https://docs.pytorch.org/docs… --- ## [NaNs with torch.compile (inductor, mode="reduce-overhead"), OOM with backend="cudagraphs"](https://discuss.pytorch.org/t/nans-with-torch-compile-inductor-mode-reduce-overhead-oom-with-backend-cudagraphs/224248)
**Author:** [@XiDianZuoYun](https://discuss.pytorch.org/u/XiDianZuoYun)\ **Replies:** 0\ **Last updated:** [December 23, 2025, 7:58am UTC](https://discuss.pytorch.org/t/nans-with-torch-compile-inductor-mode-reduce-overhead-oom-with-backend-cudagraphs/224248 "2025-12-23T07:58:24Z")
Hi, I’m running into two related issues when trying to speed up a very large model with torch.compile and CUDA Graphs: NaNs when using backend="inductor", mode="reduce-overhead" OOM when using backend="cudagraphs" … --- ## [How to save torch.compile so we don't need to re-compile](https://discuss.pytorch.org/t/how-to-save-torch-compile-so-we-dont-need-to-re-compile/215726)
**Author:** [@vgoklani](https://discuss.pytorch.org/u/vgoklani)\ **Replies:** 2\ **Last updated:** [December 23, 2025, 2:19am UTC](https://discuss.pytorch.org/t/how-to-save-torch-compile-so-we-dont-need-to-re-compile/215726 "2025-12-23T02:19:37Z")
Is it possible to save the torch.compile state so we don’t need to re-compile each time when running the same exact model? It takes over 15 minutes to compile, and it’s presumably doing the same exact work each time. We … --- ## [How to load state\_dict if saved as torch.save(model.state\_dict(), fs)](https://discuss.pytorch.org/t/how-to-load-state-dict-if-saved-as-torch-save-model-state-dict-fs/224055)
**Author:** [@lostalot](https://discuss.pytorch.org/u/lostalot)\ **Replies:** 12\ **Last updated:** [December 21, 2025, 10:02pm UTC](https://discuss.pytorch.org/t/how-to-load-state-dict-if-saved-as-torch-save-model-state-dict-fs/224055 "2025-12-21T22:02:05Z")
I am testing torch.compile() with basic cifar10 and resnet, but cannot seem to figure out why I cannot load the saved state\_dict. I am sure it is something fundamental that I am missing. Might someone please be kind e… --- ## [Issue when using aot\_module with torch.autograd](https://discuss.pytorch.org/t/issue-when-using-aot-module-with-torch-autograd/224236)
**Author:** [@nahso](https://discuss.pytorch.org/u/nahso)\ **Replies:** 0\ **Last updated:** [December 19, 2025, 5:54am UTC](https://discuss.pytorch.org/t/issue-when-using-aot-module-with-torch-autograd/224236 "2025-12-19T05:54:26Z")
I want to dump the fx graph when executing the torch.autograd(y, W) of a linear module y = b + x \* W^T. But the fx graph in bw\_compiler returns a graph of computing every gradients, I only want the code of computing grad… --- ## [Let HOF (torch.cond, torch.while\_loop) skip compilation of input functions if needed](https://discuss.pytorch.org/t/let-hof-torch-cond-torch-while-loop-skip-compilation-of-input-functions-if-needed/224208)
**Author:** [@lucasegidio1](https://discuss.pytorch.org/u/lucasegidio1)\ **Replies:** 0\ **Last updated:** [December 17, 2025, 11:51am UTC](https://discuss.pytorch.org/t/let-hof-torch-cond-torch-while-loop-skip-compilation-of-input-functions-if-needed/224208 "2025-12-17T11:51:05Z")
Hi all! Great work with the HOFs :slight\_smile: For debugging, it would be nice if there was a switch I could flip to let torch.while\_loop( cond\_fn, body\_fn, carried\_inputs, ) behave exactly like the pytho… --- ## [Autotune cache key for matrix multiplication](https://discuss.pytorch.org/t/autotune-cache-key-for-matrix-multiplication/224174)
**Author:** [@vadimkantorov](https://discuss.pytorch.org/u/vadimkantorov)\ **Replies:** 0\ **Last updated:** [December 11, 2025, 7:15pm UTC](https://discuss.pytorch.org/t/autotune-cache-key-for-matrix-multiplication/224174 "2025-12-11T19:15:00Z")
I’m having autotune spending ages on sizes like \`AUTOTUNE mm(8192x2304, 2304x256)\` (where 2304 is something like seqlen). Does AUTOTUNE mm round up to multiple of e.g. 32 elements and caches results under the appropriat… --- ## [Torch compile cache sharing across different GPUs](https://discuss.pytorch.org/t/torch-compile-cache-sharing-across-different-gpus/223964)
**Author:** [@aerialmyth](https://discuss.pytorch.org/u/aerialmyth)\ **Replies:** 3\ **Last updated:** [December 10, 2025, 9:14pm UTC](https://discuss.pytorch.org/t/torch-compile-cache-sharing-across-different-gpus/223964 "2025-12-10T21:14:19Z")
Hi, I am trying to use torch.compile cache to compile the model on one GPU and then reuse the compiled model across different GPUs connected to the same node. For instance, I have compiled a model on GPU0 (rank0) on a 8… --- ## [Coordinate\_descent\_tuning errors out with torch.AcceleratorError: CUDA error: invalid argument](https://discuss.pytorch.org/t/coordinate-descent-tuning-errors-out-with-torch-acceleratorerror-cuda-error-invalid-argument/223000)
**Author:** [@sytelus](https://discuss.pytorch.org/u/sytelus)\ **Replies:** 12\ **Last updated:** [December 5, 2025, 3:15pm UTC](https://discuss.pytorch.org/t/coordinate-descent-tuning-errors-out-with-torch-acceleratorerror-cuda-error-invalid-argument/223000 "2025-12-05T15:15:26Z")
It seems coordinate\_descent\_tuning bombs out on rather trivial NanoGPT 117M model: Error: torch.AcceleratorError: CUDA error: invalid argument Docker image: nvcr.io/nvidia/pytorch:25.08-py3 Pytorch version: 2.8.0a0+34… --- ## [\_GLIBCXX\_USE\_CXX11\_ABI incompatibility](https://discuss.pytorch.org/t/glibcxx-use-cxx11-abi-incompatibility/224086)
**Author:** [@Himalayjor](https://discuss.pytorch.org/u/Himalayjor)\ **Replies:** 2\ **Last updated:** [December 3, 2025, 7:13pm UTC](https://discuss.pytorch.org/t/glibcxx-use-cxx11-abi-incompatibility/224086 "2025-12-03T19:13:07Z")
can we modify “pip installed torch” to use \_GLIBCXX\_USE\_CXX11\_ABI=1 for torch compilation ? It seems all pip installed torch comes with \_GLIBCXX\_USE\_CXX11\_ABI=0The reason is torch compiled model using AOT (Ahead of time… --- ## [Compile option for older CPUs?](https://discuss.pytorch.org/t/compile-option-for-older-cpus/224052)
**Author:** [@QuantumSphinx](https://discuss.pytorch.org/u/QuantumSphinx)\ **Replies:** 0\ **Last updated:** [November 28, 2025, 2:06pm UTC](https://discuss.pytorch.org/t/compile-option-for-older-cpus/224052 "2025-11-28T14:06:34Z")
I was reading the documentation on GitHub and saw there’s some specific options for CPU-only builds, which I have followed. The issue is I’m working with some older home lab hardware that is equipped with CPUs that only … --- ## [Proposing to add generator param to rand\*\_like functions](https://discuss.pytorch.org/t/proposing-to-add-generator-param-to-rand-like-functions/224017)
**Author:** [@alexander-t-ho](https://discuss.pytorch.org/u/alexander-t-ho)\ **Replies:** 1\ **Last updated:** [November 26, 2025, 12:32am UTC](https://discuss.pytorch.org/t/proposing-to-add-generator-param-to-rand-like-functions/224017 "2025-11-26T00:32:16Z")
Hi! I’m interested in contributing to PyTorch by implementing the generator parameter for randn\_like/rand\_like/randint\_like. Before I start, I wanted to confirm: 1. Is this still needed? 2. Any preferred implementation a… --- ## [Torch.export with custom\_op](https://discuss.pytorch.org/t/torch-export-with-custom-op/224011)
**Author:** [@JuanFMontesinos](https://discuss.pytorch.org/u/JuanFMontesinos)\ **Replies:** 0\ **Last updated:** [November 24, 2025, 1:30pm UTC](https://discuss.pytorch.org/t/torch-export-with-custom-op/224011 "2025-11-24T13:30:02Z")
I was writting CUDA kernels using cute DSL. My hope defining those kernels as pytorch custom ops and levarage torch.export to capture them (as cuda graphs maybe?) Is it really possible to use torch.export with custom op… --- ## [InductorError in autogenerated code in torchinductor\_cache - triton, rocm](https://discuss.pytorch.org/t/inductorerror-in-autogenerated-code-in-torchinductor-cache-triton-rocm/223995)
**Author:** [@pya](https://discuss.pytorch.org/u/pya)\ **Replies:** 0\ **Last updated:** [November 22, 2025, 2:17pm UTC](https://discuss.pytorch.org/t/inductorerror-in-autogenerated-code-in-torchinductor-cache-triton-rocm/223995 "2025-11-22T14:17:23Z")
Hi, I am trying to run the TRM model on a machine with rocm. The following code is auto-generated by torch.compile. import triton import triton.language as tl from triton.compiler.compiler import AttrsDescriptor from … --- ## [Choice of torch.compile vs. triton](https://discuss.pytorch.org/t/choice-of-torch-compile-vs-triton/195604)
**Author:** [@CDhere](https://discuss.pytorch.org/u/CDhere)\ **Replies:** 8\ **Last updated:** [November 18, 2025, 6:28am UTC](https://discuss.pytorch.org/t/choice-of-torch-compile-vs-triton/195604 "2025-11-18T06:28:03Z")
Hi, I’m new to torch.compile and the doc says torch.compile is the latest method to speed up your PyTorch code! torch.compile makes PyTorch code run faster by JIT-compiling PyTorch code into optimized kernels, all whil… --- ## [Any specific reason for chosing the default batch sizes for the torch dynamo benchmark suite](https://discuss.pytorch.org/t/any-specific-reason-for-chosing-the-default-batch-sizes-for-the-torch-dynamo-benchmark-suite/201348)
**Author:** [@Abhishek\_Ghosh](https://discuss.pytorch.org/u/Abhishek_Ghosh)\ **Replies:** 3\ **Last updated:** [November 18, 2025, 6:25am UTC](https://discuss.pytorch.org/t/any-specific-reason-for-chosing-the-default-batch-sizes-for-the-torch-dynamo-benchmark-suite/201348 "2025-11-18T06:25:34Z")
I was trying to test the TorchDynamo Benchmark Suite. In the model list files: torchbench\_models\_list huggingface\_models\_list timm\_models\_list Or in the yml files: torchbench.ymlbatch\_size: training: demucs: … --- ## [Why torch==2.9 compile qwen3 model with block ptr will crash?](https://discuss.pytorch.org/t/why-torch-2-9-compile-qwen3-model-with-block-ptr-will-crash/223941)
**Author:** [@TracyMac1](https://discuss.pytorch.org/u/TracyMac1)\ **Replies:** 0\ **Last updated:** [November 14, 2025, 7:52am UTC](https://discuss.pytorch.org/t/why-torch-2-9-compile-qwen3-model-with-block-ptr-will-crash/223941 "2025-11-14T07:52:50Z")
torch==2.8 compile with “torch.\_inductor.config.triton.use\_block\_ptr = True“ is ok, 2.9 torch will crash as shown in the figure --- ## [Performance bottleneck with autocast in AOT Inductor export: repeated weight casting due to disabled cache](https://discuss.pytorch.org/t/performance-bottleneck-with-autocast-in-aot-inductor-export-repeated-weight-casting-due-to-disabled-cache/223930)
**Author:** [@shykoe](https://discuss.pytorch.org/u/shykoe)\ **Replies:** 0\ **Last updated:** [November 11, 2025, 4:31pm UTC](https://discuss.pytorch.org/t/performance-bottleneck-with-autocast-in-aot-inductor-export-repeated-weight-casting-due-to-disabled-cache/223930 "2025-11-11T16:31:23Z")
Problem Description When I export my model with AOT Inductor, I notice that weights are being repeatedly cast to the target precision (fp16/bf16) on every forward pass, which creates a significant performance bottleneck. … --- ## [Is it possible to combine compiled vectorized function without recompiling already compiled part?](https://discuss.pytorch.org/t/is-it-possible-to-combine-compiled-vectorized-function-without-recompiling-already-compiled-part/223912)
**Author:** [@Jiace\_Sun](https://discuss.pytorch.org/u/Jiace_Sun)\ **Replies:** 0\ **Last updated:** [November 7, 2025, 9:49pm UTC](https://discuss.pytorch.org/t/is-it-possible-to-combine-compiled-vectorized-function-without-recompiling-already-compiled-part/223912 "2025-11-07T21:49:24Z")
My rough understanding of the fusing of torch.compile is to compile the following code def func(x): y = f1(x) z = f2(y) return z to a C++ code roughly as follows: for(int i=0;i\ **Author:** [@Jinhwan](https://discuss.pytorch.org/u/Jinhwan)\ **Replies:** 0\ **Last updated:** [November 6, 2025, 8:52am UTC](https://discuss.pytorch.org/t/how-do-i-know-which-placeholder-op-is-an-input/223824 "2025-11-06T08:52:50Z") Hello, I’m making custom torch.compile backend for our hardware and have a question about torch.fx.Graph. First, our chip is hard to implement dynamic-shaped tensors (memory map for tensors should be planned before run … --- ## [What requirements do I need to use Torch.compile?](https://discuss.pytorch.org/t/what-requirements-do-i-need-to-use-torch-compile/223804)
**Author:** [@Emad\_Younan](https://discuss.pytorch.org/u/Emad_Younan)\ **Replies:** 0\ **Last updated:** [November 3, 2025, 10:28pm UTC](https://discuss.pytorch.org/t/what-requirements-do-i-need-to-use-torch-compile/223804 "2025-11-03T22:28:50Z")
Hi, everyone. I’m using Python 3.13.9. With PyTorch 2.9 and CUDA 13.0, my GPU is an RTX 380 12 GB. from this link pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu130 But I can’t use it;… --- ## [Torch.export + fx.GraphModule: Unable to lift constants (parameters/buffers) to placeholders for memory sharing across modules](https://discuss.pytorch.org/t/torch-export-fx-graphmodule-unable-to-lift-constants-parameters-buffers-to-placeholders-for-memory-sharing-across-modules/223764)
**Author:** [@jttzhang](https://discuss.pytorch.org/u/jttzhang)\ **Replies:** 0\ **Last updated:** [October 31, 2025, 9:41am UTC](https://discuss.pytorch.org/t/torch-export-fx-graphmodule-unable-to-lift-constants-parameters-buffers-to-placeholders-for-memory-sharing-across-modules/223764 "2025-10-31T09:41:10Z")
Issue Description: I am trying to lift Parameter and buffer nodes from get\_attr to placeholder in a graph module obtained via torch.export, so that multiple modules can share the same tensor memory at inference time. H… --- ## [OSError: \[Errno 36\] File name too long](https://discuss.pytorch.org/t/oserror-errno-36-file-name-too-long/223758)
**Author:** [@xgbj](https://discuss.pytorch.org/u/xgbj)\ **Replies:** 3\ **Last updated:** [October 31, 2025, 6:44am UTC](https://discuss.pytorch.org/t/oserror-errno-36-file-name-too-long/223758 "2025-10-31T06:44:55Z")
Hello, we encountered a ‘file name too long’ bug while using PyTorch compile. The specific error is: OSError: \[Errno 36\] File name too long: '/tmp/torchinductor\_hadoop-pnc/triton/3/b13585fb327ef97fcb905e2d1fea4abede414a… --- ## [Backend Compiler Graph Break](https://discuss.pytorch.org/t/backend-compiler-graph-break/223754)
**Author:** [@walrus\_23](https://discuss.pytorch.org/u/walrus_23)\ **Replies:** 0\ **Last updated:** [October 30, 2025, 12:53am UTC](https://discuss.pytorch.org/t/backend-compiler-graph-break/223754 "2025-10-30T00:53:36Z")
See GitHub issue #166582 on the main PyTorch repo. TLDR: I have an int8 matmul on a custom ASIC (in simulation) that I am compiling to replace all linear layers of any PyTorch model. However, the model is trained using … --- ## [About dynamo.disable](https://discuss.pytorch.org/t/about-dynamo-disable/223743)
**Author:** [@Vaen\_Joe](https://discuss.pytorch.org/u/Vaen_Joe)\ **Replies:** 0\ **Last updated:** [October 29, 2025, 7:55am UTC](https://discuss.pytorch.org/t/about-dynamo-disable/223743 "2025-10-29T07:55:03Z")
Hi, I have a question about how torch.\_dynamo.disable behaves inside nested model structures. In my setup, I have a deeply nested model, and I applied @torch.\_dynamo.disable to a small inner function at the deepest leve… --- ## [PyTorch with CUDA support is not installed, although everything worked before that](https://discuss.pytorch.org/t/pytorch-with-cuda-support-is-not-installed-although-everything-worked-before-that/223662)
**Author:** [@GameSD](https://discuss.pytorch.org/u/GameSD)\ **Replies:** 1\ **Last updated:** [October 22, 2025, 3:06pm UTC](https://discuss.pytorch.org/t/pytorch-with-cuda-support-is-not-installed-although-everything-worked-before-that/223662 "2025-10-22T15:06:10Z")
Everything worked fine for about 2 years. Unexpectedly, I discovered an error connecting to CUDA today. Now the torch.cuda.is\_available() function is False, and the torch.zeros(1).cuda() command returns the error: Torch… --- ## [Compiling a loop on MPS - exceeding limit of buffers](https://discuss.pytorch.org/t/compiling-a-loop-on-mps-exceeding-limit-of-buffers/223631)
**Author:** [@maciejkorzepa](https://discuss.pytorch.org/u/maciejkorzepa)\ **Replies:** 0\ **Last updated:** [October 17, 2025, 6:26am UTC](https://discuss.pytorch.org/t/compiling-a-loop-on-mps-exceeding-limit-of-buffers/223631 "2025-10-17T06:26:20Z")
Hi, I’m working on compiling a loop on MPS device, but I’m hitting an error “error: number of constant buffers exceeds maximum supported (31)” while compiling the backward pass. Forward pass compiles fine. I realize thi… --- ## [Why does custom passes before joint graph (joint\_custom\_pre\_pass) executed twice?](https://discuss.pytorch.org/t/why-does-custom-passes-before-joint-graph-joint-custom-pre-pass-executed-twice/223625)
**Author:** [@QZero\_Chen](https://discuss.pytorch.org/u/QZero_Chen)\ **Replies:** 0\ **Last updated:** [October 16, 2025, 8:34am UTC](https://discuss.pytorch.org/t/why-does-custom-passes-before-joint-graph-joint-custom-pre-pass-executed-twice/223625 "2025-10-16T08:34:31Z")
And Why does joint\_custom\_pre\_pass executed twice, is this a mistake or a design by purpose? [Previous page](https://discuss.pytorch.org/c/compile/41.md) [Next page](https://discuss.pytorch.org/c/compile/41.md?page=2)