# torch.compile **URL:** https://discuss.pytorch.org/c/compile/41.md?page=2 [Latest](https://discuss.pytorch.org/latest.md) · [Categories](https://discuss.pytorch.org/categories.md) **Page:** 3 --- ## [Why does custom passes before joint graph (joint\_custom\_pre\_pass) executed twice?](https://discuss.pytorch.org/t/why-does-custom-passes-before-joint-graph-joint-custom-pre-pass-executed-twice/223625)
**Author:** [@QZero\_Chen](https://discuss.pytorch.org/u/QZero_Chen)\ **Replies:** 0\ **Last updated:** [October 16, 2025, 8:34am UTC](https://discuss.pytorch.org/t/why-does-custom-passes-before-joint-graph-joint-custom-pre-pass-executed-twice/223625 "2025-10-16T08:34:31Z")
And Why does joint\_custom\_pre\_pass executed twice, is this a mistake or a design by purpose? --- ## [About some issue about torch.median()](https://discuss.pytorch.org/t/about-some-issue-about-torch-median/223608)
**Author:** [@pistachio](https://discuss.pytorch.org/u/pistachio)\ **Replies:** 1\ **Last updated:** [October 15, 2025, 9:50pm UTC](https://discuss.pytorch.org/t/about-some-issue-about-torch-median/223608 "2025-10-15T21:50:14Z")
when i want to use torch.median(), i found that the median of even number of data point is not as expected why col\_medians is not tensor(\[2.5, 4.5, 3.5, 5.5\]) the version of torch is 2.8.0+cu128. --- ## [Torch.compile \[WARNING\] not enough SMs to use max\_autotune\_gemm mode](https://discuss.pytorch.org/t/torch-compile-warning-not-enough-sms-to-use-max-autotune-gemm-mode/184405)
**Author:** [@anass\_017](https://discuss.pytorch.org/u/anass_017)\ **Replies:** 4\ **Last updated:** [October 14, 2025, 6:51pm UTC](https://discuss.pytorch.org/t/torch-compile-warning-not-enough-sms-to-use-max-autotune-gemm-mode/184405 "2025-10-14T18:51:54Z")
I have a small model with 900K in wights compiled\_model = torch.compile(net\_model,backend="inductor",mode="max-autotune") When I compile the model using max\_autotune model, he is giving me this warning : \[WARNING\] … --- ## [Is it normal for compiled model outputs to have different tensor strides?](https://discuss.pytorch.org/t/is-it-normal-for-compiled-model-outputs-to-have-different-tensor-strides/223469)
**Author:** [@tinywisdom](https://discuss.pytorch.org/u/tinywisdom)\ **Replies:** 3\ **Last updated:** [October 14, 2025, 2:50am UTC](https://discuss.pytorch.org/t/is-it-normal-for-compiled-model-outputs-to-have-different-tensor-strides/223469 "2025-10-14T02:50:33Z")
Hello everyone, I recently compiled my model using torch.compile, and I noticed that the results after compilation have some layout differences compared to the original (eager) execution. When I check the outputs using… --- ## [TS2EPConverter and \`dynamic\_shapes\`](https://discuss.pytorch.org/t/ts2epconverter-and-dynamic-shapes/223472)
**Author:** [@olokobayusuf](https://discuss.pytorch.org/u/olokobayusuf)\ **Replies:** 0\ **Last updated:** [October 5, 2025, 2:27pm UTC](https://discuss.pytorch.org/t/ts2epconverter-and-dynamic-shapes/223472 "2025-10-05T14:27:54Z")
We are working with pipelines where we have to convert pre-existing torch.jit models to downstream AI serving formats (e.g. ONNX, ExecuTorch). The current torch.export.export API does not support these, so instead we use… --- ## [CUDA Graphs with FSDP 2 and flex attention?](https://discuss.pytorch.org/t/cuda-graphs-with-fsdp-2-and-flex-attention/223437)
**Author:** [@zaptrem](https://discuss.pytorch.org/u/zaptrem)\ **Replies:** 0\ **Last updated:** [October 3, 2025, 2:05am UTC](https://discuss.pytorch.org/t/cuda-graphs-with-fsdp-2-and-flex-attention/223437 "2025-10-03T02:05:17Z")
Hello, Is torch.compile with cuda graphs (either via reduce-overhead or max-autotune) compatible with FSDP 2 and flex attention? When both cuda graphs and FSDP 2 are on I get seemingly unending messages like: \`\` torch… --- ## [Module 'torch.library' has no attribute 'custom\_op'](https://discuss.pytorch.org/t/module-torch-library-has-no-attribute-custom-op/222610)
**Author:** [@Casey\_KimScripts](https://discuss.pytorch.org/u/Casey_KimScripts)\ **Replies:** 6\ **Last updated:** [September 24, 2025, 3:38pm UTC](https://discuss.pytorch.org/t/module-torch-library-has-no-attribute-custom-op/222610 "2025-09-24T15:38:38Z")
First time posting here - the context: I just wanted to try the below sourcecode from HuggingFace #%matplotlib inline import torch from diffusers.pipelines import DiffusionPipeline from matplotlib import pyplot as… --- ## [How many patterns does torch compile match and replace for now?](https://discuss.pytorch.org/t/how-many-patterns-does-torch-compile-match-and-replace-for-now/223291)
**Author:** [@ljh](https://discuss.pytorch.org/u/ljh)\ **Replies:** 0\ **Last updated:** [September 24, 2025, 12:55pm UTC](https://discuss.pytorch.org/t/how-many-patterns-does-torch-compile-match-and-replace-for-now/223291 "2025-09-24T12:55:19Z")
Hi, I’m trying to investigate how pattern matching works in torch compile. I tried search for “register\_replacement” and “gen\_register\_replacement”, which seem to be the way of registering patterns for matching. But I … --- ## [PyTorch Error: Compiler: cl is not found](https://discuss.pytorch.org/t/pytorch-error-compiler-cl-is-not-found/220205)
**Author:** [@RudeyPunk](https://discuss.pytorch.org/u/RudeyPunk)\ **Replies:** 4\ **Last updated:** [September 23, 2025, 3:15pm UTC](https://discuss.pytorch.org/t/pytorch-error-compiler-cl-is-not-found/220205 "2025-09-23T15:15:38Z")
I started generating an error whenever I try to build with PyTorch: BackendCompilerFailed: backend=‘inductor’ raised: RuntimeError: Compiler: cl is not found. I cannot figure out what changed. To test this on a machi… --- ## [Torch.compile + flex\_attention with custom score\_mod fails (FlexibleLayout / NoValidChoicesError)](https://discuss.pytorch.org/t/torch-compile-flex-attention-with-custom-score-mod-fails-flexiblelayout-novalidchoiceserror/223192)
**Author:** [@V1kstrand](https://discuss.pytorch.org/u/V1kstrand)\ **Replies:** 0\ **Last updated:** [September 19, 2025, 5:29am UTC](https://discuss.pytorch.org/t/torch-compile-flex-attention-with-custom-score-mod-fails-flexiblelayout-novalidchoiceserror/223192 "2025-09-19T05:29:25Z")
(I also made a pytorch github post here) :bug: Describe the bug Compiling a module that wraps torch.nn.attention.flex\_attention with a custom score\_mod fails under torch.compile (Inductor). Eager execution works corre… --- ## [Are there any related works on "megakernel"?](https://discuss.pytorch.org/t/are-there-any-related-works-on-megakernel/223139)
**Author:** [@X\_Wang](https://discuss.pytorch.org/u/X_Wang)\ **Replies:** 0\ **Last updated:** [September 17, 2025, 3:50am UTC](https://discuss.pytorch.org/t/are-there-any-related-works-on-megakernel/223139 "2025-09-17T03:50:37Z")
Hi all, are there any related works on “megakernel” here? I’m planning to research this topic using torch.compile. According to the article: “We bypass a key issue by merging the entire Llama-1B forward pass into a sin… --- ## [In Dynamo+AOTAutograd, why run Faketensor through the code multiple times?](https://discuss.pytorch.org/t/in-dynamo-aotautograd-why-run-faketensor-through-the-code-multiple-times/223093)
**Author:** [@jinsun-yoo](https://discuss.pytorch.org/u/jinsun-yoo)\ **Replies:** 0\ **Last updated:** [September 15, 2025, 3:34am UTC](https://discuss.pytorch.org/t/in-dynamo-aotautograd-why-run-faketensor-through-the-code-multiple-times/223093 "2025-09-15T03:34:24Z")
I posted a series of questions on the forum today. I’ve listed them in hope that putting them together would better shed light on what I do know and don’t know. In Dynamo+AOTAutograd, why run Faketensor through the cod… --- ## [Better understanding why AOTAutograd decomposes \`fused\_rms\_norm\_backward\` for CUDA, but not for Meta tensors](https://discuss.pytorch.org/t/better-understanding-why-aotautograd-decomposes-fused-rms-norm-backward-for-cuda-but-not-for-meta-tensors/223096)
**Author:** [@jinsun-yoo](https://discuss.pytorch.org/u/jinsun-yoo)\ **Replies:** 0\ **Last updated:** [September 15, 2025, 5:54am UTC](https://discuss.pytorch.org/t/better-understanding-why-aotautograd-decomposes-fused-rms-norm-backward-for-cuda-but-not-for-meta-tensors/223096 "2025-09-15T05:54:49Z")
I posted a series of questions on the forum today. I’ve listed them in hope that putting them together would better shed light on what I do know and don’t know. In Dynamo+AOTAutograd, why run Faketensor through the cod… --- ## [Compile: \`DataLoader.\_\_setattr\_\_\` should not be traced](https://discuss.pytorch.org/t/compile-dataloader-setattr-should-not-be-traced/223072)
**Author:** [@Laurent1](https://discuss.pytorch.org/u/Laurent1)\ **Replies:** 2\ **Last updated:** [September 13, 2025, 6:28pm UTC](https://discuss.pytorch.org/t/compile-dataloader-setattr-should-not-be-traced/223072 "2025-09-13T18:28:48Z")
I ran into an issue with pytorch 2.7.1 with compile on CPU. I got: \[…\] File “/home/laurent/train.py”, line 1358, in forward\_backward\_pass\_model configs = model(batch) File “/home/laurent/python3.13\_pytorch2.7\_/lib/p… --- ## [Torch.compile error on backward](https://discuss.pytorch.org/t/torch-compile-error-on-backward/223046)
**Author:** [@richardzzj](https://discuss.pytorch.org/u/richardzzj)\ **Replies:** 0\ **Last updated:** [September 12, 2025, 3:26am UTC](https://discuss.pytorch.org/t/torch-compile-error-on-backward/223046 "2025-09-12T03:26:44Z")
The compiling of forward worked just fine, but failed on backward: torch/\_inductor/ir.py, line 2455, in \_normalize\_size: assert len(new\_size) == len(old\_size) torch.\_inductor.exc.LoweringException: AssertionError: … --- ## [Do I need to rerun torch.compile between training and inference?](https://discuss.pytorch.org/t/do-i-need-to-rerun-torch-compile-between-training-and-inference/222945)
**Author:** [@Saad\_Amin](https://discuss.pytorch.org/u/Saad_Amin)\ **Replies:** 1\ **Last updated:** [September 9, 2025, 11:21pm UTC](https://discuss.pytorch.org/t/do-i-need-to-rerun-torch-compile-between-training-and-inference/222945 "2025-09-09T23:21:24Z")
Suppose I have the following code: model = Model() model.compile() model.train() train\_model(model, ...) model.eval() test\_model(model, ...) When I switch over to evaluation model, does torch.compile automatically r… --- ## [Autofunctionalize v2 / Reinplace](https://discuss.pytorch.org/t/autofunctionalize-v2-reinplace/222984)
**Author:** [@Jerome\_Ku](https://discuss.pytorch.org/u/Jerome_Ku)\ **Replies:** 0\ **Last updated:** [September 9, 2025, 4:53pm UTC](https://discuss.pytorch.org/t/autofunctionalize-v2-reinplace/222984 "2025-09-09T16:53:28Z")
@bdhirsh On pytorch nightly 2.9.0.dev20250907+cu128, running the following inplace op (see original context) withenable\_auto\_functionalized\_v2 enabled / disabled doesn’t seem to have an effect: import torch import torc… --- ## [Torch.compile does not work after first graph break](https://discuss.pytorch.org/t/torch-compile-does-not-work-after-first-graph-break/222885)
**Author:** [@richardzzj](https://discuss.pytorch.org/u/richardzzj)\ **Replies:** 0\ **Last updated:** [September 5, 2025, 9:23am UTC](https://discuss.pytorch.org/t/torch-compile-does-not-work-after-first-graph-break/222885 "2025-09-05T09:23:30Z")
I have an auto generated ‘forward’ func: def forward(a, b, c): … tmp=torch.lstm(a, (torch.tensor(…), torch.tensor(…)), self.lstm.weight, self.lstm.bias) … return ret After using ‘torch.\_dynamo.explain‘, i got one… --- ## [How to understand the files generated by TORCH\_COMPILE\_DEBUG](https://discuss.pytorch.org/t/how-to-understand-the-files-generated-by-torch-compile-debug/222854)
**Author:** [@Guitenbay](https://discuss.pytorch.org/u/Guitenbay)\ **Replies:** 0\ **Last updated:** [September 4, 2025, 3:30am UTC](https://discuss.pytorch.org/t/how-to-understand-the-files-generated-by-torch-compile-debug/222854 "2025-09-04T03:30:32Z")
I use torch.compile to run the resnet50 model. The env are: TORCH\_COMPILE\_DEBUG=1 INDUCTOR\_ORIG\_FX\_SVG=1 INDUCTOR\_POST\_FUSION\_SVG=1 TORCHINDUCTOR\_FX\_GRAPH\_CACHE=0 then I get some files: I know orig\_fx\_graph\_diagram… --- ## [Dynamo logs generated by torch.distributed](https://discuss.pytorch.org/t/dynamo-logs-generated-by-torch-distributed/222601)
**Author:** [@githubsgi](https://discuss.pytorch.org/u/githubsgi)\ **Replies:** 5\ **Last updated:** [September 3, 2025, 9:38pm UTC](https://discuss.pytorch.org/t/dynamo-logs-generated-by-torch-distributed/222601 "2025-09-03T21:38:58Z")
How should I interpret the following dynamo log for distributed calls. \_dispatch.py:152\] Dispatching op\_call: Op(op=aten.native\_layer\_norm.default, args\_schema=Spec(S(1) on (2, 16, 16)), \[16\], Spec(R on (16,)), Spec(R … --- ## [Reducing compiled call overhead](https://discuss.pytorch.org/t/reducing-compiled-call-overhead/222740)
**Author:** [@crizz](https://discuss.pytorch.org/u/crizz)\ **Replies:** 0\ **Last updated:** [August 29, 2025, 10:52am UTC](https://discuss.pytorch.org/t/reducing-compiled-call-overhead/222740 "2025-08-29T10:52:44Z")
Hello, I have been optimizing a Soft Actor Critic implementation of mine using torch compile, and I feel like I am hitting a limit in how much I can reduce cuda calls overhead. It looks like this is because the PyTorch c… --- ## [How can we reliably figured out what the parameters of a generated PTX kernel are with torch.compile()?](https://discuss.pytorch.org/t/how-can-we-reliably-figured-out-what-the-parameters-of-a-generated-ptx-kernel-are-with-torch-compile/222650)
**Author:** [@anneouyang](https://discuss.pytorch.org/u/anneouyang)\ **Replies:** 0\ **Last updated:** [August 25, 2025, 9:39pm UTC](https://discuss.pytorch.org/t/how-can-we-reliably-figured-out-what-the-parameters-of-a-generated-ptx-kernel-are-with-torch-compile/222650 "2025-08-25T21:39:38Z")
When we run torch.compile() on torch.matmul(A, B), we get a PTX function with signature: \`\` .visible .entry triton\_( .param .u64 triton\_\_param\_0, .param .u64 triton\_\_param\_1, .param .u64 triton\_\_param\_2, .param .u6… --- ## [C++/Cuda / aot.compile and cuda graph](https://discuss.pytorch.org/t/c-cuda-aot-compile-and-cuda-graph/195902)
**Author:** [@Benjamin\_Sergeant](https://discuss.pytorch.org/u/Benjamin_Sergeant)\ **Replies:** 2\ **Last updated:** [August 23, 2025, 6:17am UTC](https://discuss.pytorch.org/t/c-cuda-aot-compile-and-cuda-graph/195902 "2025-08-23T06:17:16Z")
Hi there, We are playing with ahead of time compilation (AOTInductor), to be able to produce models that can be executed by a C++ program ; we’ve seen great results, but results seems even better when the ‘cuda-graph’ o… --- ## [Training with flex attention is extremely slow due to torch.compile settings](https://discuss.pytorch.org/t/training-with-flex-attention-is-extremely-slow-due-to-torch-compile-settings/222581)
**Author:** [@yijiong\_yu](https://discuss.pytorch.org/u/yijiong_yu)\ **Replies:** 0\ **Last updated:** [August 22, 2025, 12:14pm UTC](https://discuss.pytorch.org/t/training-with-flex-attention-is-extremely-slow-due-to-torch-compile-settings/222581 "2025-08-22T12:14:48Z")
When I was running an LLM supervised fine-tuning code, where each training sample has a different sequence length, I found that when using flex\_attention, the training speed was nearly twice slower than using sdpa or ea… --- ## [Is it possible that node's name can change during torch.compile?](https://discuss.pytorch.org/t/is-it-possible-that-nodes-name-can-change-during-torch-compile/222579)
**Author:** [@ChrisRR](https://discuss.pytorch.org/u/ChrisRR)\ **Replies:** 0\ **Last updated:** [August 22, 2025, 9:54am UTC](https://discuss.pytorch.org/t/is-it-possible-that-nodes-name-can-change-during-torch-compile/222579 "2025-08-22T09:54:52Z")
I encountered a fatal error when the function get\_kernel\_metadata executes: Traceback (most recent call last): ... File "/home/.pyenv/versions/3.11.9/lib/python3.11/site-packages/torch/\_inductor/utils.py", line 597,… --- ## [Torch.compile Numpy code throws mean() arguments error](https://discuss.pytorch.org/t/torch-compile-numpy-code-throws-mean-arguments-error/222544)
**Author:** [@Sleriss](https://discuss.pytorch.org/u/Sleriss)\ **Replies:** 0\ **Last updated:** [August 21, 2025, 10:57am UTC](https://discuss.pytorch.org/t/torch-compile-numpy-code-throws-mean-arguments-error/222544 "2025-08-21T10:57:32Z")
Hello, I have a piece of Numpy code that is running fine, containing a function named depth\_consistency which returns a 1D array. When I call this function, I then usually have to compute the mean of absolute values us… --- ## [Using RNG Generator in torch.compile](https://discuss.pytorch.org/t/using-rng-generator-in-torch-compile/222486)
**Author:** [@crizz](https://discuss.pytorch.org/u/crizz)\ **Replies:** 2\ **Last updated:** [August 20, 2025, 10:53am UTC](https://discuss.pytorch.org/t/using-rng-generator-in-torch-compile/222486 "2025-08-20T10:53:09Z")
Hi, I was trying to find some info on how random number generation works in torch compile. I have some code that generates quite often random numbers (an RL environment), and I am trying to speed everything up using comp… --- ## [How to turn off inlining / force materialization in TorchInductor during torch.compile?](https://discuss.pytorch.org/t/how-to-turn-off-inlining-force-materialization-in-torchinductor-during-torch-compile/222407)
**Author:** [@skv66](https://discuss.pytorch.org/u/skv66)\ **Replies:** 0\ **Last updated:** [August 16, 2025, 11:04pm UTC](https://discuss.pytorch.org/t/how-to-turn-off-inlining-force-materialization-in-torchinductor-during-torch-compile/222407 "2025-08-16T23:04:43Z")
Hi all, I’m trying to reproduce the “Without inlining” ablation from the PyTorch 2.0 paper (the table that reports speedups for “All TorchInductor optimizations”, and then “Without inlining”, “Without fusion”, and “With… --- ## [Some questions about torch.compile](https://discuss.pytorch.org/t/some-questions-about-torch-compile/222378)
**Author:** [@wyzqwq](https://discuss.pytorch.org/u/wyzqwq)\ **Replies:** 0\ **Last updated:** [August 15, 2025, 12:52pm UTC](https://discuss.pytorch.org/t/some-questions-about-torch-compile/222378 "2025-08-15T12:52:23Z")
Hi Pytorch team, The article PyTorch + vLLM = :heart: – PyTorch shows 1.05x–1.9x speedups with torch.compile for models like Llama4/Qwen3/Gemma3. To better understand: What benchmark setup (hardware, input shape) was … --- ## [\`torch.compile\` (w/ Torch Inductor) benchmarks/models for Multi GPU](https://discuss.pytorch.org/t/torch-compile-w-torch-inductor-benchmarks-models-for-multi-gpu/222147)
**Author:** [@Abhishek\_Ghosh](https://discuss.pytorch.org/u/Abhishek_Ghosh)\ **Replies:** 0\ **Last updated:** [August 7, 2025, 5:22am UTC](https://discuss.pytorch.org/t/torch-compile-w-torch-inductor-benchmarks-models-for-multi-gpu/222147 "2025-08-07T05:22:20Z")
I have understanding of the models from the PyTorch 2.0 paper using torch.compile, which use the default Inductor backend. The PyTorch benchmark/CI has scripts to test them out. But they all seem to make use of torch.com… [Previous page](https://discuss.pytorch.org/c/compile/41.md?page=1) [Next page](https://discuss.pytorch.org/c/compile/41.md?page=3)