Windows support timeline for torch.compile

torch.compile torch._dynamo
1

Is there a rough timeline for when/if we can expect Windows to be supported for torch.compile and torch._dynamo.optimize?

A cursory search didn’t yield anything so posting it here.

1 Like
2

torch.compile will work, it’s just that inductor the default compiler which leverages triton under the hood does not.

You can +1 this if you like Is there a plan to support Windows? · Issue #1640 · openai/triton · GitHub

3

@marksaroufim I didn’t understand. Can you please elaborate ?

4

A simplified view of torch.compile on GPU is that it takes your python code and generates openai/triton code (we call the compiler that does this transformation inductor) and because triton does not support Windows then it’s not obvious how torch.compile will support Windows GPU

However supporting Windows on CPU should be fine and there are other backends you can pass in to torch.compile(backend="example_backend") that might have better Windows support

5

Thanks for a detailed reply. From what I understand, you are saying that torch.compile should work on CPU on windows. Am I correct in my understanding. I am trying to do torch.compile on CPU only

6

Sort of. I have triton installed on native windows along with deepspeed. you just gotta find the wheels or build them your selves. A wonderful gentleman by the name of “wkpark” makes triton wheels and their libraries for windows here is my link to the ones i have installed, but you can find the official ones from him in GitHub. Here is the install i made to use unsloth on windows at the bottom you’ll find a home.md where the instructions are at the very bottom, and at the bottom of the github issue, there is detailed pictures for certain requirments. I got unsloth running in native windows. · Issue #210 · unslothai/unsloth · GitHub

Blockquote

7

for what it’s worth, I now have torch.compile working on CPU using the openvino backend, though you have to get some updated packages.

pip install torch==2.3.1+cpu openvino --extra-index-url https://download.pytorch.org/whl/cpu

transformer = torch.compile(transformer, fullgraph=True, backend=‘openvino’)

I’m on an older AMD CPU so I’m not really getting a substantial benefit from openvino, through I do think my test cases have gone from crushingly slow to just very slow, which is nice.

See here for some other discussion: