Compromised PyTorch-nightly dependency chain between December 25th and December 30th, 2022

torch.compile torch._dynamo
1

Important Security update for those installed nightly version between 25th and 30th of December

Note: torchdynamo seemed relevant since it is an important motivation for trying out nightly version 2.0

2

Thanks for cross-posting it here. I’ll change the topic a bit to clarify the security risk.

EDIT: let me also pin the topic

4

@ptrblck Quick question: Is there any new step in CI/CD pipeline to make sure that this type of particular supply chain attack, where one can make sure that there is no free pypi package that can overwrite those from Pytorch nightly index. Or will the pytorch team make sure to create ‘dummy-packages’ of their own on pypi ?

Also, I saw the message from pypi webpage of malicious package. It says

This is not the real torchtriton package but uploaded here to discover dependency confusion vulnerabilities. You can get the real torchtriton from https://download.pytorch.org/whl/nightly/torchtriton/

Did the author really create the package to ‘discover’ or is it just a misdirection?

5

Creating placeholder packages is the common approach to prevent such attacks, which is also what NVIDIA does with CUDA packages and other libs.
In fact @malfet already created a placeholder for pytorch-triton which is the new name of the package to avoid future attacks. I don’t know if specific tests are executed in CI to double check it.

At this point in time I would not trust this message and assume the description is misleading.

2 Likes
7

Does anyone know if running either of the following with the vulnerable nightly packages would have executed the torchtriton malware:

import torch
torch.cuda.is_available()

python -m torch.utils.collect_env

Based on the blog post it says you would have had to have imported torchtriton, but does importing pytorch also import torchtriton?

The malicious binary is executed when the triton package is imported, which requires explicit code to do and is not PyTorch’s default behavior.

Also, does anyone know around what time in GMT on December 30th the impacted nightlys were removed from pip? I see the commit to remove torchtriton in the github.com repo at Dec 30th at 22:49 GMT and am hoping the pip packages were removed around then, too.

Thanks!