FSDP: size_based_auto_wrap_policy import error?

distributed
1

I am using

python 3.8
torch 1.12

And I meet this:
ImportError: cannot import name ‘size_based_auto_wrap_policy’ from ‘torch.distributed.fsdp.wrap’
Could anyone provide some suggestion? Thank you!!

In my win10, I have pytorch 1.12.1 py3.9_cuda11.6_cudnn8_0 pytorch
In my linux server, I have torch 1.12.0a0+bd13bc6 pypi_0 pypi
my win10 can find the size_based… but server can not!! Why?

2

That’s expected as your server build is too old.
bd13bc6 was merged on March 10th, while size_based_auto_wrap_policy was merged on May 10th in #76858.

3

Sorry, I am still a learner…I am not sure what is ‘server build’? Is that someting I can change? Or a server hardware that I can not change? (Previously I thought I can update pytorch to solve this…)
Thank you!!!

4

“Server build” refers to the PyTorch version built from source on your “linux server”:

Update it from the latest binaries or rebuilt ir from the current master and it should work.

1 Like