Dcp.save straight to cloud storage

distributed torchtitan

Sam_Shleifer (Sam Shleifer) March 17, 2025, 9:59pm 1

Is there an example somewhere of doing dcp.save straight to cloud storage, like s3 or google cloud storage?

yf225 (PyTorch Developer, Meta) March 21, 2025, 6:13am 2

Curious would these help?

AWS S3: Amazon S3 Connector for PyTorch now supports Distributed Checkpoint - AWS
Google Cloud Storage: Connector for PyTorch | Cloud Storage | Google Cloud