distributed

torchtitan Use this subcategory for all discussions related to end to end composability of distributed technologies using torchtitan. distributed-rpc

Topic	Replies	Views	Activity
About the distributed category distributed	2	2840	November 28, 2025
8xH100 training issue distributed	2	54	December 8, 2025
Node 0 cannot connect to itself distributed	2	48	December 1, 2025
DDP: model not synchronizing across gpu's distributed	8	5505	November 28, 2025
Help with DDP in kaggle notebook distributed	2	282	November 26, 2025
Optimizer_state_dict with multiple optimizers in FSDP distributed	1	109	November 20, 2025
Alternating Parameters in DDP distributed	1	262	November 17, 2025
In a multi-GPU DDP environment, if the loss on one rank is NaN while the others are normal, could this cause the all-reduce to hang? distributed	1	51	November 12, 2025
RPC cannot run in jetson orin because of the specific uuid of orin distributed-rpc	3	80	November 11, 2025
Distributed Training causes model to output NaN values after resuming from snapshot distributed	0	27	November 7, 2025
Pipeline Parallelism performance with distributed-rpc on Jetson Nano devices distributed-rpc	3	1136	November 6, 2025
Problem: Pipeline Parallelism with distributed-rpc on Jetson Nano devices distributed-rpc	1	219	October 28, 2025
FSDP2 and gradient w.r.t. inputs distributed	2	83	October 28, 2025
Using Symmetric Memory One Shot All Reduce distributed	1	699	October 27, 2025
Tensor parallelism in image models like Unet distributed	4	505	October 27, 2025
Windows DDP on RTX 50-series only: use_libuv was requested but PyTorch was built without libuv support (works on 40/20-series) distributed-rpc	0	260	October 25, 2025
CPU thread slow to enqueue GPU and communication kernels torchtitan	2	80	October 20, 2025
Get `state_dict` from `DataDistributedParallel` model while other thread is running `backward` distributed	0	28	October 19, 2025
Suggested design for multiprocess federated learning distributed	1	470	October 13, 2025
Use fsdp training, 80 h800 gpu can run success, but 160 h800 gpu oom distributed	0	23	October 11, 2025
I am running the below code, which is wrong, but still the torch run command runs without any errors? How do I debug this? distributed	3	66	October 6, 2025
Using debugpy with DDP results in driver leaking GPU memory distributed	1	46	October 2, 2025
Model.to(device) vs. tensor.to(device) distributed	3	263	September 28, 2025
Does torch support custom stream for nccl commucation now? distributed	5	144	September 28, 2025
"Cannot allocate memory" for multinode training distributed	2	54	September 23, 2025
How to apply selective activation checkpointing on _grouped_mm torchtitan	0	103	September 20, 2025
DistributedDataParallel init hangs distributed	1	228	September 20, 2025
Process stuck by the dist.barrier() using DDP after dist.init_process_group distributed	2	490	September 20, 2025
Proper way to call torch.distributed.send/recv distributed	4	103	September 18, 2025
Understanding relation of FSDP and TP distributed	0	72	September 16, 2025