https://blog.ans20xx.com/tags/ddp/Recent content in DDP on 安橙的博客Hugo -- 0.163.3zhSat, 20 Jun 2026 00:00:00 +0000https://blog.ans20xx.com/posts/ai/day15/Sat, 20 Jun 2026 00:00:00 +0000https://blog.ans20xx.com/posts/ai/day15/进入 AI Infra 分布式训练阶段:理解进程组、rank/world_size、torchrun 启动模型,掌握 AllReduce、AllGather、ReduceScatter、Broadcast 四类集合通信,并跑通一个 DDP MNIST。https://blog.ans20xx.com/posts/ai/day17/Sat, 20 Jun 2026 00:00:00 +0000https://blog.ans20xx.com/posts/ai/day17/进入分布式训练的第一条主线:从 DataParallel 到 DistributedDataParallel,拆开梯度同步时机、Reducer、bucket、overlap 与 no_sync;阅读 torch/nn/parallel/distributed.py 关键路径,并用 torchrun 跑一个可观测的 DDP 实验。