https://blog.ans20xx.com/tags/pipeline-parallel/Recent content in Pipeline Parallel on 安橙的博客Hugo -- 0.163.3zhSat, 20 Jun 2026 00:00:00 +0000https://blog.ans20xx.com/posts/ai/day20/Sat, 20 Jun 2026 00:00:00 +0000https://blog.ans20xx.com/posts/ai/day20/拆开 Pipeline Parallel:理解模型按层切 stage、micro-batch 如何填流水线,对比 GPipe、1F1B 与 Megatron Interleaved 1F1B,掌握 bubble 时间计算与 pipeline 调参方法。https://blog.ans20xx.com/posts/ai/day22/Sat, 20 Jun 2026 00:00:00 +0000https://blog.ans20xx.com/posts/ai/day22/把 Day19-21 的 TP、PP、DP、SP/CP 组合起来,在单机多卡上用 Megatron-LM 跑一个小 GPT,并通过调整 tensor-model-parallel-size 与 pipeline-model-parallel-size 理解并行维度的取舍。