DualPipe: An innovative bidirectional pipeline parallelism algorithm

Viewed 43
DualPipe, developed by Jiashi Li, Chengqi Deng, and Wenfeng Liang, introduces a novel approach to pipeline parallelism with two-way communication. Users engaged with the post express interest in understanding its functionalities and compare it with other algorithms like 1F1B and ZB1P. There's also mention of possible features like SM re-allocation that might have been anticipated but are not addressed in the current context.
0 Answers