Hugging Face Blog·· 2022-05-02AI 评分36
用 PyTorch FullyShardedDataParallel 加速大模型训练：Accelerate 无需改代码即可启用 FSDP
Accelerate Large Model Training using PyTorch Fully Sharded Data Parallel
AI 导读
Hugging Face 介绍如何用 Accelerate 库无需改动代码即可启用 PyTorch FullyShardedDataParallel（FSDP）训练大模型。
来源：Hugging Face Blog · huggingface.co