REAP: One-Shot MoE Pruning Method Open-Sourced

Cerebras introduced REAP (Router-weighted Expert Activation Pruning), a one-shot method for compressing trillion-parameter MoE models that removes up to 50% of experts while retaining 97.6% of coding ability, with code and pruned checkpoints open-sourced on HuggingFace.