szha.ai

Compute-optimal is not cluster-optimal

yuxin_tang · 13 points · 0 comments · 17小时前 · 打开原文

https://arxiv.org/pdf/2608.10605: Our new paper folds the systems stage into the scaling-law stage. Price every candidate architecture on what the cluster actually delivers, and the answer changes: the sparsity an MoE 'should' have depends on the cluster you train it on.

评论

1 条预览评论 · 正在加载完整讨论
unknown17小时前

[deleted]