← Home
Whittemore examines the growing debate over whether scaling laws for large language models are plateauing. Reports from multiple labs suggest diminishing returns from simply making models bigger.
This summary was generated from show notes and public descriptions, not from a full transcript review. Details may contain inaccuracies.
Highlights
•
The 'scaling wall' narrative may be premature — labs have historically found new scaling axes when old ones plateau
Whittemore reviews previous scaling plateaus: compute scaling plateaued, then data scaling emerged. Data scaling plateaued, then RLHF improved quality without more data. Each apparent wall was actually a transition to a new axis.•
Even if scaling slows, the integration of current capabilities into the economy has barely begun
Whittemore: the debate about future model capabilities distracts from the fact that current models are massively underutilized. Most industries have barely begun integrating GPT-4-class capabilities.Was this useful?