Dwarkesh Patel says model size growth stalled after GPT-4, and argues the return to bigger models is partly explained by RL scaling, citing a possible 5T-parameter direction.
parameter scaling had stalled for a long time after GPT-4's trillion+ parameters
only now are models getting bigger again.
Partially it’s RL scaling
This finding is one of many signals tracked across Artificial Intelligence. The live feed updates every few hours with new expert voices, debates, and emerging ideas.
← Back to Artificial Intelligence