The rapid advancement of large language models (LLMs) has once again validated the bitter lesson: General methods that leverage computation ultimately outperform alternatives—by a large margin. This superiority becomes increasingly evident as more computation becomes available. General methods scale effectively with more data, computation, and memory, whereas alternative approaches often struggle due to their reliance […]
Category: Thoughts
The lesson from “The Bitter Lesson”
Rich Sutton’s note "The Bitter Lesson" is often quoted by others. The most frequently quoted sentence is the following: The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. Based on such a statement, a […]