Those whose acquaintance with scientific research is derived chiefly from its practical results easily develop a completely false notion of the mentality of the men who, surrounded by a skeptical world, have shown the way to kindred spirits scattered wide through the world and through the centuries. Only one who has devoted his life to […]
Category: Thoughts
On Finding Principles for Building Agents with Their Own Goals and Purposes
The rapid advancement of large language models (LLMs) has once again validated the bitter lesson: General methods that leverage computation ultimately outperform alternatives—by a large margin. This superiority becomes increasingly evident as more computation becomes available. General methods scale effectively with more data, computation, and memory, whereas alternative approaches often struggle due to their reliance […]
The lesson from “The Bitter Lesson”
Rich Sutton’s note "The Bitter Lesson" is often quoted by others. The most frequently quoted sentence is the following: The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. Based on such a statement, a […]