LLM, what they are, what they ain’t
Yet another insightful tweet by François Chollet (@fchollet):
People seem to be falling for two rather thoughtless extremes:
1. "LLMs are AGI, they work like the human brain, they can reason, etc."
2. "LLMs are dumb and useless."Reality is that LLMs are not AGI — they're a big curve fit to a very large dataset. They work via…
— François Chollet (@fchollet) February 9, 2024
People seem to be falling for two rather thoughtless extremes:
- "LLMs are AGI, they work like the human brain, they can reason, etc."
- "LLMs are dumb and useless."
Reality is that LLMs are not AGI — they’re a big curve fit to a very large dataset. They work via memorization and interpolation. But that interpolative curve can be tremendously useful, if you want to automate a known task that’s a match for its training data distribution.
Memorization works, as long as you don’t need to adapt to novelty. You don’t *need* intelligence to achieve usefulness across a set of known, fixed scenarios.
In fact, that’s the entire story of the field of AI so far: achieve increasing levels of usefulness and automation, while bypassing the problem of creating intelligence.
