Blog Odyssée

Jen-Hsun Huang: Stanford student and Entrepreneur, co-founder and CEO of NVIDIA

Great sequence about failure, innovation and "intellectual honesty" to guide the process.

Yet another insightful tweet by François Chollet (@fchollet):

People seem to be falling for two rather thoughtless extremes:

1. "LLMs are AGI, they work like the human brain, they can reason, etc."
2. "LLMs are dumb and useless."

Reality is that LLMs are not AGI — they're a big curve fit to a very large dataset. They work via…

— François Chollet (@fchollet) February 9, 2024

People seem to be falling for two rather thoughtless extremes:

1. "LLMs are AGI, they work like the human brain, they can reason, etc."
2. "LLMs are dumb and useless."

Reality is that LLMs are not AGI — they’re a big curve fit to a very large dataset. They work via memorization and interpolation. But that interpolative curve can be tremendously useful, if you want to automate a known task that’s a match for its training data distribution.

Memorization works, as long as you don’t need to adapt to novelty. You don’t *need* intelligence to achieve usefulness across a set of known, fixed scenarios.

In fact, that’s the entire story of the field of AI so far: achieve increasing levels of usefulness and automation, while bypassing the problem of creating intelligence.