Why There Will Be No AGI
August 2023
There is a lot of debate about the capabilities and limitations of large language models like GPT-3, GPT-4, Claude, and Llama. Do they display emergent capabilities ? Do they merely display memorization but not generalization powers ? Is it correct to imply that they have reasoning abilities ? Do they display human-level natural language understanding ? How do we even define human-level natural language understanding ? Will it be ever possible to get rid of the hallucination problem ? Is the Natural Language Processing field obsolete (in a Fukuyama End of History style) ?
The November surprise
First, let me say something that is not so original, because it echoes a lot of the thoughts I have seen around on LLMs. It is incredible to see what next word prediction models can do. No one anticipated that we would be able to do so many useful things with them, until OpenAI opened Pandora’s box and released ChatGPT in November 2022. It is certainly a testament to the power of language and specifically context (given that attention is all we need), that we can do summarization, translation, and question answering among other things with the same GPT model.
There is so much power in language, so much wisdom, knowledge. There is such immense power in written language. It’s beautiful that we are able to…
