> ## Content Index
> Fetch the complete content index at: https://tat.chessservices.de/llms.txt
> Use this file to discover other available public pages before exploring further.

# AI Can’t Teach AI New Tricks
- URL: https://tat.chessservices.de/ai-cant-teach-ai-new-tricks/
- Published: 2024-10-22T15:48:57.000Z
- Updated: 2024-10-22T15:48:57.000Z
- Description: Anthropic CEO Dario Amodei predicts that “powerful AI” will surpass human intelligence in many areas by 2026. OpenAI claims its latest models “ They can reason through complex tasks and solve harder problems.” Thinking? Reasoning? Will AI become humanlike? Conscious?
- Author: Jacques Bopp
- Tags: Science, Economy, #Import 2026-08-20 14:53

**A lack of human prose leads to what’s called ‘model collapse’ and pure gibberish.**

*WSJ,* [*Andy Kessler*](https://www.wsj.com/news/author/andy-kessler?ref=tat.chessservices.de)*, Oct. 20, 2024* 

Artificial intelligence is our economy’s next productivity power pack. It better be—OpenAI just [raised](https://www.cnbc.com/2024/10/03/openai-gets-4-billion-revolving-credit-line-on-top-of-latest-funding.html?ref=tat.chessservices.de) $6.6 billion, the largest venture-capital funding of all time, at a $157 billion valuation. Oh, and the company will [lose](https://finance.yahoo.com/news/report-reveals-openais-44-billion-145334935.html?guccounter=1&ref=tat.chessservices.de) $5 billion this year and is projecting $44 billion in losses through 2029.

We are bombarded with breathless press releases. Anthropic CEO Dario Amodei [predicts](https://www.forbes.com/sites/johnwerner/2024/10/14/anthropics-dario-amodei-posits-an-ai-future/?ref=tat.chessservices.de) that “powerful AI” will surpass human intelligence in many areas by 2026\. OpenAI [claims](https://openai.com/index/introducing-openai-o1-preview/?ref=tat.chessservices.de) its latest models are “designed to spend more time thinking before they respond. They can reason through complex tasks and solve harder problems.” Thinking? Reasoning? Will AI become humanlike? Conscious?

Not so fast. Large language models are impressive, but they are still statistical models mimicking human thinking. You can’t just throw cheaper chips at the problem and expect growth. More than Moore’s Law (chips doubling in density every 18 months) is in play. I hate to be the one to throw it, but here’s some cold water on the AI hype cycle:

Moravec’s paradox: Babies are smarter than AI. In 1988 robotics researcher Hans Moravec [noted](https://digitaleconomy.stanford.edu/news/the-turing-trap-the-promise-peril-of-human-like-artificial-intelligence/?ref=tat.chessservices.de) that “It is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility.” Most innate skills are built into our DNA, and many of them are unconscious.

Mr. Moravec went on: “Encoded in the large, highly evolved sensory and motor portions of the human brain is a billion years of experience about the nature of the world and how to survive in it.” DNA is the carrier of life’s success signals. Yes, since we were fish.

AI has a long way to go. Last week, [Apple](https://www.wsj.com/market-data/quotes/AAPL?ref=tat.chessservices.de) AI researchers seemed to agree, [noting](https://arxiv.org/pdf/2410.05229?ref=tat.chessservices.de) that “current LLMs are not capable of genuine logical reasoning; instead, they attempt to replicate the reasoning steps observed in their training data.” 

Summing up the paradox, Harvard psychology professor Steven Pinker told me last week, “Things that are easy for us, like manipulating the 3-D world, are hard for AI. We’re not going to get an AI plumber anytime soon. Things that are hard for us, like diagnosing disease or writing code, may be easy for AI.”

Linguistic apocalypse paradox: As I’ve [noted before](https://www.wsj.com/articles/no-ai-machines-cant-think-and-arent-going-to-kill-us-all-c6942878?mod=article%5Finline&ref=tat.chessservices.de), AI smarts come from human logic embedded between words and sentences. Large language models need human words as input to become more advanced. But some researchers [believe](https://arxiv.org/html/2211.04325v2?ref=tat.chessservices.de#:~:text=In%20order%20to%20scale%20the,et%20al.,%202022%29%20.) we’ll run out of written words to train models sometime between 2026 and 2032.

Remember, you can’t train AI models on AI-generated prose. That leads to what’s known as [model collapse](https://www.nature.com/articles/s41586-024-07566-y?ref=tat.chessservices.de). Output becomes gibberish. Think of it as inbreeding—AI needs new human input to provide fresh perspective. One study [suggests](https://x.com/xuandongzhao/status/1845330594185806117?ref=tat.chessservices.de) if even 1% of data is synthetic, it’s enough to spoil training models. More humans are needed—which is precisely why OpenAI cut a content deal with Dow Jones, and [another](https://openai.com/index/hearst/?ref=tat.chessservices.de) last week with Hearst for more than 60 magazines and newspapers.

Current models [train](https://github.com/togethercomputer/RedPajama-Data?ref=tat.chessservices.de) on 30 trillion human words. To be Moore’s Law-like, does this scale 1000 times over a decade to 30 quadrillion tokens? Are there even that many words written? Writers, you better get crackin’.

Scaling paradox: Early indications suggest large language models may follow so-called power-law curves. Google researcher Dagang Wei [thinks](https://medium.com/@weidagang/demystifying-scaling-laws-in-large-language-models-14caf8ac6f80?ref=tat.chessservices.de) that “increasing model size, dataset size, or computation can lead to significant performance boosts, but with diminishing returns as you scale up.” Yes, large language models could [hit a wall](https://news.ycombinator.com/item?id=39917696&ref=tat.chessservices.de).

Spending paradox: Data centers currently have an almost insatiable demand for graphics processing units to power AI training. [Nvidia](https://www.wsj.com/market-data/quotes/NVDA?ref=tat.chessservices.de) [generated](https://www.fool.co.uk/2024/09/30/heres-the-growth-forecast-for-nvidia-stock-investors-through-to-2026/?ref=tat.chessservices.de) $30 billion in revenue last quarter and expectations are for $177 billion in revenue in 2025 and $207 billion in 2026\. But venture capitalist David Cahn of Sequoia Capital [wonders](https://www.sequoiacap.com/article/ais-600b-question/?ref=tat.chessservices.de) if this is sustainable. He thinks the AI industry needs to see $600 billion in revenue to pay back all the AI infrastructure spending so far. Industry leader OpenAI expects $3.7 billion in revenue this year, $12 billion next year, and forecasts $100 billion, but not until 2029\. It could take a decade of growth to justify today’s spending on GPU chips.

[Goldman Sachs](https://www.wsj.com/market-data/quotes/GS?ref=tat.chessservices.de)’s head of research wrote a report, “GenAI: Too much spend, too little benefit?” He was being nice with the question mark. Nobel laureate and Massachusetts Institute of Technology economist Daron Acemoglu thinks AI can perform only 5% of jobs and [tells](https://finance.yahoo.com/news/ai-only-5-jobs-says-130000626.html?ref=tat.chessservices.de) Bloomberg, “A lot of money is going to get wasted.” Add to that the cost of power—a ChatGPT query [uses](https://www.npr.org/2024/07/12/g-s1-9545/ai-brings-soaring-emissions-for-google-and-microsoft-a-major-contributor-to-climate-change?ref=tat.chessservices.de#:~:text=To%20generate%20its%20answers,%20AI,sophisticated,%20it%20needs%20more%20energy.) nearly 10 times the electricity of a Google search. [Microsoft](https://www.wsj.com/market-data/quotes/MSFT?ref=tat.chessservices.de) is [firing up](https://www.technologyreview.com/2024/09/26/1104516/three-mile-island-microsoft/?ref=tat.chessservices.de) one of the nuclear reactors at Three Mile Island to accommodate rising power needs. Yikes.

I’m convinced AI will transform our lives for the better, but it isn’t a straight line up.