Is AI Actually Getting Smarter?

Hey folks,

It’s Wednesday, and time for a new Deep Dive and Analysis.

Every new AI release seems to come with the same message:
smarter, better, more capable than before.

Benchmarks improve. Demos look sharper. Answers feel more confident and well-structured.
It’s natural to assume this means AI is becoming smarter in a broad, human-like sense.

But that assumption deserves a closer look.

What Do We Mean by “Smarter”?

When people say AI is getting smarter, they usually mean things like:

• Better understanding

• Better reasoning

• Better judgment

• Better handling of unfamiliar situations

These are human notions of intelligence.

In contrast, AI progress is typically measured using:

• Task accuracy

• Benchmark scores

• Performance on specific evaluations

Right away, there’s a mismatch.
We’re using one word “smarter” to describe two very different ideas.

Where AI Has Clearly Improved

Before questioning the narrative, it’s important to acknowledge real progress.

Modern AI systems are undeniably better at:

• Producing fluent, coherent language

• Following instructions

• Handling multi-step tasks

• Writing and refactoring code

• Retrieving and combining information

These improvements are real, measurable, and useful.
AI today can do things that were not possible just a few years ago.

This isn’t stagnation. It’s progress.

What’s Driving These Improvements

The main drivers behind recent gains are well understood:

• Larger models

• More training data

• More compute

• Better fine-tuning and evaluation methods

Crucially, these improvements come from scale and optimisation, not from AI systems developing new internal forms of understanding, intent, or awareness.

The models are better trained, not fundamentally different in how they operate.

Better-Trained vs Smarter

This distinction matters.

Modern AI systems:

• Learn statistical patterns from large datasets

• Generate outputs by predicting likely next tokens

• Do not form beliefs, goals, or internal models of truth

When an AI produces a well-reasoned answer, it’s generating text that resembles reasoning, based on patterns it has seen, rather than performing reasoning in the human sense.

In simple terms:

Why AI Still Fails in Surprising Ways

This helps explain a familiar experience.

AI can:

• Explain complex topics clearly

• Sound confident and authoritative

Yet it can also:

• Be confidently wrong

• Contradict itself when questions are rephrased

• Struggle with edge cases

• Miss obvious constraints unless explicitly stated

If AI were becoming “smarter” in a general, human-like way, these failures would diminish more consistently. Instead, they remain a predictable part of how these systems behave.

Why Benchmarks and Demos Amplify the Perception

Benchmarks and demos play an important role, but they show a narrow slice of reality.

Benchmarks:

• Test specific, well-defined tasks

• Reward performance under controlled conditions

Demos:

• Showcase best-case scenarios

• Avoid ambiguity and edge cases

Real-world use is very different. It’s open-ended, messy, and full of incomplete information, conditions where current AI systems are much less reliable.

This gap makes AI feel smarter than it often is in practice.

So, Is AI Actually Getting Smarter?

The most accurate answer is a nuanced one:

• AI is getting better at many tasks

• It is not becoming smarter in a human-like, general sense

• The progress is real, but different from how people intuitively imagine intelligence

Understanding this distinction helps set realistic expectations.

Why This Distinction Matters

Overestimating AI intelligence leads to over-trust.
Over-trust leads to poor decisions.

When people understand how AI improves, and where its limits remain, they can use it more effectively, more safely, and with better judgment.

AI progress is impressive.
But intelligence is more than fluent answers and rising scores.

The real challenge isn’t whether AI is getting smarter.
It’s whether we’re getting better at understanding what AI actually is.

Curated Prompt Library

Kickstart any task with our Curated Prompt Library, packed with ready-to-use prompts for everyday work. You can also create your own custom prompts, save them privately, and share them with your team, so everyone stays consistent, faster, and on the same page.

Why Your AI Costs Keep Increasing (And How to Fix It)

Cheers,

Keval, Editor