The AI effect: why AI keeps moving the goalpost

In February 1996, Douglas Hofstadter watched Deep Blue win one game against Garry Kasparov and told the New York Times on 19 February 1996 (linked copy), “My God, I used to think chess required thought. Now, I realize it doesn’t.”

That reaction has a name, the AI effect, and it has repeated for as long as the field has existed. A capability is called intelligent while it is out of reach. The day it works, it becomes software, and the goalpost moves.

Edsger Dijkstra put the problem in one sentence in 1984. In a keynote at the ACM South Central Regional Conference in Austin, he said that Alan Turing “thought about criteria to settle the question of whether Machines Can Think, a question of which we now know that it is about as relevant as the question of whether Submarines Can Swim.” The talk is EWD898, “The threats to computing science”. It was a remark in passing. In my words:

A submarine can’t swim. It crosses the ocean anyway.

The timeline

The timeline below runs from the first neuron model in 1943 to this year. It has eight eras, nine technology bands and 55 milestones, each with a source link. As you scroll, the goalpost moves.

Each card is labelled by what stands behind it. Where someone said it on the record, the card carries the quote and the link. Where it is my read of how a capability looks today, it says so and does not wear quotation marks. The counter counts only the documented ones.

What the pattern tells us

Three things stand out once you have seen it laid out.

• Each win gets smaller in hindsight. Deep Blue’s win in May 1997 was a milestone for the field. Within weeks, McCarthy was calling it fruit flies. The timeline repeats that sequence for speech, vision, Go, translation and code
• The goalpost is moving faster. Stanford’s SQuAD reading benchmark was released in mid 2016 and beaten by machines on 11 January 2018, about 19 months. SuperGLUE was built in 2019 specifically to be harder and was passed on the leaderboard by 6 January 2021, about 20 months. ImageNet error went from 15.3 percent in 2012 to 4.94 percent in February 2015, under the 5.1 percent estimate for humans
• The new goalposts are jagged. Stanford’s AI Index 2026 reports SWE-bench Verified rising from 60 percent to near 100 percent in a single year, and in the same report the top model reads analog clocks correctly 50.1 percent of the time. SWE-bench Verified is a curated subset, so treat the near 100 as a reported figure. The clock is the 2026 version of the old reply that this is not real thinking

Every solved problem becomes software. Every unsolved one is still called AI.

Naming the pattern

Joseph Weizenbaum described the mechanism in 1966, in the paper that introduced ELIZA. Once a program is unmasked and its inner workings explained in plain language, he wrote, “its magic crumbles away”. That is thirteen years before the effect had a name.

Two books in 1979 put it on the record.

Pamela McCorduck, in Machines Who Think, wrote that every time somebody figured out how to make a computer do something, “there was a chorus of critics to say, but that’s not thinking.”

Douglas Hofstadter, in Godel, Escher, Bach, gave us the line most people now know as Tesler’s Theorem: AI is whatever hasn’t been done yet. Credit is due to Larry Tesler, and the wording matters. Tesler’s own site gives what he said as “Intelligence is whatever machines haven’t done yet”. The popular version is Hofstadter’s rendering of it, as Quote Investigator traces. His site dates it to about 1970, which is his own recollection, and I found no print source from that time to confirm it.

The idea kept turning up in other words. In April 1985 Roger Schank and Larry Hunter wrote in Byte that “When an AI idea is turned into a useful system, in some sense it isn’t AI anymore.” Bertrand Meyer later recalled hearing John McCarthy describe the same phenomenon, which is where the often repeated “as soon as it works, no one calls it AI anymore” comes from. That line is attributed to McCarthy by Meyer. I found no written McCarthy source for it.

McCarthy did put something similar in print. In June 1997, reviewing Deep Blue’s win in Science, he compared computer chess to genetics if geneticists had spent decades breeding racing fruit flies: “We would have some science, but mainly we would have very fast fruit flies.” The man who co-founded the field was telling us the goalpost had moved.

Hofstadter did not move the goalpost out of stubbornness. He wrote one of the great books on machine intelligence, and in that 1996 interview he added that you can bypass deep thinking in chess the way you can fly without flapping your wings. He looked at how the machine did it and decided that was not what he meant by thought.

Why AI is so hard to define

The goalpost moves partly because nobody agreed where it was in the first place.

Stuart Russell and Peter Norvig organise the field’s definitions into four groups in the opening chapter of their textbook. Machines that think like humans, machines that think rationally, machines that act like humans and machines that act rationally. Four goals, and they are not the same finish line.

• Acting humanly is the Turing test, where the aim is to be indistinguishable from a person
• Thinking humanly asks whether the machine reasons the way we do, which is where Hofstadter was standing in 1996
• Thinking rationally is logic and proof, the symbolic tradition
• Acting rationally is an agent that does the right thing given what it knows, which is where most of today’s engineering lives

Turing saw the definition problem coming. In 1950 he replaced “Can machines think?” with a game and predicted that by the end of the century “the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted.” He expected the word to stretch to fit the machine. What happened is closer to the opposite. The word held still and the machine was moved out of it.

This is where the submarine earns its place. Swimming describes how a fish moves. Crossing the ocean describes what a vessel does. Debate the first one for as long as you like and the second one is already finished. I read the AI effect as the same mistake made in public. We keep asking whether the machine really thinks, which is a question about mechanism. The useful question is what it does. That matters most for anyone buying or building with it, because a capability does not need to count as AI to be worth deploying.

What to do with it

The AI effect is partly a mistake and partly a fair point. Weizenbaum was right that explaining the mechanism takes the magic out. It is also fine to say that a spam filter is not a mind. It goes wrong when the label decides whether we value the thing.

I lead enterprise AI for a living and build Pelaris on the side, and I watch the same thing happen on both. In my experience nobody in a service desk calls the thing that resolves their ticket artificial intelligence. They call it the virtual agent. A virtual agent is no less useful because someone can explain how it works. At Orica, our virtual agent took the IT service desk’s deflection rate from 18 percent to 94 percent. Nobody on that desk is waiting for it to count as AI.

A few habits follow if you are planning with this:

• Judge the system by the outcome it produces. Whether it counts as AI is a debate for the philosophers
• Expect today’s benchmark to saturate. If a task is hard enough to be a goalpost, plan for it to be cheap software within a couple of years, and read what falling cost does to that bet
• Watch where the humans move. When a capability turns into software, people shift to the judgment around it, which is the shape I described in the missing middle
• Let the citizen builders in. The more capabilities become ordinary software, the more people can assemble them, and vibe coding in the enterprise is about doing that with guardrails

Where this leaves us

Look at where the timeline ends. In January 2026 METR measured the length of task agents can finish at 50 percent reliability and found it doubling every 131 days since 2023, with Claude Opus 4.5 at about 5.3 hours of human-equivalent work. A year earlier, METR put Claude 3.7 Sonnet at about an hour. The reliability caveat is real, and the same AI Index that reports the leaps reports the clock. The line is still jagged.

Here is my read, and it is a view, not a forecast. In 1943 McCulloch and Pitts modelled a neuron as a logic gate. More than eighty years on, agents work through multi-hour tasks, and we already file most of the earlier miracles under software. If this is where we have come from and where we are, imagine where we will be.

I am excited about it. The goalpost will keep moving, and I would rather be pushing it than debating whether it counts.

---

Written by Bradley Hunt. For more on AI-first building and enterprise architecture, see the writing index.