How fast is AI really getting better?

In January 2025, researchers introduced Humanity’s Last Exam (HLE): 2,500 difficult expert questions across mathematics, natural sciences and the humanities with verifiable answers. In the official baseline table, GPT-4o scored 2.7%.

Roughly 15 months later, on 7 April 2026, Anthropic reported 56.8% without tools and 64.7% with tools for Claude Mythos Preview. How does that steep jump fit with our everyday experience that AI helps enormously on some tasks while failing surprisingly on others?

• Humanity’s Last Exam: Official project page
• HLE Paper (arXiv:2501.14249v8)
• Anthropic: Project Glasswing / Claude Mythos Preview

A benchmark is a standardized comparison test: it takes a fixed set of tasks, presents them to a model under defined rules and compares the outputs against known reference solutions.

When comparing the early 2.7% with the later 64.7%, four conditions belong to the figure:

• Experimental setup: This is a historical comparison across model generations, not an identical controlled lab experiment.
• Tools: Without tools Mythos Preview scored 56.8%; with tools such as code execution and search it reached 64.7%.
• Possible memorization: Anthropic explicitly notes that overlap with training data or solution paths online cannot be completely ruled out.
• Self-contained questions: HLE tests individual questions with fixed answers, not multi-day projects with evolving requirements.

The research organization METR therefore evaluates complete assignments across software engineering, research and security. It measures the 50% time horizon: how long does a skilled human typically need for a task on which the AI system succeeds half the time?

Three distinct clocks matter here:

1. Human task time: the scale on the METR axis (for example, a task that takes a human two hours).
2. AI runtime: how many minutes or hours the system actually spends computing.
3. Feedback delay: how long it takes in the real world before anyone can tell whether the result actually worked.

In the TH1.1 release of 29 January 2026, the estimated doubling time for the trend since 2024 was 88.6 days (just under three months), and 196.5 days in the longer historical series since 2019. A conditional example calculation illustrates the pace: if a 2-hour horizon doubles four times (2 → 4 → 8 → 16 → 32), the arithmetic reaches 32 human task hours. That is an if-then calculation, not a guaranteed forecast.

• METR Time Horizons
• METR: Measuring AI Ability to Complete Long Tasks
• METR: Time Horizon 1.1 Update (January 2026)

Why does even a long software task horizon not automatically mean AI can take over every complex decision? Because real-world tasks have very different feedback loops:

• Seconds to minutes: You fix a weekly shift table for a bakery and immediately check whether every shift is covered and the totals add up.
• Weeks: You redesign an online shop's checkout form. Whether customers actually complete orders more smoothly only becomes clear after weeks of real usage.
• Years: You choose the location for a new branch or make a long-term investment. An AI can summarize reports and model scenarios, but reality only reveals years later whether the decision paid off.