How much work can you trust an AI agent to complete on its own?
For a growing number of organizations, that question has moved beyond experimentation. Teams are using agents to write code, research problems, analyze information, and carry out tasks that once required a person at every step. As those capabilities improve, leaders need a way to decide where agents can help, where people still need to be involved, and how those decisions might change over time.
One measure of that progress comes from METR, an independent AI evaluation organization. In 2025, its researchers found that the length of software-related tasks frontier AI systems could complete at a 50% success rate had doubled roughly every seven months since 2019.
That finding sits at the center of Dr. Jon Krohn’s keynote, “Riding the Exponential: How to Thrive as Agents Double Task Horizons Every 7 Months,” at ODSC AI NYC 2026. It also raises a practical question for anyone building or managing a team: What does a faster-moving AI frontier actually mean for the work your organization does?
The answer starts with understanding what the measurement tells us—and where its limits are.
The curve has moved since the seven-month estimate
In January 2026, METR published Time Horizon 1.1, which expanded its task suite from 170 to 228 tasks. The long-run doubling time held at about 196 days, or 6.5 months. The recent trend moved. Measured from 2023 onward, the doubling time was 131 days (about 4.3 months). Measured from 2024 onward, it was 89 days (about 2.9 months). The original paper had already noted that the trend “may have accelerated since 2024.”
The same update put Claude Opus 4.5 at a 50% horizon of 320 minutes, a little over five hours, with a confidence interval of 170 to 729 minutes. GPT-5 came in at 214 minutes and o3 at 121 minutes. Mythos Preview, evaluated in March, landed at 16 hours or more.
The UK AI Security Institute found the same pattern in cyber tasks. Its February 2026 estimate put the doubling time for the length of cyber tasks models complete autonomously at roughly 4.7 months from late 2024, faster than its November 2025 estimate of eight months. In May, AISI reported that Claude Mythos Preview and GPT-5.5 substantially exceeded both trends, and it said it is unclear whether that marks a new, faster trend.
For planning, the seven-month figure reads best as a long-run average for software work, and a conservative one for recent years. METR writes that the trend “is somewhat sensitive to task composition.” The confidence intervals are wide, and the longest tasks in the suite have few human baselines.
Where the acceleration holds
METR’s suite draws on software engineering tasks, machine learning research engineering tasks, and short software-developer tasks, all scored automatically. The paper states that “time horizon is always measured relative to a domain, task distribution, and human baseliners’ level of skill and context.” Within that domain, the trend has been consistent. On SWE-bench Verified, a separate software benchmark, the paper finds the same exponential shape with a shorter doubling time, about 70 days against 143 days for METR’s combined suite.
The session description for Krohn’s keynote says the trend is “visible in coding, AI engineering, data analysis and a widening range of knowledge work.” The public measurements reviewed here concentrate on software, machine learning engineering, and cyber tasks, and they do not break out data analysis separately. The practical reading is that the curve applies first where a result can be checked automatically, such as tests that pass, a validator that accepts an output, or a reconciliation that balances.
Where it breaks down
Reliability sets the first limit. METR’s paper reports that 80% time horizons are “4-6x shorter” than 50% horizons. By that ratio, a model with a ten-hour 50% horizon would have an 80% horizon of roughly two hours. The two doubling times are similar, so the gap persists as the curve rises.
Task messiness sets the second. The same paper finds that “controlling for task length, we find models perform worse on tasks that have higher messiness scores.” Real workflows carry unclear success criteria, shifting context and other people, and the paper notes that realistic tasks differ from benchmark tasks in automatic scoring, agent interaction, and dynamic environments.
Office deliverables show the gap most plainly. The Remote Labor Index, a Scale Labs benchmark, tests agents on 240 freelance projects sourced from Upwork, and human evaluators judge whether each deliverable would satisfy a reasonable client. At launch in October 2025, the best agent, Manus, completed 2.5% of the projects. The live leaderboard now lists Opus 5.5 at 21.25% and GPT 6 Astra at 20.83%, more than an eightfold increase in about a year.
Nearly four of every five projects still fall short of an acceptable result. The Center for AI Safety, reporting on the index in July, added that fitting a time-horizon curve to the data works poorly, so the authors keep automation rate as their main metric. It also reported that an automated judge calibrated on earlier models overestimated the newest models’ scores by roughly 2.3 to 2.9 times, which is why people review every deliverable.
OpenAI’s GDPval offers a third view. It covers 1,320 tasks across 44 occupations, and on its open 220-task subset Claude Opus 4.1 produced deliverables graded as wins or ties against human experts 47.6% of the time. OpenAI’s own frontier models improved “roughly linearly over time” on it, a different shape from METR’s compounding curve. The paper is direct about scope: “Tasks are precisely-specified and one-shot, not interactive,” and when context was withheld, “the models struggled to figure out context.”
What the keynote covers
Dr. Jon Krohn presents “Riding the Exponential: How to Thrive as Agents Double Task Horizons Every 7 Months” on December 2, from 9:00 to 9:25 AM, at ODSC AI NYC 2026. His session description follows the same two-sided approach as the evidence above. It begins by “defining precisely what an AI agent is and what it’s not,” then examines the empirical evidence behind agents’ progress, “including where the trend holds, where it breaks down, and what it suggests about the next three to five years.” The final section turns to strategies for data scientists, engineers and leaders.
Krohn co-founded Y Carrot, a New York AI software company, and hosts the SuperDataScience podcast. He holds a doctorate from the University of Oxford, where he applied machine learning to genetic, behavioral, and brain-imaging data, and he wrote Deep Learning Illustrated, which has been translated into seven languages.
Three steps to prepare your workforce
You do not need to predict exactly where AI capabilities will be in five years to start preparing. You can begin by building a clearer picture of the work your team does today.
1. Map tasks, not just job titles
Choose a few recurring workflows and break them into their component tasks. Identify where success can be checked with a test, a calculation, a source, or a clear set of criteria. Then identify where the work depends more heavily on judgment, relationships, or context. For example, an agent might help a data team write a query and check it against a known result. A person may still need to determine whether the question itself reflects the business problem and whether the data is appropriate to use.
This exercise can also reveal skills worth developing across the team: writing clearer specifications, building evaluations, reviewing outputs, and identifying the context an agent needs.
2. Run a small, repeatable evaluation
Select a sample of recurring tasks across a range of difficulty and task lengths. Record how long a competent colleague takes to complete each one, define what success looks like, and run the agent multiple times under documented conditions. Have qualified reviewers assess the results, and track more than whether the agent finished. Record the time spent preparing instructions, reviewing outputs, correcting errors, and handling failures. Include relevant costs and any constraints on data or tool access.
A small pilot may not produce a statistically reliable 50% or 80% time horizon, but it can show where the agent is useful, where it struggles, and what a more rigorous evaluation should measure.
3. Revisit the results
A model that struggled with a task a few months ago may perform differently after an update—or with a better workflow around it. Revisit your sample regularly, such as once a quarter, and document changes to the model, tools, prompts, and evaluation criteria. Use those results to adjust training and oversight. If an agent becomes more capable, employees may spend less time on an initial draft or routine implementation and more time defining the problem, checking the result, and deciding what happens next.
Explore the evidence at ODSC AI NYC
The seven-month horizon is a reason to pay attention, not a reason to assume every workflow will change at the same speed. The strongest plans account for both sides of the evidence: Agents are gaining capabilities quickly, and reliable performance still depends on the task and the conditions around it.
At ODSC AI NYC 2026, Dr. Jon Krohn will explore the evidence behind that progress in his keynote, “Riding the Exponential: How to Thrive as Agents Double Task Horizons Every 7 Months.” He’ll examine what AI agents are, where the trend holds, and how individuals and organizations can prepare.
Krohn is a co-founder of Y Carrot, host of the SuperDataScience podcast, and author of Deep Learning Illustrated. He holds a doctorate from the University of Oxford, where he applied machine learning to genetic, behavioral, and brain-imaging data.
ODSC AI NYC takes place December 2–3, 2026, in New York City. Explore the preliminary program and event details.
