AI Output Needs Oversight: What The Studies Mean For The Workplace
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: AI Output Needs Oversight: What The Studies Mean For The Workplace on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on office and shipping supplies

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

TL;DR

Research cited by Georgia Tech cognitive engineer Karen Feigh shows that AI can save time on some tasks while adding work or reducing accuracy on others. A randomized trial found experienced developers took 19% longer with AI tools, while other experiments reported faster work within AI’s capabilities and more errors outside them. The findings support measuring review and correction alongside first-draft speed.

Georgia Tech cognitive engineer Karen Feigh argued in a November 16, 2026, essay that workplace AI can create a “verification burden”: workers spend time and mental effort checking whether polished AI output is accurate and appropriate. Research she cited found mixed results, including a randomized software trial in which experienced developers took 19% longer with AI tools, underscoring why employers need to measure checking and correction, not just how quickly AI produces a first draft.

In the METR randomized trial, 16 experienced open-source developers completed 246 real tasks in large software projects. Tasks were randomly assigned to allow or disallow AI. In the study’s February-to-June 2025 setting, developers mainly used Cursor Pro and Anthropic’s Claude 3.5 or 3.7 Sonnet. METR reported that developers took 19% longer on average when AI was allowed. The result measured task completion, including work after code review, rather than only the time needed to generate an initial draft.

The study also found a gap between perceived and recorded performance. Before the trial, developers expected AI to reduce their completion time by 24%; afterward, they estimated a 20% reduction despite the measured slowdown. Developers accepted fewer than 44% of AI generations, and 75% said they read every line of AI-generated code. On a subset of 44 tasks with usable screen recordings, about 9% of working time went to reviewing and cleaning AI output; another 4% went to waiting for the AI.

The results differ across work and task types. In a preregistered experiment involving 758 knowledge workers with Boston Consulting Group, researchers reported that AI users completed tasks within the tools’ capabilities 25.1% faster and completed 12.2% more tasks. On a complex task set outside those capabilities, AI users were 19% less likely to produce a correct solution. A separate experiment with 2,784 people found that participants often caught typos in AI-style suggestions but achieved only 31% accuracy on errors requiring them to understand task rules.

At a glance
reportWhen: Feigh’s essay was published November 16…
The developmentA November 16 essay by Georgia Tech’s Karen Feigh links workplace AI research showing that productivity gains can be offset by verification work and mistakes.

AI and work · explainer

The verification loop: where AI’s time savings can go

The draft is fast. Knowing whether it is right is the work that follows.

Time for one task (schematic)

AI draft Checking Fixing Re-checking What a speed metric sees What the work costs

Draft: secondsgood and bad parts in the same confident voice About 9% of the timewent to reviewing and cleaning AI output in METR’s trial with AI allowed Speed is not productivitycount the whole loop, errors included

How the hidden cost builds up

  1. DraftThe AI answers in seconds, fluent and confident“few clues about where scrutiny is needed”
  2. CheckDecide what to check and how, without knowing if or where an error iscognitive load
  3. FixFinding an error does not end the work, it starts a correction cycletime
  4. Re-checkEvery fix needs checking again; when nothing is wrong, the effort feels wastedmotivation
  5. Count it allProductivity is the whole human-AI workflow, not the first draftnet, not gross
The real measure: “whether the entire human-AI workflow produces trustworthy work at a lower total cost”

Source: Karen Feigh, Georgia Tech, The Conversation, Nov 16, 2026 (the three burdens, the quotes); METR, arXiv 2507.09089, p. 20 (time spent reviewing and cleaning AI output). The bars are a schematic, not data.

Graphic: Powered by Thorsten Meyer AI

Why Review Time Changes the Productivity Picture

The findings matter because a quick, fluent answer is not the same as a correct, usable result. If workers must inspect, test, rewrite or reject AI output, those activities belong in any account of the tool’s effect on productivity. Counting only the time to generate a draft can make a workflow appear faster than the full task actually is.

Errors can also be difficult to detect. In the experiment with 2,784 reviewers, performance was much weaker on mistakes requiring understanding of the rules than on visible typos. That suggests that proofreading alone may not be enough: some outputs need a reviewer with relevant subject knowledge, time to check the reasoning and access to sources or tests that can verify it.

Feigh’s concern is about the division of work and responsibility. People may remain accountable for decisions made using AI even when the tool’s confidence does not indicate which parts are wrong. For employers, the research points toward evaluating complete workflows, including corrections and consequences, rather than treating adoption or output speed as evidence of a productivity gain.

KitchenAid Artisan Mini Tilt-Head Stand Mixer, 3.5 Quart, Aqua Sky Blue

KitchenAid Artisan Mini Tilt-Head Stand Mixer, 3.5 Quart, Aqua Sky Blue

De'Longhi Magnifica Duo Fully Automatic Espresso Machine with Milk Frother, Silver

De'Longhi Magnifica Duo Fully Automatic Espresso Machine with Milk Frother, Silver

As an affiliate, we earn on qualifying purchases.

Different Studies, Different Task Boundaries

Feigh’s essay in The Conversation is an expert analysis, not a new experiment. It brings together research on AI use across software, consulting and professional writing. The results should not be treated as a single estimate for every job: studies differed in participants, tools, tasks and measures, and their findings show that performance depends in part on whether a task fits the system’s capabilities.

For example, a 2023 study by Shakked Noy and Whitney Zhang, published in Science, found that ChatGPT users completed mid-level professional writing tasks 40% faster and produced work rated 18% higher in quality. That result describes those tasks and study conditions; it does not establish the same gains for other work. Likewise, the METR software trial tested a specific group of experienced developers using particular tools on mature projects. METR said its findings do not show that current AI systems are useless in other economically relevant settings or rule out future gains.

Feigh connects the oversight problem to a longstanding concern in automation research: humans may retain responsibility while automated systems exercise practical influence over the work. Her framing is that workers are learning to supervise a fast, polished intern whose confident output can be both useful and flawed. That is an analogy, not a result measured by the cited trials.

“AI’s productivity gains are often countered by time and mental energy spent determining whether the output is accurate and appropriate.”

— Karen Feigh, Georgia Tech cognitive engineer, in The Conversation

Where the Findings May Not Apply

The studies do not establish one overall productivity effect for workplace AI. The METR trial involved 16 experienced developers, specific coding tools and tasks in mature projects; its result may not transfer to other roles, tools or later systems. The consulting experiment distinguished tasks within and outside a defined capability frontier, but its results do not tell employers precisely where that boundary lies for every worker or assignment.

The supplied evidence also does not quantify the total cost of verification across workplaces, including the effect of training, experience or changes in workflow. Nor does it show how often reviewed errors reach clients or affect consequential decisions. Those questions remain open; the reported figures describe the particular experiments and measures, not universal workplace outcomes.

Measure the Full Work Cycle

The research described here does not announce a shared policy or a single next milestone. Feigh’s practical recommendation is to measure the entire workflow: time spent prompting, checking, testing, editing and correcting, alongside accuracy and output quality. Employers can compare AI-assisted tasks with existing methods under clearly defined conditions rather than assuming faster generation means faster completion.

Organizations also need to decide who reviews AI output and whether that person has the subject knowledge, time and tools to do it. Further studies across occupations, tasks and newer systems will be needed to establish when AI produces reliable net gains and what level of oversight those gains require. Until then, the cited evidence supports a measured approach: treat AI output as work requiring evaluation, and include that evaluation in productivity calculations.

Key Questions

Did the METR study find that AI always slows workers down?

No. It found that 16 experienced developers took 19% longer on average with AI allowed on the trial’s software tasks. METR said the result does not show that AI is unhelpful in other settings.

How can AI make some tasks faster but increase errors on others?

The cited consulting experiment found faster completion on tasks within the AI tools’ capabilities, but lower accuracy on one task outside them. The results suggest that performance can depend on whether a task matches the tool’s abilities.

What does “verification burden” mean?

Feigh uses the term for the time and mental effort workers spend checking whether AI output is accurate and appropriate, then correcting or rejecting it when needed.

What should employers measure when evaluating workplace AI?

They should assess the full task, including review, testing, editing and correction, as well as completion time, accuracy and quality—not only how quickly the system generates a draft.

Source: ThorstenMeyerAI.com

This content is for general information only and is not financial, tax or legal advice. Consult a qualified professional for decisions about your money.
You May Also Like

Global tech stocks fall as Asian memory chipmakers hammered

Global technology equities declined sharply amid heavy losses in Asian memory chipmakers, impacting markets worldwide.

14 Mechanical Keyboards To Match Your Work And Gaming Needs

A 14-keyboard roundup compares layouts, switch feel, wired and wireless use, and features for different work and gaming setups.

Kill-Switch-Proof: How to Build So Washington Can’t Take Your AI Stack Down

US government shutdowns of top AI models highlight the need for self-hosted, configurable AI stacks. Here’s how organizations can build kill-switch-proof systems.

Prepare For The AI Leap: 10 Trends In 2026

An analysis of the 10 key trends shaping AI in 2026, including technological advances, industry impacts, and future developments.