📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
AI systems have achieved near-complete automation of engineering tasks, with benchmarks indicating saturation. However, AI’s ability to fully automate research remains uncertain, leaving a residual gap. This shift could reshape AI development timelines.
Recent research indicates that AI systems are now capable of automating the majority of engineering tasks related to AI development, while the automation of research activities remains an open question, with significant implications for the future of AI progress.
Thorsten Meyer, analyzing Jack Clark’s latest work, reports that multiple AI capability benchmarks—covering research reproduction, Kaggle competitions, and kernel design—are approaching saturation or have been effectively ‘solved.’ For instance, the CORE-Bench, which measures the reproducibility of research papers, has reached 95.5% reliability, with the author calling it ‘solved.’ Similarly, the MLE-Bench, assessing performance on Kaggle competitions, has seen AI reach roughly two-thirds of human-level performance, with the leaderboard paused for adjustments as capabilities outpace previous benchmarks.
Clark’s analysis suggests that AI’s engineering skills are now largely automated, reducing the bottleneck in AI development to the residual challenge of automating research activities, which may be inherently more complex and less amenable to automation. The evidence indicates a rapid progression in engineering automation, driven by advances in kernel design, code optimization, and reproducibility, while research remains a less certain frontier.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
1000 AI Tools Directory 2026: The Ultimate Guide to AI Tools for Business, Productivity, Content Creation, Marketing, Coding, Design, Research and Automation
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
![Claude AI for Beginners Bible: [5 in 1] The Ultimate Guide to Automate Your Work, Save Hours Every Week, and Use AI for Real-World Results](https://m.media-amazon.com/images/I/415+fSJacsL._SL500_.jpg)
Claude AI for Beginners Bible: [5 in 1] The Ultimate Guide to Automate Your Work, Save Hours Every Week, and Use AI for Real-World Results
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
The Empirical Research & Laboratory Operations Logbook: A Forensic Framework for Methodological Reproducibility and Data Traceability
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
LAFVIN AI Chatbot Kit for ESP32-S3, Preloaded OpenAI & Deepseek Voice Assistant Projects, Voice Wake-up & Real-time Interruption, Suitable for Learning AI and IoT Projects.
【POWERFUL ESP32‑S3 CONTROLLER】Built‑in Xtensa 32‑bit LX7 dual‑core processor, 512KB SRAM, 8MB PSRAM, 16MB Flash for stable AI voice…
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications of Engineering Automation for AI Development
The rapid automation of engineering tasks could accelerate AI development cycles, reduce costs, and shift the focus towards higher-level research and innovation. If research automation remains limited, it could slow the overall progress, creating a residual gap that needs to be addressed. This development challenges assumptions about the timeline for achieving full AI autonomy in research and engineering, with potential impacts on industry, academia, and AI governance.
Recent Advances in AI Capabilities and Benchmarks
Over the past 18 months, multiple independent benchmarks have shown that AI systems are rapidly approaching or have achieved saturation in core engineering skills relevant to AI R&D. The CORE-Bench, measuring research reproduction, improved from 21.5% to 95.5% in fifteen months. The MLE-Bench, evaluating Kaggle competition performance, rose from 16.9% to 64.4% over sixteen months. Concurrently, research papers demonstrate ongoing advances in kernel design, including automated GPU kernel optimization and code generation, indicating that engineering automation is transitioning from experimental to production-grade.
Clark’s analysis emphasizes that the pattern across these benchmarks suggests that engineering automation may be nearing completion, while the residual challenge remains in automating the creative and investigative aspects of research, which may involve different cognitive and structural hurdles.
“The pattern of progress across multiple benchmarks indicates that engineering automation is approaching saturation, but research automation remains uncertain.”
— Thorsten Meyer
Unresolved Questions About Fully Automating Research
It remains unclear how much of AI research can be automated, given the complex, creative, and investigative nature of research activities. While engineering tasks are approaching full automation, the structural and cognitive differences in research mean that progress may be slower or fundamentally different. The timeline for achieving comprehensive research automation is still uncertain and subject to ongoing developments.
Next Steps for AI Automation and Research Progress
In the coming 32 months, the focus will likely shift toward refining engineering automation tools and exploring the limits of automating research processes. Researchers and organizations should monitor emerging benchmarks, invest in understanding the structural differences between engineering and research, and prepare for potential breakthroughs or bottlenecks. Policymakers and industry leaders should consider how these developments could impact AI development timelines and strategic planning.
Key Questions
How close are AI systems to fully automating AI research?
While engineering tasks are nearing full automation, the automation of research activities remains uncertain and likely slower due to the complex, creative nature of research.
What benchmarks show AI’s progress in automation?
Benchmarks such as CORE-Bench for research reproduction and MLE-Bench for Kaggle competitions demonstrate rapid progress, approaching saturation or being effectively ‘solved.’
What are the implications for AI development timelines?
If engineering automation continues at this pace, it could significantly accelerate AI development, but residual research challenges may temper overall progress and timelines.
Could research automation change the AI field?
Yes, if research automation advances significantly, it could transform how AI is developed, potentially reducing the time and cost of innovation but also raising new questions about oversight and control.
Source: ThorstenMeyerAI.com
