Simon Obstbaum is a software engineering researcher at Stanford University, specializing in measuring and improving software engineering productivity with modern methods, including AI and large language models
Nov 23, 2025
•
5 min read
Prioritize architecture over implementation. AI excels at code generation but fails at high-level design. A clean, flexible architecture is the key skill for enabling rapid adaptation to future business needs.
Master AI tools for a competitive edge. Early adoption of advanced AI workflows (pipelining, agents) creates a compounding productivity advantage, making engineers 2-5x more effective over time.
AI research confirms clean code boosts productivity for both humans and AI. This finding highlights a common paradox: companies often clean up codebases for AI agents but not for their human engineers, despite identical benefits.
A new research paper on AI adoption is embargoed until Saturday. The paper, based on 120k engineers, will be presented at the AI Engineer conference in NYC.
Question: What skills should new engineers (born 2002-2003) develop to stay relevant in the next 2-5 years?
Core Principle: Continuous learning is the most valuable skill. The ability to quickly adapt to new technologies is more important than mastering any single tool.
AI's Role: While AI can generate code, engineers must retain a deep understanding of the underlying logic to effectively debug, adjust, and validate AI-generated output.
Problem: AI excels at solving individual coding problems but is poor at high-level architectural design.
Solution: Engineers must master architecture to design flexible systems that can adapt to changing business requirements.
Key Distinction:
Over-engineering: Building for a scale far beyond current needs (e.g., millions of users for a 10k user app).
Flexible architecture: Designing a system that can be adjusted to scale later, without overbuilding for it now.
Rationale: A clean architecture allows for messy, fast-paced initial implementation. The clean structure enables targeted refactoring of only the parts that need to scale later.
Example: Crunchyroll's monolithic architecture became a major bottleneck when its user base grew from a projected 500k to 15M+, proving the cost of inflexible design.
Simon shared embargoed findings from a new research paper (co-authored with Yegor Denisov Blank, Stanford) based on data from 120,000 engineers.
Finding 1: The Rich Get Richer
Early adoption of advanced AI workflows (pipelining, agents) creates a compounding productivity advantage.
Engineers who master these techniques can become 2x, 3x, or even 5x more productive over time.
Finding 2: Smart Spend Over High Spend
High token spend does not guarantee high productivity.
There is a "valley of death" where spend is high but gains are low.
Insight: Providing better context to LLMs is more effective than simply increasing token volume.
Finding 3: Clean Code Benefits Both Humans & AI
AI agents perform significantly better in clean, well-tested codebases.
Paradox: Companies are investing in code cleanup for AI, but often resisted doing so for their human engineers, despite identical productivity benefits.