What makes a business succeed? Many frameworks reduce it to some variation of three things:

Market – You are competing somewhere worth winning. It is better to be a mediocre company in a great market than a brilliant company in a dying one.

Strategy – You differentiate in the right way and get the product into customers’ hands before someone else does. You give customers a reason to choose you and a reason to stay.

Execution – You repeatedly deliver higher quality at lower cost. You improve faster than the competition. You turn learning into a system.

Different eras reward different strengths. Buffett built Berkshire largely around market selection and capital allocation. Salesforce built an empire around strategy and workflow entrenchment. Toyota turned execution into a quasi religion.

For the last twenty years, execution has often been the quiet third variable. Many companies succeeded with strong markets and strong strategy while execution ranged from excellent to surprisingly mediocre. High margins, venture capital, network effects, and digital distribution covered a lot of operational inefficiency. The lesson many organizations internalized, even if they never said it explicitly, was that execution mattered, but it did not matter as much as being in the right market at the right time with the right positioning.

And that’s when AI showed up…

AI reduces the cost of doing work. That part is obvious and widely discussed. What is less discussed is that AI also reduces the cost of learning. Teams can build faster, but they can also gather feedback faster, synthesize it faster, analyze failures faster, generate alternatives faster, and iterate faster. The build loop accelerates and the learning loop accelerates. Both sides of the equation improve at the same time.

Because of that, execution should be thought of less as effort or culture and more as a measurable capability:

Execution is the number of high-quality build-and-learn cycles an organization can complete in a given period of time.

Once you think about it this way, the impact of AI becomes clearer. AI increases the number of cycles you can run AND improves what you learn from each cycle. The organizations that benefit most will not necessarily be the ones with the best strategy documents or the most ambitious AI roadmaps. They will be the organizations that can run more high-quality cycles than everyone else.

Many in the venture capital industry have been beating the drum making similar points. Marc Andreessen beats the drum on OODA loops (observe, orient, decide, act). At a recent investor panel, I heard Ethan Choi of Khosla Ventures repeatedly emphasizing that the primary attributes they look for in companies were the founding team’s ability to execute with velocity.

This is where the idea of the gingerbread man moat comes in. The gingerbread man does not win by being bigger or stronger or cuter. Instead, he wins by being difficult to catch. He keeps moving, and when he stops moving, he gets eaten. For many industries right now, that is the situation. Traditional moats still exist. Regulation still matters. Brand still matters. Distribution still matters. Capital intensity still matters. Data still matters. But velocity (execution) is becoming a moat available to far more companies than before.

The companies that can iterate, learn, and improve faster will feel very difficult to compete against because they keep widening the gap. Most organizations are not built for that.

The Recipe for Velocity

When you look closely at teams that execute quickly, a pattern shows up repeatedly. They tend to have more decision makers (without complete decentralization) with real authority operating closer to the work. That sounds simple, but it requires three things to be true at the same time.

Empowered – They are allowed to make decisions and they have the resources to act on those decisions. Authority without resources results in flailing. Resources without authority create frustration. Both authority and resources are required.

Aligned – They are accountable for the outcome, they believe in the direction, and their incentives point in the same direction as the organization. They receive feedback not only on the product, but also on their own decisions and performance so they can improve along with the system they are building.

Informed – They are close to the user, the product, and the technology. They understand the problems and the edge cases. They are NOT operating entirely through dashboards and summaries. They have proximity to reality, and they still bring a magnifying glass.

Empowered, Aligned, Informed. Each is necessary, and none is sufficient alone.

There is also a surprisingly simple way to get a rough sense of whether an organization is structured this way. Look at the meetings. Do not count the number of meetings. Instead, count the number of attendees in the meetings.

Meeting count is a noisy metric. A team can have many short meetings and still move quickly. Or, a team can have collaborative four hour working sessions with three people. The more revealing signal is how many people need to be in the room for a quality decision to happen. When empowerment, alignment, and information are concentrated in the right people, meetings tend to be small. When those things are fragmented across layers and functions, committees appear, either formally or informally. Committees meet because they have to. If decisions require pieces that live in different places (empowerment, alignment, or information), those pieces have to be assembled regularly. Often, they fail in their attempt to assemble each piece. The more fragmented, the more people are necessary and the likelihood increases that a critical component is missed.

Large recurring meetings are often a structural signal that suggests that components are distributed across the organization in a way that slows decision making and iteration. And now with AI, iteration speed is increasingly the game.

To compress this argument to three points:

• AI has tilted the environment back toward execution. Not exclusively toward execution, but enough that execution capability is becoming more important relative to market selection and high-level strategy than it has been in a long time.

• Many organizations optimized for the previous environment where market position, strategy, and access to capital carried more weight. They built structures that were good at planning, coordinating, and managing risk across large organizations. Those are still valuable capabilities, but they do not produce high iteration velocity.

• The organizations that will do well in this environment are the ones that can empower the right people, align them to outcomes, keep them informed and close to reality, and then allow them to run many high-quality build-and-learn cycles.

Many organizations do not have those strengths and are not maniacally pursuing them. Over the next 24 months, that gap will weigh more than the pile of AI-written strategy documents attempting to dig the company out of its hole.

Architected Intelligence: Principles for Building AI-First Organizations and Technologies by Jacob Miller and Jeremy Mumford is available now. Published by Wiley, 2026.

A few years ago, I made the decision to change my title to AI Engineer. At the time, it was a decision that carried risk. Other new and trendy titles like “Prompt Engineer” were in vogue. I felt strongly that AI engineer was the descriptor that was meant for me, and time has proved me right. Prompt Engineer got laughed out as a legitimate title, but AI Engineer has managed to cement itself as a new fixture in companies. LinkedIn’s 2025 Jobs on the Rise report named Artificial Intelligence Engineer as the #1 fastest growing job position.

But what is an AI Engineer? The job report describes it as someone who will “design, develop and implement AI models and algorithms to optimize processes and solve complex problems.” Here’s the problem with that statement. Designing + developing an AI model versus implementing it to solve business problems are two completely different jobs! One of those individuals makes $3 million a year at Anthropic to design the next Claude model, and the other is on the frontlines solving business problems using generative AI as their primary tool of choice. This conflation of roles has caused many folks on the internet to crash out, decrying how “Back in my day, AI engineers actually knew how AI models worked”. And that is a fair point. Most of the roles you see open for AI engineers have skill requirements that look suspiciously like software engineers.

So I want to put my hot take out there: People who build AI models are NOT AI engineers. Call them anything else: ML engineers, data scientists, AI researchers, NLP specialists, MLOps engineers, LLM Ops engineers, LLM engineers, etc. But most AI engineer positions you see open on LinkedIn are not asking you to build an AI model, or even tinker with one. Why? Because 99% of companies believe they can gain more ROI from leveraging existing foundation models rather than hiring experts that actually know how to build and fine-tune LLMs. The assumption is that the foundation models from the Big Three (OpenAI, Anthropic, and Google)1 are so good that there is no point in trying to compete. It’s easier to just plug into what they have and focus on context engineering.

So if AI engineers are not defined by building AI models, what does define them? Is it just calling the OpenAI API? The role of an AI engineer is defined in my mind with the following statement:

Think like a data scientist,

build like a software engineer

Much of the day to day work of an AI engineer looks suspiciously like a software engineer. You are writing application code or backend services that support applications. These are hard jobs, but they aren’t new. The key difference between the AI engineer and the software engineer is that non-deterministic generative AI is a critical component in an AI engineer’s applications and platforms. Why is this significant? Let’s look at the difference between software engineers and data scientists. Software engineers can feel confident that if it works in production on day one, it will likely work in production on day 20. Data scientists do not have that same luxury. The below graph that I shamelessly stole from the internet (thanks Eric) illustrates this difference in confidence.

AI engineers straddle both disciplines. And that’s what makes it an interesting job. You need to know BOTH sides of the coin.

On the software engineer side, you need to:

• Build scalable infrastructure with IaC and CI/CD

• Create codebases that can handle dozens or hundreds of developers working simultaneously

• Navigate micro-service cross-service chaos with modern observability practices, attention to security, and thoughtful API design

• Provide real value to users (the most simple but oft overlooked skill)

On the data science side, you need to:

• Handle non-deterministic behavior

• Use benchmarks and A/B testing

• Identify trade-offs between various approaches

• Understand how AI and data combine to solve previously impossible problems

At the intersection of both of these are skills like:

• Knowing when AI is actually a subpar solution that could be replaced with deterministic code logic

• Staying on top of an avalanche of tech updates that are obscured by marketing and internet hype bros

• Evaluating when to build vs self-host vs buy, and identifying when to re-evaluate and change those decisions in a light-speed industry that constantly evolves

• Diffusing AI knowledge across the broader organization

So to repeat: AI engineers blend the skillset and daily activities of a software engineer with the mindset and caution of a data scientist. They focus on building services that leverage generative AI, understanding that they need to follow the best practices of building production worthy applications while also applying the rigorous standards of testing in the field of data science. While not every job posting looking for an AI engineer will actually match what I’ve described, this is the standard I believe it’s solidifying around, and what every AI engineer should strive for.

Quick plug: A lot of the things an AI engineer needs to know are outlined in the book I just wrote with Jacob Miller, Architected Intelligence. We directly address the skill gap that AI engineers have as they get pulled in from either the software engineering or data science side.

The bottom line: Think like a data scientist, build like a software engineer. That is the role and purpose of an AI engineer. Starting with that mantra will create a powerful foundation that will pay dividends moving forward.

That performance gap is what drove Jeremy Mumford and me to write Architected Intelligence: Principles for Building AI-First Organizations and Technologies, published by Wiley, and now available for purchase.

We are aware of the irony of publishing a physical book about the most transformational technology since the printing press. We have made peace with it.

It takes a certain amount of hubris to write a book. We know the AI community didn’t need one more AI title with a glowing robot head on the cover and a subtitle boldly claiming that we’re “unlocking tomorrow” with a “blueprint of the future.” We wrote Architected Intelligence because over the last few years we kept seeing the same modes and patterns of failure and a recurring single framework for success.

Subscribe now

Before we dive into the patterns of success, we’re certain you recognize the failures. Product teams bolted on AI summaries with a nice ✨emoji and called it innovation, while the technical teams built unscalable but impressive demos destined for the proof-of-concept graveyard. Leadership would then get frustrated as they see the wild progress on social media and make a decision to “shake things up.” However, instead of a new bold “AI strategy,” what these companies really needed was a way to think clearly about value, data, models, trust, and organizational execution as one connected system.

These companies needed a structure to architect with artificial intelligence.

The Chaotic Early Days

“The art of progress is to preserve order amid change and to preserve change amid order.” — Alfred North Whitehead

Think back to when ChatGPT launched on November 30, 2022 and the internet promptly lost its mind in the ensuing months. We were in the trenches at Pattern, navigating a high-growth ecommerce environment attempting to shape billions in revenue while our feet were being swept away in the AI tsunami.

What followed was a strange period (in retrospect) with an odd combination of enormous promise, small context windows, frequent model outages, pricey API calls, and groundbreaking but still limited practical intelligence. The technology was historic, but we all were struggling to turn it into something that mattered and created something valuable. The early constraints turned out to be useful because they forced discipline.

The traditional siloed disciplines were failing the AI transformation. In one corner, software engineering didn’t know how to handle the non-deterministic nature of LLMs, and in the other corner data science was too slow and opaque for the “move fast and break things” pace of generative AI, and knowledge management wasn’t found in any corner at all because the discipline was mostly forgotten in an “agile” world.

We needed a unified theory.

A Framework Born in the Trenches

As we worked with the first wave of generative AI tools, we found ourselves pulled into a recurring set of five questions.

• What exactly are we trying to produce?

• What data is actually needed to make that output valuable?

• What model behavior matters for this use case?

• How do we know whether the system is trustworthy in production?

• What is the path to end the “wave of experiments” and begin to scale?

Those questions eventually solidified into a simple interdisciplinary framework. It drew from software engineering, data science, data engineering, product thinking, and knowledge management. It was useful and gave us a way to build and communicate real AI products and processes. The result was a multidisciplinary approach that we call Architected Intelligence.

We further built out our framework with the builders and thinkers we admired. The Andrej Karpathy and Ethan Mollicks of the world, along with the research posted by the leading AI labs and the individual researchers and engineers posting publicly from within them. We tested these concepts in the messier places where ideas either become useful or get exposed quickly: internal planning, system design, conference talks, consulting engagements, vendor discussions, customer conversations, AI meetups, roadmap fights, and postmortems. That broader feedback loop mattered.

While we received strong encouragement and saw positive initial results, we also received sharp criticisms from smart people, further refining our frameworks and principles. To us, the tipping point was when people more brilliant than us could take our structure, adapt and apply it to their own domain, and suddenly move faster. It helped teams break down what had felt like a giant, foggy AI ambition into concrete parts they could actually reason about, communicate, and build.

That was the moment we started taking the idea of a book more seriously, so we partnered with the Wiley team to bring you a book and lay out this framework for you.

The Five Components of Architected Intelligence

Successful AI products and processes consistently execute on five components, in order:

Output — In a world where execution gets cheaper, judgment compounds. What valuable experience are you designing? What is the exact value that each AI call is responsible for? Every downstream technical decision traces back to this.

Input — Power the system with high-quality, “canon” knowledge and differentiated data delivered in the moment it’s needed. This is the AI system’s fuel. Without unique data, your brilliance will be copied and commoditized.

Model — The model that you are leveraging is the engine and NOT the vehicle. Context engineering, fine-tuning, model selection, and cost-latency-intelligence tradeoffs are where technical decisions get made, but only after you’ve defined the output and prepared the input.

Observability — Trust is the currency of AI adoption, and it’s fragile. You need to know what your system is doing, why it’s doing it, and whether that remains acceptable as conditions change. Traditional software observability and data science evaluation each solve half the problem. You need both.

Enablement — The graveyard of AI is full of great prototypes that never scaled. Enablement is the infrastructure, talent, culture, and process that transforms successful experiments into durable organizational capability.

These five components form the structure of the entire book. Each stands on its own and you can open to the chapter most relevant to your immediate problem and apply it. We believe, however, that the real payoff comes from the full picture when you see and understand how a failure in Input undermines a perfectly engineered Model and how gaps in Observability silently destroy trust in an otherwise valuable product.

A Book Built to Last

We wrote the book we wish we’d had as we built our AI platform, products, and processes. It’s grounded in what successful AI projects have in common, whether they were built on GPT-3.5 or on models that didn’t exist six months ago. The principles that govern how you define valuable output, acquire and curate data, configure models, maintain observability, and build enabling infrastructure don’t change when the underlying models do. That’s the bet we made, and we think it holds.

Who This Is For

The primary audience is technical leaders. This includes:

• engineering leaders responsible for delivering AI projects that actually work,

• technical advisors helping organizations navigate transformation

• technology leaders charting the map between their AI ambitions and their team’s current ability to execute

However, we also wrote it so that product leaders and others working close to AI could follow the arc and develop the core intuition for what quality AI projects require. If your roadmap is reliant on blind trust and crossed fingers and you’re looking for a remedy, this book was written for you.

For the Builders

In the end, this book is for the people who we believe will inherit the AI-first future. The builders. The ones who eschew the hype cycles and simply want to create something beautiful and valuable with this extraordinary mass of matrix algebra, computation, and data that has ushered in a new era.

That’s the spirit behind both this newsletter, The Why Behind AI, and our book Architected Intelligence. We hope this newsletter and this book help you architect yours.

Architected Intelligence: Principles for Building AI-First Organizations and Technologies by Jacob Miller and Jeremy Mumford is available now. Published by Wiley, 2026.

There has been significant recent hype around the latest AI models and the impending disruption across the economy.

Very few, however, walk through the variables that determine the blast radius. Fewer still provide a playbook to integrate the new technology stack across white-collar work. This article does both: what determines the level of disruption AI causes in a given domain, and the critical moves to incorporate it, whatever its degree within your space.

The Landscape

Is everything crashing down, or up, or sideways? Is your job or industry ripe for disruption?

Narrow professions, or segments of professions, have already been disrupted by AI. But software engineering is ground zero for the first major, widespread disruption across a large industry. It provides the lessons to apply everywhere else.

The immediate impact of AI is lowering the cost of doing. This means the competitive bottleneck shifts away from raw execution and toward the disciplines that determine whether execution becomes value:

a) choosing the right outcomes,

b) translating those outcomes into real constraints, and

c) learning faster than the competition through iteration and implementation.

This reframing matters because velocity is increasing, but not in a simple “everyone ships more” way. In a world with tight feedback loops, speed tends to win, especially when aimed at deeper user understanding.

Faster building also changes the economics of experimentation. The median ROI across technology products is negative, while the average ROI is shockingly high because a small number of outliers create enormous value. As AI makes building easier, we expect that pattern to intensify. More teams will run more experiments, more will fail, and the median will fall. But the winners will iterate faster, compound learning sooner, and capture more value, pulling the average higher. Velocity becomes a “fail fast” superpower when paired with disciplined feedback and a clear definition of what good looks like.

The New Roles in an AI-First World

As roles evolve, traditional boundaries blur. The classic separation between functions begins to break down. In software engineering, small teams are already outpacing larger ones and outperforming more credentialed ones.

A comparison that makes this easy to grasp is a restaurant where a robot does the cooking. Someone decides what’s on the menu and what a great meal feels like (Direction), someone designs the kitchen so it can actually produce that meal reliably at scale (Architecture), and someone tastes everything and figures out what needs fixing (Evaluation). While the robot does most of the cooking, the other three roles become more important.

These three roles have become the bottlenecks to creating value.

• Direction: the ability to precisely define a valuable end-to-end experience.

• Architecture: the ability to translate that experience into constraints, interfaces, and data relationships that create reliable systems.

• Evaluation: the ability to measure, debug, and learn through testing, verification, and user feedback.

Crucially, these are NOT necessarily three job titles or even departments. These are three categories of responsibility that will blend within individuals and teams in some companies, but also scale into specialized teams in larger organizations.

Direction

Direction is the discipline of deciding what is worth building and what “good” should feel like when it is built… in detail and within the realm of feasibility of our new, highly capable systems. This is not “an ideas person” waving their hands at science fiction. Understanding the capabilities and the “jagged edges” of AI is a prerequisite to doing Direction well.

The core artifact is a set of vivid user stories detailing walkthroughs of the user experience that are far more specific than most teams produce today. These stories capture the complete experience: how the user discovers the feature, how it behaves under normal use, what happens when something goes wrong, and, crucially, what the user trusts the system to do on their behalf.

Metrics still matter, but they rarely encode the full complexity (or welcome simplicity) of a user experience. Direction includes look and feel, tone and clarity of system responses, workflow structure, and the evidence behind the stories: user research, support tickets, win-loss notes, market signals, and qualitative insight.

A Direction function should be able to answer, with precision: Who is this for? What are they trying to accomplish? What does success feel like? What should the user never experience? When Direction is done well, teams share a definition of a great experience and understand why it creates value.

Architecture

Architecture translates Direction into a reliable system. Once Direction defines the desired experience, Architecture defines the technical realities required to deliver it. This includes: performance requirements like latency, uptime, and graceful degradation; safety requirements around privacy, security, and compliance; and the constraints that keep teams from accidentally breaking things as they move fast.

Architecture consists of the technically demanding job of defining requirements and constraints. AI can propose designs and generate code, but Architecture must recognize where AI remains weak. In software engineering some of its weakest areas include data and infrastructure complexity. These areas still require deeper technical expertise and will continue to do so for at least a couple years.

Architecture should make it easy to build quickly, now and in the future, without causing severe breakages. It specifies what must be true for the system to behave the way Direction intends, even under load, when dependencies fail, or when new features are added. When Architecture executes well, builders can move fast because they are operating inside a set of constraints and interfaces that prevent accidental catastrophes. It becomes much easier to parallelize work across teams (human and AI) because the system contracts are explicit.

Evaluation

We will never “one-shot” perfection. Evaluation provides the feedback loop for continuous iteration. It is how the organization learns whether the system is actually delivering the experience defined in Direction, within the constraints defined by Architecture.

The speed at which our AI restaurant dishes can be expertly tasted is a primary determinant of success.

Besides speed, a mature Evaluation function distinguishes between:

• Bugs that violate expectations

• Quality issues that could be improved

• Feature requests that expand scope

Each requires a different response. That triage discipline is a core element of Evaluation. It becomes a primary source of competitive advantage. “Move fast, but don’t break anything essential” is now a genuine choice.

Variables of AI Disruption for White-Collar Work

This flow of Direction, Architecture, Evaluation will quickly begin to transform most white-collar work. Software engineering is a special case because it lines up almost perfectly with AI’s current capabilities. Other fields will require more adjustment, but the framework still applies.

What determines how fast AI comes for a given domain? We identify four spectrums, in order of importance.

Fidelity to Simulation

In 2020, Google DeepMind’s AlphaFold stunned the scientific world by solving protein structure prediction. In a matter of months, a single team with relatively limited experience in the field solved a critical area of basic research that had stumped biologists for fifty years. The team combined novel AI methods with the ability to run simulations at a scale no human researcher could match.

On the other hand, to date, no AI can predict with high accuracy whether a given protein will survive FDA trials for human use. One step further downstream, and the AI hits a wall because we still lack a sufficiently faithful model of how the human body actually responds to medicine.

The deep question at play is: Can the AI check its own work? Can it simulate reality with enough fidelity to know whether it solved the problem, or whether it should keep trying?

Where the answer is yes, the AI transformation will be swift. Software engineering sits at the far end of this spectrum because an AI can attempt thousands of solutions and simulate the final outcomes with nothing but processing cost. Physics, mathematics, and many engineering disciplines share this property.

Wherever there are high fidelity simulations, AI will dominate faster than most expect.

Reversibility + Feedback Velocity

In The Incredibles, the villain [redacted for spoilers] solves his superhero problem with ruthless… inefficiency. He lures heroes to a remote island, unleashes the latest robot, and lets the fight play out. Robot loses? Upgrade it. Repeat until the robot wins… lethally.

Super villains are often not efficient in their A/B testing

We don’t have the time, resources, or morals to run experiments that way. The relevant question for AI’s potential impact in a domain is: Is this a two-way door?

Jeff Bezos’s two-way door heuristic: if a decision is easily reversible, just try it. For our purposes in AI, we also need tight feedback loops. It needs to be both reversible and visible.

Can we quickly tell if the AI goes wrong, and reverse it with little lasting damage? If yes, AI will move quickly.

Consider an AI proposing changes to a website’s checkout flow: move the cart icon and reposition the purchase button. Rather than debating, run an A/B test. A thousand users get the old experience, a thousand get the new one. Add-to-cart rates tell you everything you need to know within a week.

Where high-fidelity simulation isn’t possible, AI will dominate where there is real behavior, low stakes, fast signals, and easy rollbacks.

Bill O’Reilly also recognizes the importance of AI’s capabilities to perform live experiments.

Verification Speed

If a task falls outside high-fidelity simulation and isn’t quickly reversible, that leaves us with human verification of AI outputs.

A skilled graphic designer can request an AI to swap out a portion of an image. Upon reviewing the output, the designer points out, “The hand has six fingers. Fix it.” Verification is fast because the work is easily observable.

Other tasks are inherently harder to verify. For cancer treatment plans, large-scale data engineering, and complex legal analysis, even experts struggle to evaluate quality at a glance. At a certain point, it takes nearly as long to verify and correct the AI’s work as it does to simply do the task yourself. Here, AI may be most useful as a collaborative suggestion engine (a built-in copilot) rather than a primary executor.

The faster the verification speed, the faster AI permeates that type of work.

Digital Centricity

In August 2015, my friend Rhett bet me that autonomous vehicles would be ubiquitous across the USA within two years. Having done a deep dive in the sector recently, I bet scaled rollout across most major US metropolitan areas would occur by the end of 2025. I still undershot. We both lost.

After 11 years since that bet, autonomous vehicles are… kind of here. Even though they operate in more controlled road environments, it’s a challenging problem to tackle. However, navigating a busy city street on foot is even more challenging, and is far more chaotic than anything in cyberspace. Real-world complexity, combined with safety concerns and regulation, makes physically manipulating objects a far harder challenge than most assume.

The more value a job generates from physical-world interactions, the less AI will affect it. The HVAC technician’s hands-on work is safe. Their marketing, scheduling, accounting, and customer feedback workflows are not.

Software Engineering Catapults into AI-First

Now that we understand how and why AI will take on white-collar work generally, you’ll notice that along every dimension, software engineering is ideal for AI disruption.

We are being catapulted to a new land of AI-first, but we do get to choose which way we face when launched.

The right questions are:

• How reliably can our team transform intent into working systems, receive feedback, and iterate?

• How can AI increase the velocity at which that occurs?

The first question is our performance North Star, while the second question is how we will navigate there.

Let’s begin with a wrong answer. Even in software engineering, there are no AI-shaped holes waiting to be filled. We cannot simply sprinkle “AI dust” over legacy workflows and codebases and expect a step-change in productivity.

The right answer is building an AI-first philosophy that treats AI as a primary user of your systems, and not just a tool someone occasionally consults. We call this the User Agnosticism Tenet: products, documentation, and processes should be designed for ease of use by humans, AIs, or a combination of both. When you embrace this, a codebase becomes legible and adaptable to both the agents doing the day-to-day and the people managing them.

This transformation requires shifts along three axes. The industry is swiftly converging on this framework independently. When many teams converge upon the same “new discovery” completely independently, we consider it additional evidence that the direction is sound.

The Infrastructure (Sandbox)

We must first recognize that code does not leap out of a vacuum and execute in isolation. For an AI to be an effective collaborator, it must understand the “room” it is operating in. This means providing the AI with a cloud infrastructure, security protocols, authorization systems, and testing environments available to it.

An AI-first workspace ensures that the agent knows exactly what resources are at its disposal and, crucially, how to verify its own work. If the AI cannot autonomously spin up a container or run a battery of integration tests to ensure its contribution is stable, it will forever remain a “suggestion engine” and second-class contributor. The infrastructure must be architected so that the AI can navigate the path from “code written” to “code running” with the same level of environmental awareness as a senior systems engineer.

The Codebase (Self-Awareness)

In an AI-first world, teams must invest in creating a navigable map of each codebase. The AI requires signposts and documentation to progressively disclose where it should be working and how it should be working. The goal here is legibility.

In practice, this means embedding lightweight guide files (called agents.md and similar files) throughout the codebase that points AI agents toward correct modules, pull request standards, and known gotchas. More importantly, the hidden assumptions and user experience intent that once lived only in developers’ heads must become explicit, and embedded directly in the code. (Note: At the moment, you can’t shortcut and just have AI spin them all up for you. It doesn’t work.)

The goal here is progressive disclosure: the system should provide the right amount of context exactly when it’s needed. When an AI modifies a feature, it should understand both the how and the why. The implicit becomes explicit, and the explicit is surfaced exactly when it’s needed.

The Issues (Communication)

Even the most sophisticated model will fail if it is fed “slop.” This is the ultimate GIGO (Garbage In, Garbage Out) problem. Telling an AI to “fix the bug where the user sometimes gets an error” and then blaming the model is a flagrant skill issue. If the models improve, but quality and velocity doesn’t, we are obviously the bottleneck.

The “Issue” or work ticket becomes the primary interface for execution. Writing an excellent one requires discipline. If our North Star is, “How reliably can our team transform intent into working systems?” then we need to clarify intent into precise guidance.

We use AI agents that combine codebase context with user intent to workshop issues collaboratively until the issue is refined and sufficiently precise for the AI to successfully tackle the issue independently.

We now have created a virtuous cycle. If an AI can’t solve a well-defined issue, that’s a diagnostic signal: either the codebase is too brittle, or the issue writer needs to sharpen their thinking. Treating issue writing as an AI-enhanced craft trains product managers to translate intent into execution and trains engineers to care about user understanding.

The Result and Next Steps

AI assists you to communicate precisely to the AI system. The system’s own documentation knows both how to push you for further details and execute once the instructions are finalized. The system then simulates and validates the results before implementation.

That’s the new world of software engineering we have been thrown into, but it is beginning to transform human resources, medicine, finance, law, marketing, accounting, etc. etc. Each field will follow a similar arc of Communication, Self-Awareness, and Sandbox. The pace of that change is determined by:

1. Fidelity to simulation

2. Reversibility + feedback velocity

3. Verification speed

4. Digital centricity

So, what do you do?

First: Recognize that the value of many types of expertise has gone up and not down. Depth of expertise rather than years of experience is what to look for and develop, both in yourself and across your teams. Someone with two years of experience, but both of those years are focused within one core dimension, is more impactful than the 10 year generalist. Why?

• Communication and Design: Experts know how to design the process and communicate the nuanced jargon that can command the AI in specific ways.

• Evaluation: Experts know what good looks like and can provide high value feedback.

Second: Know where AI disruption will strike first in your domain. In software engineering, AI lags in infrastructure and data engineering but excels in traditional front and back end tasks. Map your equivalent.

Third: Get comfortable operating inside the new flow personally. It’s uncomfortable. Do it anyway on routine tasks until it’s not. You cannot assess the future state without living in it.

Fourth: Begin adapting the organization. Jobs are bundles of tasks. Teams are bundles of jobs. Organizations are bundles of teams and leaders. When the fundamental tasks change, it ripples upward. Growth and structural change will be required, incorporating, among other items, the key points laid out here.

We’re catapulting forward, but you can choose your landing.

Looking for practical guidance and frameworks to transform organizations and technologies with AI? Architected Intelligence is now available for pre-order!

This was not wrong for its era. Unfortunately, every salesperson read this sentence and collectively decided it was a phenomenal tactic. So they used your name. And then they used it again. And then again. Hi Jacob. Absolutely, Jacob. I completely understand, Jacob. Let me tell you why this is important, Jacob.

If you grew up in the era of cold calls from the 80s through the early 2000s, you developed a very specific reflex. Someone says your name too many times and your shoulders tense up. Your guard goes up. Your brain quietly labels the interaction as manipulation and prepares an exit strategy. The beautiful sound of my name became a defensive trigger.

Because of this, over time, we stopped using people’s names in conversation. It had been corrupted by bad actors. “Social antibodies” formed against our own names. For the bold, feel free to test this by using someone’s name over and over to see what happens.

Gen Alpha, however, does not have these antibodies. They did not grow up fielding dinner-time sales calls from someone reading a script filled with [First Name] fill-in-the-blank. This has created a cultural asymmetry across generations. If you’d like to show concern for people, you can once again repeat their name! (But only for those under the age of 30.) If you want to really bend people’s minds, call someone by their name over and over, but with a mixed audience across age groups.

We are now watching the same phenomenon of social antibodies in writing. Let’s do a case study on one of the most popular AI tells: —

Em dashes are great—truly. They let you add nuance, create rhythm, inject voice, and signal a thoughtful aside. They are a wonderful tool when used… sparingly.

However, through RLHF (for the uninitiated: humans providing feedback to the AI models based on their output) the models realized that raters consistently upvoted responses that looked “smart”—and nothing looks smarter to a tired human rater evaluating their 379th piece of AI generated content in a day than a confident em dash! It’s grammatically correct and humans upvote it. So, AI decides to use them everywhere. “For example” gets replaced with an em dash… “in other words,” em dash. Parenthetical aside? Em dash. You all win an em dash! Em dashes became the punctuation equivalent of a salesperson repeating your name in 1995, except this time I am not wearing a sick Rasta-colored slap bracelet.

Of course, em dashes aren’t the only sign of AI generated writing. That there are no slow burn payoffs is the most striking difference. Other obvious signs are excessive groups of three, short contrastive statements, always moving “from” something and “to” something, hyperbolic generalizations, and excessive colons. These tics and styles are the hallmark of AI writing because it looks good and feels good in small doses, but the dose makes the poison, and we’re all feeling a little sickly. Strike out all those contrastive statements in the AI-generated text and you realize that nothing of value was lost. We need meatier narrative arcs, long payoffs, and rule-breaking.

The writing ends up with the veneer of intelligence without the substance. It sounds thoughtful. It sounds measured. It sounds clear. And it is increasingly difficult to tell the difference between genuine insight and well-formatted slop. So, people adapt. Some writers are now adapting by writing a little poorly on purpose so you know a human was involved. We are the equivalent of hipsters dressing badly to signal irony, except now it is to signal our humanity.

Below I propose a different approach. This is what’s working for me in collaborating with AI, at least at the moment. I believe it also can apply to most forms of writing today.

• Sparring Partner – Fight with AI, ideally on voice mode, but you have to prompt it to be combative. Do I have anything useful to say? The AI pushes back and exposes weak reasoning. It makes for a good rubber duck armed with boxing gloves.

• Sans-AI Building Blocks – Talk or type out your key points. Record a discussion with a friend and generate a transcript or ramble into a microphone. Create as many building blocks as possible to work with: jokes, half-formed ideas, key takeaways, and insights.

• Outliner – Fight with AI on the post’s outline. Instead of “rough points,” get down into the nitty gritty and nail down a detailed play by play.

• Constructor – Take all of your original content and your outline, then tell the AI to take the content to fill your outline and to do so verbatim whenever possible. Tell it explicitly to NOT smooth out the rough edges… yet.

• Finisher – Pass it through one more time to make sure there aren’t obvious factual or grammatical errors. Gemini 3 is especially good at bolding every change it makes without resorting to a different UI. But you’ll have to pry the Millennial ellipses out of my cold dead fingers…

  • From Gemini on the final pass:
    You mention: “Gemini is especially good at bolding every change...”
    Note: Since I am Gemini, and I actually did bold the changes in this conversation, this is a funny meta-layer.

I love writing. I highly recommend it. In my eyes, I gain three benefits from writing. First, it forces clarity of thought. Paul Graham has a phenomenal short essay (go read it!) where he wrote, “writing is thinking.” Second, posting publicly acts as a forcing function to create “stage fright” and push me to think harder. Third, my thoughts might be useful to you. Having grown up in another era, I don’t feel either the possibility or allure of fame or influence, but when a person tells me what I wrote was genuinely helpful, it’s motivating. The older you get, the more gratifying it becomes to help others learn from your mistakes. If this is helpful, forward it to a friend or hit the subscribe button at the end.

And now, to conclude properly:

This is not about correctness—it is about authenticity.

This is not about AI—it is about humanity.

This is not about writing—it is about communication.

We delve deeper. We shift from noise to signal. We embrace nuance, clarity, and intentionality. 🤮🤖😂

Subscribe now

Subscribe now

Google’s last two weeks have been extraordinary. Google is crushing it. This is where we are in the two year cycle:

Nano Banana Pro

Google’s “Nano Banana” was already considered best-in-class at image generation, but the new model surpasses its predecessor. Where’s the unlock?

• Fidelity to Text: It captures text nearly perfectly in standard scenarios.

• Visual References: You can incorporate up to 14 visual references into the image with high fidelity.

• Image Quality: It generates professional-level 4k images.

• Logical Consistency: It has the ability to follow complex instructions; even complicated diagrams can be constructed accurately and consistently.

What are all of those use cases for AI generated images you’ve thought of?

• Custom flashy slides

• Detailed marketing posters

• Illustrations with consistently portrayed characters in a story or comic book

• Any job that usually takes 30 minutes or more in Adobe Photoshop

• Any touch up in Photoshop that takes less than 30 minutes, but doesn’t have to be exactly perfect.

It can do it all.

I turned most of my recent blog post for Pattern into a magazine shot, word-for-word:

Google Begins Swapping to AI Default for Search

Consider the following:

• Google has more users than any of the other large tech companies.

• Google makes insane amounts of money from search, basically acting as a monopoly on advertising search content.

• It is not obvious that 10 years from now, Google Search in its present form will be a viable business.

• There is no business model YET within AI that is as profitable as the current Google Search business model.

Google’s “self-disruption” decision to swap the default mode from Search to AI is one for the ages. They are potentially lighting hundreds of billions of dollars on fire to bet on this transition. Will they do it?

Google just started rolling out default AI mode to some users.

Old default view for Google.com:

New default view for some desktop users on Google.com:

Gemini 3 – Use It Now

Chat GPT 5.1 was released to some fanfare. It looked great in benchmarks, but the quality immediately dropped for all of my [non-coding] use cases. The reasoning would go haywire on me on a relatively frequent basis, including this fun instance where it tried to correct the spelling of “shoos.”

Driven by my frustration with 5.1 and the buzz surrounding Gemini, I came back to Google’s model, and it quickly became my go-to for most use cases. Yes, in a month or two someone else will come out with a breakthrough model and we will all switch again. But if you have access to Gemini Pro, you should start leveraging it today.

Shareable Gems Going Mainstream

Now that Gemini Gems are shareable, one of the easiest corporate AI enablement hacks is to quickly stand up a knowledge management system for a team, department, or company.

Here are the steps:

• Go to Gemini

• Create a Gemini Gem

• Drop in the company documents, guidelines, policies, and best practices you want accessible to the team.

• Share the gem with the team, just as you would a Google Drive folder

For a group of 5 to 150 people, this is the easiest way to stand up a knowledge management system. Larger groups will require more controls and scaling methods to build high-quality infrastructure, but this is the best solution for any small group right now.

Our Parting Thought

Find what’s useful. Discover lasting principles. Be wrong. Adapt.

• Technology adoption happens at viral speeds

• Early adopters thrive within the constraints of nascent technology

• Late adopters lose the talent war

• Late adopters can still win (if they’re willing to pay the price)

A Tale of Two Movies

The first movie is one you may not have heard of: Home on the Range, released in 2004. Home on the Range was made by the venerated Disney Animation Studios, who had made dozens of highly successful hand animated films over several decades. As recently as the early 90s, they had created classics such as Beauty and the Beast and The Lion King, but something started going wrong as they entered the 2000s. By the time Home on the Range released, things had come to a head. A critical and commercial failure, Home on the Range was a sign of the times, a signal that a pivot had happened somewhere in the animation industry and Disney Animation had missed it. That pivot began a mere nine years earlier.

That brings us to our second movie, one I can almost guarantee you’ve watched: Toy Story. Considered one of the greatest films of all time, its a classic that spawned a massive franchise. While the Toy Story IP is considered a bedrock of the Disney media world today, it didn’t start that way. Toy Story wasn’t even made by Disney. It was made by a little studio known as Pixar, a spinoff of Lucasfilm purchased by Steve Jobs in 1986. Pixar wanted to be in the business of creating animated films. There were a few problems though. First, Pixar had never released a full length animated movie before Toy Story. Second, they were going up against Disney, who had built an empire on decades of highly popular and successful animation films. But the third and biggest difference is that Pixar wanted to make a film using only computer generated imagery, or CGI.

No one had ever released a film that was 100% CGI. It was unheard of. Up to this point, the primary method of animating a film was through hand-animation, a laborious process that required hundreds of skilled animators and artists. Pixar’s tiny team was proposing that they could create a compelling, emotional, artistic film using CG, which at the time quite frankly looked awful.

Yet despite the odds and the limitations of the technology, Toy Story was a smash hit. So was A Bug’s Life. And Toy Story 2, and Monster’s Inc., and Finding Nemo, and The Incredibles. Between 1995 and 2004, Pixar released hit after hit, all made 100% with CGI.

Meanwhile, Disney Animation films, still done by hand, began to inexplicably see lower and lower numbers at the box office. Movies like Dinosaur and Treasure Planet received mixed reviews and struggled to reach the high box office numbers of previous films.

Home on the Range was the final nail in the coffin. Disney Animation once again released a hand animated movie, despite the writing on the wall that things were going south. Upon release, critics and audiences reacted negatively. Disney Animation declared that Home on the Range would be their final hand animated film, and that moving forward they would plan on all films being made with CGI. While there have been some exceptions to this, overall CGI has become the near-universal way that animated films are created over the last 20 years. Not only that, CGI is used in nearly every live action film as well, from The Avengers to Avatar to The Lord of the Rings.

Lesson 1: Technology Moves Fast

Nine years. That’s all it took for an industry to go from hand animation to CGI as the default choice. When the ChatGPT moment happened, many individuals made bold predictions that within a decade, practically everyone would become reliant on AI. Skeptics shook their head, believing such a seismic shift in a short period of time was impossible. But history has demonstrated that impactful technologies can rapidly proliferate in an almost pandemic-like manner. Today we’ve discussing animation, but similar tales can be found in the history of cars, airplanes, electricity, railroads, computers, the internet, and more. AI appears to be on that same track.

However, the story of Disney and Pixar has unique relevance for business leaders looking to evaluate the risks of losing the AI race. There are three questions from the Pixar story that have massive implications for AI adoption.

How did Pixar make CGI work despite its limitations?

Why did Disney Animation movies stop working?

How did Disney Animation end up winning by the end?

Let’s answer those three questions with three lessons for AI adopters.

Lesson 2: Early Adopters Work Within the Constraints of Technology

How did Pixar sell CGI movies despite the shortcomings of the technology?

Early CGI was terrible, but Pixar was able to successfully work with those limitations. If CGI looked like plastic on screen, then the solution was to make a movie about plastic toys. Thus, Toy Story was their first movie by necessity. By working within the constraints of technology, Pixar was still able to express artistic creativity and build an emotionally impactful film. The success of the first film laid the foundation for them to slowly expand the ability of CGI. Monster’s Inc. added fur simulations, Finding Nemo added water simulations, and so on. By iteratively building upon the technology and slowly expanding its capabilities, Pixar was able to overcome those early limitations.

Lesson 3: Late Adopters Lose the Talent War

The success of Pixar is only half of the equation. Why did Disney Animation films stop winning?

There is nothing inherently wrong with hand animation. Clearly, decades of success proved that hand animation worked. Movies like The Lion King are still an enjoyable watch to this day. However, Home on the Range is not. What went wrong?

The first and obvious argument is that the efficiency gains from switching to CGI allowed Pixar to serve animated films at a lower cost, at a faster rate, with a greater ability to rapidly iterate and improve scenes through their technological advantage.

Such a sterile view of art through the lens of raw efficiency gains falls short of answering our question though.

I propose a second, more important reason Disney Animation films stopped working.

Disney Animation lost their talent edge due to falling behind the technology curve.

Imagine this. You’re a recent college graduate from the top art program in the nation. Your lifelong ambition is to make animation movies. Growing up, you had watched Disney and assumed a job at Disney Animation Studios was the logical endpoint for the journey you began by studying art and animation.

But as you drew close to graduation, a new kid on the block appears: Pixar. Their movies are a breath of fresh air. Modern humor. Heartwarming stories. But most interestingly, they seem to be forging the path of where animation is going. You have an opportunity to get in on the ground floor and create art with cutting edge technology. Or, you can go work at Disney Animation and painstakingly hand draw background characters twelve times per second of film.

The choice is obvious to us, and it was obvious to talent at the time. The best artists, screenwriters, and actors were all drawn to CGI as the new medium. They saw the writing on the wall. CGI was the future of filmmaking. Not only are modern animated films made with CGI, but so is practically every other movie as well. Whether you are watching horror, comedy, drama, or action, CGI is used to enhance hundreds of scenes across cinema.

By losing the talent war, Disney was left with second tier talent and bureaucracy that refused to take risks and thought making an animated film in the style of a Western was a safe bet.

The same applies to companies today. Employees are not stupid. They see the writing on the wall as well. They want to work at companies that are quick to adopt AI and position themselves as leaders of the new technology wave.

How can large enterprises ensure their employees are positioned to succeed with AI?

The same way Disney did. After announcing they were ditching hand animation, they gave free training courses to all of their hand animators so they could learn CGI if they wanted. Many employees took the courses, re-skilled, and went on to have successful careers. Some employees chose not to, and those animators had to fight over the few remaining opportunities for hand animation.

Lesson 4: Money Can Save Late Adopters

The story doesn’t end in 2004 though. The Mouse always wins.

The truth is, I’ve left out some key details in my storytelling. During this nine year period from 1995 to 2004, Disney funded all of Pixar’s movies, and handled distribution for them as well. A complicated legal relationship between the two meant that Pixar was financially bound to Disney.

And in 2006, Disney bought Pixar outright, finally bringing the circle to a close.

Disney immediately moved the best talent into Disney Animation, where they began working on new animated films built with CGI. Within a few years, Disney Animation was back on top, with films like Tangled and Frozen dominating the box office. Meanwhile, Pixar powered through with a few more hits before slowly experiencing the same talent drain problem Disney had faced earlier. Today, Pixar struggles to make successful movies that are not sequels, with several notable box office duds in recent years. Sony Picture Animations is now the ‘new’ kid on the block, taking risks with new technologies and art styles in movies like Into the Spiderverse and KPop Demon Hunters.

Companies are discovering the same lesson today with AI. Money can atone for the sins of late adoption. Mark Zuckerberg found this out the hard way when Meta began falling behind (see our story on their trillion dollar mistake). Acqui-hires are happening left and right, with big companies snapping up smaller startups to stay competitive. If you’re a big company falling behind, consider injecting your company with new life via an acquisition. If you’re a small startup jostling for attention, consider carefully if you want to go to war with the big players or aim for acquisition.

The bottom line: The race to adopt AI is more intense than ever. Companies recognize the risks of falling behind include not just losing the technology edge, but also the people edge. As organizations evaluate how to move forward, acquisition and talent attraction remain key cornerstones for staying ahead of the competition.

If we continually A/B test food, we eventually end with chicken nuggets.
If we continually A/B test our movies, we eventually end up with p0rngraphy.
If we continually A/B test our music, we eventually end up with a four-second TikTok sound and a dance challenge.

But if we continually A/B test our sense of humor, we eventually end up with a p0rnographic chicken nugget, desperate to go viral, dancing in a TikTok challenge.

A/B testing typically evaluates small and short-term changes. In scenarios where you can steadily improve in one direction with no bumps or disjointed gaps along the way, this works splendidly.

Unfortunately, your life doesn’t work this way. Because your life doesn’t work this way, your choices shouldn’t operate this way. Your AI shouldn’t operate this way either.

Machine learning is not like Donkey Kong where the highest score wins. It’s more like golf, but the ideal score is a zero. However, it’s very easy to get stuck in a “local minimum”.

My beautiful drawing above illustrates that each little stick guy is trying to win with the lowest score possible. It achieves this by walking downhill wherever it finds itself. That’s great if you happen to start in the blue zone. You’ll walk down and achieve the lowest score.

If you’re in the red zone though? You’re toast. No matter how hard you try to walk further downhill you can’t win.

For example, you can interview for a better job in your city. You might find something better than what you have. Or… you can move cities, or states, or countries. A drastic change will land you somewhere else entirely on the curve. These larger changes are not easy to A/B test, making them risky, or at the very least they feel risky.

A/B testing is probing the ground to see which way is downhill. That’s great, but it can only get you so far. If you don’t start in the ideal place though, you’ll need to pair it with a different strategy.

Escaping Your Local Minimum, and Helping Your AI Escape as Well

In machine learning, we fight against local minimum with two standard playbooks:

• Explore wildly different approaches simultaneously

• Continue experimenting, but “take larger steps” that are more drastic

This is great advice for machine learning, but time still moves forward linearly and you can’t clone yourself in simulations (yet), so these approaches can only help so much.

For AI, and I believe this is true for ourselves as well, the key is the combination of knowledge and taste. Phenomenal AI today is influenced by data (knowledge and experience), but directed by taste.

Experts and creators act as curators of the AI system’s “taste” and “feel.” AI system designers shape experience through response quality, interaction patterns, and the intangible sense of fit as humans adapt to it over the long term. This curatorial vision requires both technical fluency and user empathy.

Essentially, expert / creator “vibe checks” capture qualities of tone, fit, and taste that benchmarks alone cannot measure. Taste is an underappreciated talent in the world of subjective AI experiences. Those with vision and taste can create unseen productivity enhancements or beautiful artistic experiences that users didn’t even know they wanted.

The nature of AI is that it magnifies the decisions of the people who wield it. Without taste and expertise you end up with AI slop. With them, the people + AI combination is unstoppable.

I believe this same principle holds true for our lives as well. Have you ever seen someone stuck in their life? Have you ever felt stuck? Taste and expertise pull us out to try something new. Because we’re restricted by our own experiences, the easiest way to drop ourselves into a new zone is either the bold agency to try something new (most common in the young), or tapping into others with taste and expertise to provide insights to push us away from A/B testing and make the jump.

Also, after playing with GPT-5 for two days, I canceled my OpenAI Pro subscription. The only substantive difference between the $20 Plus and the $200 Pro level is the advanced research mode. After running experiments, xAI’s deep research seems to be on par or better than OpenAI’s.

OpenAI now hosts a variety of models, splitting up their infrastructure and increasing both costs and brittleness. With the GPT-5 release, they removed all other models from the app except for the three GPT-5 variants. Plus users are left with smaller context windows and more restrictive thinking models caps than they had in the GPT-4 era, which I’m sure will cause some frustration. From the API side, I expect a wave of model deprecations over the next six months to consolidate their hosting into a more manageable set, keeping cost growth more in check.

Brief GPT-5 Recap:

• As expected, the all-in-one approach delivers an upgrade across most dimensions.

• Pro ($200/month) isn’t really worth it anymore unless you want access to the old context window lengths that were removed, or you prefer its deep research over xAI, Gemini, or Claude.

• In early testing, GPT-5’s coding capabilities are finally competitive with Claude’s coding models. Friends who have used it more extensively say that if you structure your codebase in AI-friendly ways, GPT-5 really shines.

• Rationalizing the model lineup and hiding them under a single “GPT-5” connection should help OpenAI keep operating costs more reasonable.

Overall, it’s a bit of a let down versus the hype, but still a step forward.

What About the Accelerationist Curve?

Global economic growth is precariously perched and praying that AI delivers. Is the accelerationist curve real?

(Derek Thompson post)

A friend called yesterday to discuss the release. He reminded me of a prediction I made a little over 18 months ago: the industry likely had two major step-changes left before plateauing until the next categorical breakthrough. After the multimodal, deep-reasoning, and efficiency gains, we may have reached that point.

(See The Information article here.)

On the other end of the spectrum, while OpenAI is struggling with marginal improvements, Meta drops a blog post out of the blue claiming that AI is now improving AI:

(Article. Excellent basic html choice.)

Where that Leaves Us

Despite the signs of incrementalism, I’m still insanely bullish.

I continue to believe that AGI will be achieved through a more complex architecture combining both LLMs and other learning models. As I’ve mentioned before, I never subscribed to the idea “just train an LLM enough and AGI will arrive.” This type of architectural breakthrough may occur soon or years from now. If soon, it will feel like it was inevitable to many. If years from now, the era of incrementalism will feel inevitable. Both sides appear loud and overconfident.

Regardless of the path of AGI advancement, after the GPT-4 series and the general multi-modality (audio, image, and voice) quality increased significantly, I began repeating a line I still believe:

“Even if AI advancements froze today, we’d have at least 10 years of building to capture the benefit of this technology.”

Like the internet or smart phones, it will take time to discover, build, and iterate the best use cases. Opportunities are everywhere to adapt the current generation of models where they consistently outperform humans, even experts.

Even if the era of AI researchers begins to fade, the era of AI builders has only begun.

First, let’s take a trip down memory lane to understand how single individuals can create or destroy economic value. Steve Ballmer, as CEO of Microsoft, led the company down unproductive paths. The Zune mp3 player was the central symbol of Microsoft’s lethargy. “Like Apple, but worse” was the general sentiment.

In August 2013 Steve Ballmer unexpectedly announced his retirement from Microsoft. How did markets react? Microsoft’s value increased by $18 BILLION dollars. Microsoft hadn’t even announced who would replace him. The market consensus on the difference between Steve Ballmer and an unknown CEO was $18 billion. Satya Nadella eventually stepped in, transformed Microsoft’s strategy, and now the company is worth many times its 2013 valuation.

(Image from Marginal Revolution)

Take any moment between five years ago and two months ago. Imagine you’re a cutting-edge LLM researcher looking to make a lasting impact in AI. Meta reaches out to you to join their AI team. There’s Meta, but what are your other options?

1. OpenAI – Post-November 2022, it’s an obviously attractive choice, but even before then it was a cutting-edge AI startup.

2. Anthropic – Solid, ambitious, and intelligent teams focused on the cutting edge.

3. Google – Slow, bureaucratic giant, but with unreal amounts of talent and resources.

4. xAI – Really smart people attacking problems at an almost maniacal pace.

5. Niche players (e.g., Midjourney, Mistral) – Cool if you’re into their niche and culture.

6. Apple – Jk. Apple doesn’t do AI. They’re praying on the sidelines that AI becomes commoditized.

Besides Apple, you have some interesting options.

How does Meta stack up? You use AI (not Google, you’re not a boomer) to review Meta’s public statements about AI and discover their Chief AI Scientist has some interesting opinions:

• “This AI Pioneer Thinks AI Is Dumber Than a Cat”

• “LLMs can do none of those [reason, plan, etc.] or they can only do them in a very primitive way and they don’t really understand the physical world. They don’t really have persistent memory. They can’t really reason and they certainly can’t plan.”

Meta’s Chief AI Scientist calls LLMs dumb parrots repeatedly and insists they are not the future. As a cutting-edge researcher… you don’t agree. LLMs are an incredible technology with limitations, but still amazing potential. Your own research has shown this.

You, along with most other teams, picked a different horse to bet on.

Meta’s Reaction

Meta eventually found themselves in last place (Apple has been DQ’d). They poured in money and turned to open source (not a bad strategy given their ecosystem alignment), but Meta still lacks a cutting-edge research team.

Eventually, Meta (likely Zuckerberg?) needed to stop losing or give up. They had not a single major model in the top 30 on arena leaderboards.

Meta began throwing amounts of money, once reserved for sports superstars, at everyone else’s top researchers. How did OpenAI feel about that?

(I started with TBPN’s post and had some fun with AI and photoshop. 😆)

For the detailed breakdown on the Meta Superintelligence team, click the image below:

My Takeaways

I am impressed that Meta doubled down instead of giving up. It’s a gutsy move. Also, I don’t completely disagree with Yann LeCun! Ultimate superintelligence likely requires a larger reasoning and reinforcement framework beyond just an LLM. In my opinion, there are too many theoretical weaknesses to reach that point with only language token reasoning. However, I still believe LLMs will be a significant or even majority component of that larger intelligence framework. I believe LeCun got lost in ideal future purity over today’s practicality.

Imagine a world where the Chief AI Scientist is an LLM bull, and Meta attracts top-tier researchers early. Without exaggeration, this could have been worth a trillion dollars to Meta.

Who are the candidates for the greatest economic value destruction by an employee of a company?

Each step is a 1-2 orders of magnitude change:

Yann LeCun >> Steve Ballmer >> Stephen Colbert

Then again, they’re in the arena. Or at least, they’re in much bigger arenas than I am.

There is a lot of context around this viral moment, so please bear with me as I share the story leading up to the event. I think it will clarify a lot of questions around how my team interviews candidates, what we had already done to modify interviews in an AI-saturated world, and my thoughts in the aftermath of the event.

Note: As always, this is 100% handwritten by me, with no AI.

Subscribe now

SOME BACKGROUND

I was fortunate enough to have a new position opened on my team. Seeing the crushing workload of other engineers at my company, I knew I was lucky to be given an opportunity to bring on an additional pair of hands. I also felt proud that I was doing a small part to combat the challenging job environment developers face. I looked forward to helping one individual escape the unpleasant anxiety and stress that unemployment brings.

My excitement was tempered by my knowledge that hiring is a challenging process in 2025. Outdated practices, awkward interactions, and of course AI cheating were all present in my mind as things to be cognizant of as I designed the interview process. I biased towards interviewing locally first, knowing that for most people, the interviewing experience is more pleasant in person. I also knew you couldn’t cheat with AI in person.

I interviewed some amazing people here in Utah. Some of them may be reading this! Unfortunately, none had quite the blend of skills and experience we were looking for. I decided to cast our net wider and started interviewing remote candidates while setting a clear expectation that this was an onsite role and would require relocation to Utah. Many of those remote candidates were also wonderful folk.

RESUME GRINDING

I read through at least 100 resumes during our search. The resumes were a foreshadowing of what was to come. At first they all looked fine. Some good, some weak. But on resume number 60, I noticed something odd. My eyes, now seasoned, noted that this resume felt… off. This guy worked at Nvidia and Uber, and now wanted what was basically a Senior engineer role? This would be a serious step down. That made no sense. Searching for his LinkedIn and Github confirmed my suspicion, as both had no history on them. “This is a fake”, I told my recruiter. He was surprised but not shocked. “We’ve seen this a few times,” he nodded knowingly. A few times!? How many of the resumes in front of me had been fabricated by AI? I ended up spotting one or two more deceptive resumes by the end, which makes me think I definitely missed some fakes.

INTERVIEW STRUCTURE

When tasked with hiring someone, I knew there was no textbook on what our interview structure should look like for an AI team. To what degree should we allow the use of AI in our coding challenge? What kind of experience were we looking for? How could we find out in under an hour what a candidate actually knew? I’ll dive into the nitty gritty of our technical interview soon, but first let’s look at the bigger picture of the entire interview process.

I had originally structured our first two rounds of interviews to focus on experience in the first round, then do a technical interview as a follow up. I thought this would be the better approach, given that experience is harder to fake. However, we found that a lot of candidates were good at sharing their experience, but ended up struggling with the technical interview. I’m proud of those candidates for their honest attempts without cheating their way through it. While they may not have gotten the job, they should sleep well at night knowing they are honest people.
We then decided to switch it up and do the technical interview before the experience interview. This format would have sped things up a lot with previous candidates who got far only to hit a wall with the technical interview, so I was optimistic on this approach. I should also note that after these first two interviews, regardless of order, there is a 3rd round interview that does a DEEP dive into experience and culture fit, along with a lot of reference calls. After that is an intense process of blue teams and red teams that compete to persuade executives to approve or not approve hiring a candidate. There are also screening rounds by our recruiters before I even look at candidates and get them started on the interviews. It’s very hard for anyone to completely fake their way into working at our company.

Ironically, our cheater happened on the first instance of switching to doing the technical interview first. It makes me wonder how it would have gone if we had done the reverse order. Would I have rejected him during that initial experience interview? Let’s now walk through the actual technical interview, and how the cheater performed throughout the process.

PART ONE: TECHNICAL DISCUSSION

The first half of my technical interview is what I like to call technical discussion. I ask a variety of open-ended questions, many of which ask the candidate to describe the pros and cons of two competing or complimentary technical concepts or problems in the world of AI engineering. I’ve found that candidates with experience do well on these questions. They can describe the problems of each approach, yet also the benefits they present. The tradeoffs are nuanced. Good answers come from people who have gotten their hands dirty with both.

Early on, I noted that the candidate seemed to have serious communication skill challenges. His answers were mumbled and rambling, with little to no depth. There was no passion.

“Poor communication skills”, my notes say, “not excited about him.”

It got weirder. For example, I asked him to explain MCP (a communication protocol used to standardize AI tool use), and he described it as a ‘father/child’ relationship. An unusual analogy, to say the least. He also claimed it facilitated multi-agent collaboration. That’s a stretch. While MCP can be used for agent to agent communication, that is not it’s designed purpose.

PART TWO: CODING CHALLENGE

After how poorly part one had gone, I had already mentally marked him as no. However, in good faith I went forward with the 2nd half of the interview. (I’ve always found the concept of cutting someone off in the middle of an interview distasteful. It’s terribly disheartening for someone who puts in a lot of work. Being rejected a day later lets you assume the interviewer found a better candidate rather than just immediately wrote you off.)

The 2nd half of the coding interview is a simple coding challenge involving data manipulation of a JSON object. It does not require any sort of dynamic programming skills or re-ordering tree structures. It reflects one of the most common scenarios we face on a daily basis, which is handling JSON data with Python.

As the role is on our AI team, my AI coding rules are somewhat progressive IMO. Here is the exact text we share with candidates:

You can use tab-complete. You can ask AI for help with Python syntax or specific problems. Do not one-shot the problem with AI. Do not paste the problem into your IDE, or else tab-complete will one shot it. The goal is to see your thinking process.

Over the dozens of interviews I’ve given, I’ve seen a huge range of behavior here. Some candidates forgo using AI entirely. Others heavily rely on it during the interview. I’ve seen some candidates get completely lost following the AI, while others have tab-completed their way to victory. At my current company, engineers are expected to use AI coding on a daily basis, so I feel this challenge reflects our day to day reality. And while many on the internet may opine that restricting wholesale use of AI in this interview is hypocritical, I believe this is a good approximation of the reality of working in large, mature projects, given the 30 minute constraint of the coding challenge. It lets us see both their ability to use AI, and also their ability to understand code. We are not in the business of hiring people who blindly yeet AI generated code into the repo with no understanding of what it means.

THINGS GO SOUTH

This particular candidate read through the rules and problem description, then began sharing his screen. I immediately noticed an odd overlap on top of his screen. The candidate would drag it in and out of view to check what it said. After looking at it a few times, my suspicions were confirmed.

This candidate was using Cluely.

Cluely is a product built by a Columbia student who was kicked out of college after he built an AI interview cheating software that evolved into what is now called Cluely. It openly and brazenly proclaims itself as a ‘cheating’ tool for interviews, tests, and more. Their ads are extremely cringy, and both the VC backed company and founder have attracted significant controversy.

Cluely is supposed to be hidden during zoom calls. It’s supposed to hide itself from the view of the screenshare. Only for this candidate, he was apparently too dumb or too lazy to figure out how to do that.

“Jeremy, that’s some harsh language. Shouldn’t you support people using AI?” Normally, yes, I’m all for AI. Even as my blood pressure raised, I mentally gave him the benefit of the doubt. “This is exactly the kind of people we are looking for,” I told myself, “People who pro-actively use AI.” I assumed (and I hoped!) that he was using it as an assistant, not as a cheating tool. Yet the app was clearly showing a complete solution to my coding challenge, in direct contradiction of the rules he had just read.

So I confronted him. “What’s with the AI app?” I asked. He immediately froze. “Huh?” He feigned confusion. “The app on your screen with AI,” I nodded towards my computer. He made a big show of rotating the view through his different virtual desktops, not realizing that his AI cheating tool was clearly visible. “I…. I’m not using any AI tools!” (I’m cleaning up his grammar, his embarrassed mumbling was not that clear.)

I became angry. Angrier than I’ve been in a long time. “The rules of this coding challenge were clear. You cannot dump the problem into AI. This interview is over. Have a nice day.” I immediately ended the call, fuming. My biggest pet peeve by far is lying. I find it absolutely despicable. Passing off the work of either other humans or AI as your own work is unacceptable in my playbook. If he had owned up to using the AI my reaction would have been very different. Instead, our call ended 25 minutes early and turned into a 9 page rant that you see in front of you.

THOUGHTS IN THE AFTERMATH

A few interesting thoughts came to my mind as I grumbled and stewed over this frustrating experience. I had found myself a victim of the exact thing I had published a paper on in college: the inability of humans to detect the deceptive use of AI. I started thinking on how I could connect my research discoveries with my current experience.

My research had found a few interesting things:

1. Education and being on the lookout improves the ability of younger individuals to detect AI deception.

2. However, older people had the opposite experience. Constantly looking out for AI deception made them worse at spotting AI generated content.

If I began approaching every interview highly skeptical of candidates, would I get better at spotting cheating? Or would I become some pre-occupied with catching cheaters that I’d miss genuinely good candidates? Or worse, falsely accuse them of using AI.

My college research was built on the assumption that AI deception is a game of cat and mouse. Every time AI detection software is updated, the cheating software evolves and outwits it. Undoubtedly, we should work with our recruiting team to update our video conferencing software to add cheating detections, along with AI reviews for incoming resumes. I’ve also been approached by headhunters and external recruiting agencies who offer to solve my problem (for a price). But no matter how many softwares or services I use, I fundamentally believe you cannot fully immunize yourself from cheaters when interviewing remotely.

That’s why part of me wants to just put the hammer down and ban remote interviews. It would simplify the interviewing process. Interviewing candidates in person for onsite roles is so valuable. Much is lost over a two dimensional, half second delayed, head and shoulders view of an individual. Biggest of all, you can’t cheat in person. (At least not as easily. The majority of in person cheating is someone taking lots of bathroom breaks to check their phone.) Luckily, I’m done hiring for the next little bit. I’m happy I won’t have to face this problem again for a while. Interviewing candidates in person comes with it’s own set of challenges. If you can’t find someone locally that fits the role, you have to fly people in. Hopefully my next company is like Nvidia and has cash to burn on plane tickets!

The other thought I had was that the candidate was still terrible. Even with the AI. Or maybe because of it? I’m not sure. But he was not good. The Cluely subreddit r/cluely is filled with questions from users who clearly have no idea what is going on. This subredditor posted this minutes after my call with the cheater ended (in case you are wondering if it’s the same person, I doubt it. We use zoom, not teams):

I was caught using cluely in interview, not sure I've used browser teams call but the interviewee caught me using cluely

Grammar? Out the window. Are they asking for help? Not sure. Are they complaining about the software or just confessing their sins? Your guess is as good as mine.

In my head, there are two outcomes with my cheater.

1. He reforms from his cheating ways.

2. He figures out how to turn on the basic settings that actually hides the dang app. Or at least keeps it on another screen.

While I would have rejected him even if I hadn’t seen him blatantly using AI to cheat, I’m disturbed by the fact that I likely would not have realized he was cheating had he used two of his brain cells to configure the cheating software properly. It’s completely altered my worldview. Not to be dramatic, but seriously, what is real nowadays? I suspect that the value of in person conferences, in person interviews, and in person interactions will skyrocket in value. The flood of AI slop drowning our inboxes, our social media feeds, and now our interviews threatens to destroy our trust in the digital world. Private community chatrooms and servers are seeing a huge resurgence that is reminiscent of the early days of the internet, except now they are powered by advanced software applications like Discord and Slack.

Bottom line: The unethical use of AI should never be tolerated. Misrepresenting the work of either other humans or AI as your own is unacceptable. Always be transparent in your actions and authentic in your personality. When hiring candidates, look for those that can both use AI and understand the code that is generated.

… ooph.

After playing around with it since its release last year, how is Amazon AI shopping performing generally?

The responses? OK, but not great.

User experience? Pretty clunky.

The responses are pretty mid and don’t infer and adapt to my background very well. The overall functionality is not intuitive. Amazon, being one of the largest companies on the planet, probably has eighteen layers of safeguards to avoid anything remotely “risky” or too creative slipping through. The result is a strictly vanilla experience with a janky UI.

To be fair, nobody has really nailed intuitive AI interfaces yet. Trillions in market cap, but still struggling with an AI shopping experience. With their resources and talent, I’m sure it will steadily improve. Still disappointing, but also, unsurprising coming from a very large player in a new and fast moving space.

This leaves a massive opportunity to get AI shopping right. Your response, along with many many hundreds of others, might be, “I’ll just build a new general AI shopping experience to compete with… Perplexity… ChatGPT… Amazon… Google… and many many others.”

If you solve this at the general shopping level, your only moat is sprinting faster than the competition (see my earlier thoughts on the Gingerbread Man moat). Velocity would be your primary competitive advantage.

What’s a smarter alternative?

Build a purposeful and trustworthy AI shopping experience with specialized integrations. Don’t just slap a “Shopping” tab onto something like Perplexity or Google have. Instead, create a dedicated and purposeful AI shopping experience with specialized data sources and genuinely collaborate with the user.

For example:

Of the many possible data sources, you could connect your Oura ring, Garmin, and dieting app to get tailored recommendations. For example,

• What supplements should a triathlete consider that I’m not already taking?

• What diet adjustments could optimize my performance?

In this example, you can create the bespoke AI experience for physically active users.

Is this possible? Yes, I do it right now. Current platforms can do this in a clunky, DIY way. I personally hack it together, but I’m weird and comfortable jumping through hoops based on my knowledge of AI. For the non-weirdos out there? They’d kill for a polished intuitive product. Honestly, I’d love that too.

Now imagine that implementation across every shopping niche. Can a one-size fits all platform dominate all that specialized data and nuanced experience and win the day? Maybe. I wouldn’t count it out, but I’d put the odds below 50% for a few reasons:

• Some of the “pretty large” companies maintain agility, but the mega corps are inevitably slooooow

• Specialized integrations create too large of a surface area for one company to dominate effectively

• Design opportunities for niches are too numerous for single companies to dominate

This isn’t the same story as Amazon dominating the original web shopping experience. Companies will be going to where the users are instead of driving the user to their website. Accordingly, smaller players will eat away at the winners and a winner-take-all market is unlikely.

As the saying goes, predictions are hard, especially about the future. However, I expect a few mega AI shopping platforms with a swarm of smaller specialized AI shopping experiences better customized and responsive to your unique tastes and journey.

One of the most straightforward and common initial approaches is to pursue a segment where others aren’t playing. Many start this way with a “new idea” and “new approach.” This is wonderful right up until the moment you succeed and others, whether startups or existing players, decide that they want a piece of that.

• Pebble launched a successful and viral smartwatch and app. This was great right up until every major technology hardware company decided to build their own and Pebble went bankrupt.

“Nobody else is doing this yet” is not a moat. So, what can be a moat? Until recently, I’ve liked simplifying it down into three D’s: Distribution, Design, and Data. Before we go over why AI upended our apple cart of competitive moats, let’s walk through what these moats traditionally looked like.

Distribution consists of two abilities: a) acquire customers, b) and retain them.

• Uber was legendary at simultaneously building “two-sided” markets of drivers and users in new cities. A virtuous cycle of network effects sustained that advantage, but city entrance skills were key.

• Atlassian (Jira, Confluence, Trello, etc.) has minimal network effects across companies, but it reached a $20 billion market cap essentially without a sales team. Instead they pursued a word-of-mouth, marketing, and minimal friction sign up.

Data is one of the most commonly recognized moats. In some categories, it’s difficult to acquire customers without data, and it’s difficult to acquire data without customers. The data could come directly from your users, or they could come from a parent company or other relationship. The key to a data moat is that whoever possesses the most data will have the best product.

• Bloomberg has gained a reputation as the “comprehensive repository of accurate, real-time data.” That’s a long hill to climb to provide parity with Bloomberg, and your best hope to compete is to carve small niches.

• Because Google captures how customers respond to search results, they can serve the most relevant search results. The design underlying technology is important, but Google’s granularity for human behavior on search engines was unparalleled for decades.

Design moats create a customer and product experience that creates brand loyalty and equity.

• Headspace is a mindfulness and meditation app that lives on a principle of Design.

  • A user’s 10‑minute session does not improve because millions of others subscribe.

  • The catalog is proprietary, but guided‑audio IP is easy for Calm, Insight Timer, or YouTubers to replicate

  • Headspace’s moat relies on the instantly recognizable design language and trusted tone

• Squarespace’s library of high‑gloss, editorial‑grade templates sets the aesthetic bar for DIY sites.

  • One store or portfolio does not accrue extra value from another customer’s site, so little network effect exists.

  • Competing builders (Wix, Webflow) can match features, and page‑analytics.

  • What keeps many creatives paying is the perception that Squarespace = tasteful design out‑of‑the‑box.

AI is coming for your moats.

Data moat erosion – Because AI has picked up information across the entire internet (stored in its “latent space”), you often only need to tap into that data by providing customized user information and scenarios into the AI. Once you do that, you can generate high quality analyses, classifications, and decisions in return.

New product surface area – Google previously knew “everything” about you, yet in just a few months of ChatGPT use, users willingly divulged far more detailed and valuable personal information.

Tech Speed Unlock – Both incumbents and startups can develop technology faster than ever with AI. Incumbents can rapidly build and deploy AI features to existing customers. Startups can quickly launch competitive AI features within just a couple of months.

These examples are only the beginning. The landscape has shifted.

Only the Gingerbread Man Moat Remains

What is the last moat standing? Velocity. Go faster than the competition. Many used to poke fun of “velocity” as a competitive edge. Erik Torenberg from a16z reminded me of the strategy Ben Thompson once dubbed as “The Gingerbread Man” moat.

Cursor is considered one of the most successful companies of the AI era due to the speed of its high-quality implementation. The design is sharp, the data is helpful, and the distribution strategy is solid. However, if Cursor paused development for six months, it probably wouldn’t survive.

Currently, it feels like velocity is the only moat that matters in AI. If you pause for six months, your company is effectively dead. That wouldn't have spelled doom for any of today's trillion-dollar market-cap companies. Speed has always been important, but Amazon, Facebook, and others likely would have endured a six-month pause. Once your company gains visibility, expect a pack of rabid wolves at your back for years. It’s exciting for bystanders… and exhilarating and terrifying for participants.

How extreme is the velocity shift? GitHub Copilot just flipped to open source. They open-sourced their product because they were falling behind Cursor. If GitHub alone can't outperform Cursor, perhaps the open-source community can at least put them on equal footing.

It seems like a crazy move, but GitHub can afford it because Copilot is a critical moat supporting GitHub’s core product. If GitHub Copilot were a standalone product, they would be fighting for their lives.

“Work smart not hard” only works against dumb people, and AI unleashed a lot of smart in the world. The competition is working both very smart and very hard.

Velocity is the moat that matters. AI’s pack of rabid wolves is nipping at our heels. Life would probably be a lot chiller if I ran an HVAC business right now.

You can feel the echoes from earlier tech eras facing similar skepticism. Personal computers, internet, smartphones, and cloud all had their skeptics, but the pace plodded and the ferocity seemed a bit more tame than today. Perhaps this higher intensity is simply a combination higher velocity of progress coupled with social media contagion.

(To be fair, I was a major skeptic of the iPad. To this day I don’t understand the idea of “a laptop, but worse” unless you’re looking for a portable television or a mediocre gaming device.)

In the mid-2010s, an acquaintance at a neighborhood barbecue asked me about my work. In simple terms, I mentioned building a machine learning recommender system. He literally LOL’d. Absolutely incredulous that anyone would seriously invest in something called "machine learning" that would recommend products or TV shows you might like. It’s all another tech hype cycle to hoodwink the investors and management.

That reaction stuck with me. “Highly confident and dismissive on a subject he knows nothing about.” Where are my similar blind spots? Where are yours?

Today, millions of people hold similar attitudes toward AI, claiming that it fundamentally "doesn't do anything" in their field. You can argue that AI is currently overhyped (fair). You can insist that it's not quite ready for certain complex or nuanced applications (very fair). You might even voice genuine concerns about potential risks or ethical issues (fair-ish depending). Those are valid conversations.

But to categorically claim AI doesn't work is to ignore reality. Millions of individuals and thousands of organizations are actively leveraging AI today, creating tangible value across companies and industries. They’re busy grinding out incremental improvements in healthcare, finance, logistics, retail, and manufacturing.

This stubborn skepticism feels like déjà vu to earlier times. I remember the late '90s when plenty of smart, capable individuals insisted the internet was nothing more than a passing fancy, a solution looking for a problem. But this current dismissiveness feels even more out of step. It’s less “late '90s skepticism,” and more “2006 stubbornly insisting that the internet would never significantly impact retail, even as traditional stores visibly struggle.”

At this stage, confidently proclaiming that AI "doesn't work in my field" is bragging about one's ignorance. It's publicly diagnosing oneself with a severe skill issue.

Perhaps it's time to level up.

Subscribe now

Lately I’ve been enjoying more episodes from the a16z podcast, the official podcast from the venture capital firm Andreessen Horowitz. In a recent episode they reviewed the oft-debated question, “What is an AI Agent?”

Part of me agrees with one of the participants, Matt Bornstein, pointing out that straining over the definition of a made-up concept is a bit of a waste of time. He said, “I think we’ve all just been nerd-sniped. I just think it’s, like, humanities people love classifying and they draw kind of like fine distinctions between different types of things, entities, whatever.”

However, in our day-to-day work we settled on a clear, functional definition that has been helpful for coordinating work among teams. Instead of a buzzword to toss around, “agentic” and “agent” have practical definitions that help us align planning and actions across teams.

We settled on this definition only once we settled on an overarching framework of the underlying properties for what is and is not an agent.

A Definition and a Framework

For clarity, we break it down into a simple 2x2 matrix made up of two parts, orchestration and action.

Orchestration – Does the AI orchestrate the solution?

Action – Can AI be used to take an action?

That’s it. Apple’s Siri was an AI Router, using AI (primarily through NLP) to decide what to do. An AI workflow would be an LLM evaluating a refund request and making a recommendation based on company policy. An Agent does both the orchestration and can choose to take AI actions.

Below is a more detailed explanation, and we’ll provide more in-depth examples in follow-up posts (e.g., how “agentic” is a system?). But for today, we’ve found this a clean place to start from and enhance communication across the team.

Skip to Next Part Unless You Want the Gory Details

Briefly walking through each quadrant:

1. AI Orchestration + AI Actions = Agents

Full AI agents. An AI directs the processes and tools, actively determining how tasks are executed to achieve its goals. The AI agent decides which AI workflows and tools, along with their corresponding actions, to trigger.

Agents offer the greatest flexibility and adaptability. However, complexity, uncertainty, and risk also peak here.

2. AI Orchestration + Non-AI Actions = AI Routers

These systems orchestrate decisions, but rely on non-AI actions or tools. These tools are often predefined system processes. Virtual assistants such as Siri or Alexa typify this quadrant. They decide how to respond to user queries, yet the actual actions, such as playing a song, setting an alarm, or sending a message, remain relatively straightforward and predetermined.

AI Routers simplify execution. They retain adaptive benefits of AI orchestration, but with lower complexity and fewer uncertain outcomes due to the simpler actions executed.

3. Non-AI Orchestration + AI Actions = AI Workflows

This quadrant represents the traditional AI workflows where non-AI orchestration triggers and manages predefined workflows, and these workflows perform AI-driven actions.

AI Workflows allow your teams to harness powerful AI capabilities while controlling orchestration through predictable, rules-based methods. This reduces complexity and simplifies management, at the cost of adaptability.

4. Non-AI Orchestration + Non-AI Actions = Traditional Software Engineering (SWE)

Here we find traditional, rules-based software systems with no significant AI integration. An example would be an automated email-reporting system triggered each morning by a scheduled job. Both orchestration (cron jobs, scheduled triggers) and actions (sending a formatted report) follow predefined rules without AI-driven decisions or adaptive capabilities.

Traditional SWE offers maximum predictability, reliability, and simplicity, but lacks adaptability, innovation, and the powerful analytical leverage provided by AI.

No quadrant is inherently superior. The optimal choice depends on the objectives, the complexity of tasks, and the level of adaptability required.

Contrasting Definitions

How does our definition match up with many definitions offered by organizations, companies, academics, and your neighbor’s dog? Of all of the definitions we’ve seen bandied about, Anthropic’s definition aligns closest to what we feel like is an effective practical definition:

This definition captures a concrete distinction. Workflows follow deterministic paths, which are a series of steps executed consistently each time. In contrast, AI agents select and orchestrate among available workflows and tools, continuously evaluating options and outcomes until they achieve their goal, determine it's not worthwhile to proceed, or are deactivated. However, we diverge from Anthropic's framing in one key aspect: AI orchestration does not necessarily require an LLM. Instead, orchestration might be executed through alternative forms of AI, potentially ones better optimized for rapid feedback loops and decision making.

In the end, don’t get too hung up on definitions. These are evolving terms. However, we’ve settled on a practical definition that lines up relatively closely with one of the major companies and enhances communication among team members. Among the technical members, this works. If you’re looking for how to craft marketing hype for AI agents… that’s for a different post.

You can catch a16z’s podcast episode on “What is an AI Agent?” here if you’re interested: https://a16z.com/podcast/what-is-an-ai-agent/

(I highly recommend the podcast, especially if you’re the type who knows which episodes early on you should skip.)

Every walk of life is a good description for what I saw in my visit to New York for this conference. You had hungry startups, proudly sharing the cutting edge discoveries and innovations they had built in the hopes of finding more users. The VC sharks were circling the waters, cornering you the moment you looked available and trying to extract as much information from you as possible about you and your company to see if you fit their profile. You had representatives from the big dogs: OpenAI, Anthropic, and Cohere, each trying to position themselves as the one true path to success with AI and agents. And you had a few brave souls such as myself from regular companies, with common reactions of shock when people realized I was neither a VC nor a startup. Alas, I simply represent a B2B company that has a shockingly high amount of annual revenue yet is practically unheard of outside of Utah.

Some things felt normal at this conference: the endless decks, the mingling around food, and cringy jokes from the hosts. Other things felt less normal. Someone kept bumping the projector screen button, causing it to constantly go up and down to the embarrassment of the hosts, not to mention various other technical difficulties with lights turning on and off, struggles to get slides to display, and more. Overall though, I walked away feeling that the conference experience had been positive and worth my time.

There are several themes I heard repeated throughout this ‘AI Agent’ conference. The most common themes included:

• Explaining the difference between an AI agent (which loops over tools) and an AI workflow (which is a one time call/response)

  • I was disappointed not to hear anyone try to bridge the difference between these two. I felt like Harrison Chase from LangChain has a great way of uniting these two ideas under the term ‘agentic systems’, but people seemed incredibly proud just to point out that there are, in fact, two ways of using LLMs. To me, this seemed obvious, but for others was perhaps a revelation.

  • One thing I was happy to hear was that people recognized you can mix and match agents and workflows. Agents can call workflows, and workflows can call agents.

• Pointing out the weaknesses of LLMs and then proceeding to explain how their platform fixed it

  • Generally there are four ways people ‘fixed’ LLMs:

  • 1. Improve the underlying model with finetuning, LORAs, or other methods

  • 2. Improved data and context

  • 3. Evals

  • 4. Frameworks that abstract how you interact with LLMs

• The commerce section showed an intense interest from the audience on the topic of GEO (Generative Engine Optimization) and how consumers are finding brands and products through ChatGPT

  • A Shopify exec in attendance who is responsible for many of their AI initiatives stated that the OpenAI + Shopify integration was rolled out ‘very fast’ and that they are now going to be playing catch up with advising brands on how to they can improve their ability to appear in ChatGPT results

  • The startup Profound is one of several that are leading the way in this new field, with some fascinating insights. Did you know ⅓ of all LLM citations are from comparative data? Think blog posts comparing the top 5 smartphones.

• Predicting the future

  • Lots of people had graphs showing where things are going. Most people are convinced that agents will be everywhere by next year. But once you got off stage and talked with people, whispered conversations discussed the reliability issues of agents, how none of us actually had any in production, and doubt as to whether the challenges of agents could ever truly be overcome

• AX (Agent Experience), AI-Native, and AI-First

  • A lot of buzzwords that boil down to the same thing: Are you making agents your TOP priority?

  • My favorite comparison was a comparison to Blockbuster calling themselves a ‘digital company’ by adding email and a website, compared to Netflix who went literally all in on digital

  • Likewise, companies that are truly ‘AI First’ need to make that sort of transformation

  • Unfortunately, this can have some public backlash if not done right (see Duolingo). Maybe don’t go bragging about it on social media

  • Maybe a better way of phrasing it would be “We are putting humans first by making sure they are 10x more effective via AI”. Doesn’t roll off the tongue though…

• Evals, evals, evals

  • Lots of emphasis on the importance of evals

  • Everybody liked to tote that evaluation was what set you apart. Unfortunately, evals are a difficult thing to get going, and I think many people exaggerated how much they are using evals

The bottom line:

I won’t bore you with repeating everything I heard, but suffice it to say that there is still much in flux in the world of AI. Time will tell which of these ideas and companies will stand the test of time. The best thing each of us can do is continue to be curious and constantly learning. As pieces of the puzzle solidify, the industry will coalesce around best practices and leaders, and new layers of complexity and opportunity will appear.

In 1999, a book titled The Pragmatic Programmer was published. I’ve never read it. But this book introduced and popularized many concepts which I later learned in school and at work. Some of these concepts programmers may recognize; DRY (don’t repeat yourself) is likely the biggest. But The Pragmatic Programmer introduced a second concept that gained widespread recognition, which was titled ‘rubber duck debugging’.

I was first introduced to rubber duck debugging late one night during school. It was dark outside, and I had been coding for hours. I couldn’t seem to solve a pernicious bug in the assignment our teacher had given us. At the end of my ropes, I called my friend Koby Lewis, who was and likely still is a far more talented programmer than I am. “Koby, I know it’s late, but I’d really like your help figuring out this problem”. Koby graciously agreed to help me out. In order to help me, Koby asked me to walk him through the problem. I began explaining, but Koby quickly pointed out that I was not providing enough detail for him to understand what the problem was. We dove deeper and deeper, until I ended up monologuing for about 5 minutes on exactly how the code worked. At the end of this five minute description, my detailed explanation to Koby revealed a logical mistake I had made in my program. “Aha!” I said. “I figured it out! Thanks Koby!” Koby laughed and responded, “Happy to be a rubber duck for you.” Huh? What did he mean?
Koby explained to me that rubber duck debugging was a term used to explain the concept of debugging a problem by literally explaining it out loud. In The Pragmatic Programmer, the author describes a software engineer literally carrying around a rubber duck, constantly explaining to it what he was stuck on. I realized, as I progressed through school, that my programming classes were teaching me something far more important than technical concepts, syntax, and best coding practices.They were teaching me how to problem solve. And the rubber duck helped me learn that skill.

I strongly believe that programming is one of the most intense problem solving exercises a human being can perform, excluding highly academic fields such as theoretical physics and math. Rubber duck debugging was a powerful method to help me learn how to think through problems. I eventually purchased a rubber duck and placed it on my desk at my internship. While I never verbalized out loud to the duck what I was stuck on, I would often look at the duck and be reminded that if I was stuck, I likely needed to just think harder about the problem rather than go running to my senior dev. The rubber duck tradition was one I grew proud of. At Christmas, I bought members of my team rubber ducks, and I credit the small rubber duck for helping me become the programmer I am today. People have gifted me several rubber ducks over the years, as I went from intern to full time developer, and I now have a small collection on my desk.

I sometimes wonder if I am the last student to leave college with a rubber duck. My final year of college, ChatGPT entered the scene. Now, students have a rubber duck that talks back to them. In many ways, this is a huge win. Students are able to overcome problems far more quickly. It’s like adding GPS to your car. However, as someone who grew up with GPS, I’ll be the first to tell you that without GPS, I have a much harder time getting around. Oh, I can still manage, but it’s not natural to me. It gives me anxiety. In a similar vein, I wonder about the next generation of students emerging from college. They’ve been given GPS, in a sense. I know they can get stuff done with AI. They’ll likely be building faster than students coming out of college 5 years ago. But part of me is sad that students won’t get to experience rubber duck debugging. Will the joy of solving a particularly nasty programming bug be the same if it’s ChatGPT that solved it instead of you? Influencers like Primagen often talk about this deep love of programming that they have. ChatGPT robs you of some of that. It transforms the programmer from director to actor, the same way that GPS lobotomizes drivers into mindlessly following instructions, even if it takes you off of a cliff.

I counteract that concern with the optimism that we still get to build amazing things with code, and take pride in the end result. The joy of programming was often in the journey before the destination. But now that the journey is automated, we programmers will have to learn to enjoy the destination a bit more. Even if new programmers don’t get to experience the pleasure of rubber duck debugging, they still get to enjoy the amazing things they build. So now my rubber ducks now slowly gather dust while my list of conversations with ChatGPT grows longer and longer. They sit there, a reminder of the way things used to be, a relic for new programmers visiting my desk to ask about.

GPT-4o stands for Ghibli Photo Transformer

Gotta start with the most viral of updates. After the last few updates from OpenAI ranged from disappointing (GPT-4.5 was big but expensive, GPT-4o-transcription was accurate but lacked timestamps) to highly niche (the new agents SDK is cool, but is targeted at keeping developers in the OpenAI ecosystem), they finally dropped a banger. A year ago, OpenAI had teased native image generation from GPT-4o. They’ve finally added that image capability in the ChatGPT UI, with availability in the API promised in the coming weeks.

To describe the image generation capabilities of GPT-4o as state of the art is an understatement. To be honest, I had wondered if OpenAI had given up on image generation as DALLE-3 lagged further and further behind competitors with the passage of time. With the release of native image generation via GPT-4o, OpenAI has reclaimed their title and put the doubters to rest.

GPT-4o can view and output images as tokens, generating images pixel by pixel, row by row. This is in contrast to the vast majority of image models, which use a diffusion approach (noisy image -> not noisy image). It raises the question of whether the transformer model is the machine learning model to rule them all. The ability of GPT-4o to generate images or modify existing ones (like the one above I made from a family photo in England) stands above the many other options on the market. This has led to a huge wave of people posting images in the style of Studio Ghibli, which once again raises the question of copyright laws and the rights of artists in the age of AI.

Model Context Protocol Gains Traction

Last year, Anthropic released the Model Context Protocol (MCP), an ambitious attempt at standardizing how LLMs interact with tools and functions. While MCP still needs to be polished and expanded beyond niche desktop applications, in the last couple months it has rapidly gained traction and generated excitement.Perhaps the most telling sign is OpenAI’s quiet adoption of MCP into their Agents SDK and UI. Companies are putting out MCP servers left and right, and OpenAI sees the writing on the wall. The MCP roadmap is visionary and has the potential to become the bedrock for unlocking the next level of agentic systems if executed well. If Anthropic can successfully pull off their roadmap, building agents will become ridiculously easy. Companies will be able to quickly spin up agents that can access hundreds of applications.

Models, Models, Models

Gemini 2.5 experimental released and is crushing benchmarks. What’s crazy is it’s literally free to use. Google is desperately fighting to re-establish it’s dominance in the AI sphere. Meanwhile, Chinese firms are dropping models left and right, but the West doesn’t seem to be losing it’s mind like it did with Deepseek-R1. Speaking of which, Deepseek-v3.1 released, and it’s also crushing benchmarks. Qwen 2.5-omni by Alibaba, a powerful multimodal model, and T1 by Tencent also released.

Vibe Coding

Everyone is vibe coding, but what does that even mean? Nobody knows. For some, it means the next generation of coding efficiency. For others, it means churning out AI slop that will break once you try to scale it past 10 users. Either way, it’s here to stay. Will code become like clothes? We used to maintain the same set of clothes and constantly repair them, but factories made it cheaper to buy new clothes rather than repair. Will generating code become so easy we throw away codebases and generate new ones because it’s easier to start from scratch with AI rather than try to fix the AI slop? A terrifying thought….

Bottom line - AI isn’t slowing down, and it’s never been easier to violate copyright laws.

In 2024, two companies had their “GPT 3.5” moments for AI music. Udio launched strongly in April with steady improvements throughout the year, while Suno released their impressive v4 model on November 15, 2024, less than two years after the ChatGPT debut. AI-generated music had become nearly indistinguishable from human-created tracks.

Over the past few months, I rabbit-holed into AI music. After releasing an album created completely with AI, I have compiled my list of lessons learned. Below, I’ll share details about the album and then walk you through the lessons learned.

Album Context

Before we jump into lessons learned, here are a few interesting and important pieces of context about the album. For lyrics I adapted my favorite poems (each written over a hundred years ago) into a modern medium. This choice avoided copyright concerns and added personal significance. For the music I sought the genre that the original authors may have actually chosen. Except Rudyard Kipling. I don’t think Appalachian bluegrass would have been the English author’s first pick, but it fits the poem quite well. A few album highlights include:

• The album opens with an orchestral rock version of Ozymandias, featuring a rap verse from Destruction of Sennacherib that emphasizes the fleeting nature of our lives.

• What if Edgar Allan Poe were alive in the early 2000’s composing an emo acoustic guitar album? Alone, my favorite poem in the early 2000’s, provides a glimpse of what this may have sounded like.

• My two favorite World War I poems blend poignantly together in The Soldier in Flanders Fields.

• At my wedding luncheon, my wife recited Sonnet 115 from Shakespeare and I Gave Myself to Him by Emily Dickinson. The fusion worked well.

Where to listen?

• Spotify

• Apple Music

• YouTube

• Amazon Music

• Pandora

Lessons Learned from the Album Project

Lesson 1: AI Music has Arrived, but Progress is Ongoing.

While the current models are impressive, there are still obvious, low-hanging improvements waiting to be made. Over the last six months the improvements have been consistent and will continue. The mind blowing moment for me is when the AI, unprompted, added a sword swooshing sound to the song Jabberwocky.

Lesson 2: Model Diversity will Continue.

Different companies are taking distinct approaches, with each major player developing models that exhibit unique strengths and weaknesses with no signs yet of convergence.

Lesson 3: Component-Based AI Music Creation Is Key.

Focusing on constructing individual components of music rather than complete tracks will be incredibly valuable. The first to master this approach will likely dominate the field. By separating tracks for different instruments, a new realm of possibilities opens up for both amateurs and professionals.

Lesson 4: AI Music Sparks Controversy.

Unlike text, where LLMs generally learn from the entire internet and do a decent job of avoiding plagiarism, AI music raises unique concerns. Some musicians develop a distinctive sound, and AI can replicate that unique feel, which is a valid objection. Some drafts of my songs sounded very similar to specific artists. There’s an opportunity to make AI music more artist-friendly by combining a streamlined, powerful audio mixing platform with models trained in ways that satisfy participating artists, potentially even compensating them. One innovative company to watch in this “artist-friendly” space is Kits.ai, known for their impressive vocal models. Their Text-to-Vocals model diagram may not be surprising at first glance, but the output is remarkably sharp.

Lesson 5: Music as a Personal Emotional Journey

Music can be profoundly personal. This album features some of my favorite poetry, much of which my late mother introduced me to. While these pieces were deeply moving to me, creating novel and deeply meaningful music as a generally weak musical artist sparked powerful feelings.

Lesson 6: The Imminence of On-Demand Personalized Music

I would wager we are less than 2.5 years away from on-demand personalized music, though I’m not convinced we’re ready for it. I once spent three hours listening to 20 different versions of Edgar Allan Poe’s Alone. The idea of music becoming the next addictive TikTok equivalent hasn’t fully hit most people yet.

I’ll share more about the potential dangers stemming from Lessons 5 and 6 at a later time. For now, enjoy the rhythms. 🎶

Despite the barrage of praise and engagement, you did not find the LinkedIn sales funnel bots motivating. Despite the DMs showing interest and *ahem* encouragement, you did not find X’s 0nly fans bots motivating. (Although, more people find the latter encouraging, which, in retrospect, isn’t altogether unexpected. LinkedIn bots are demotivating.)

No, instead you overwhelmingly chose Tom from Myspace.

Our first friend. (This is ancient pre-Facebook days for you young people). Yes, we would prefer to go back to 2005 to receive periodic encouraging messages and announcements from Tom. Tom, the only person I ever included in my “top friends” list on the MySpace platform.

The progress with AI is going to be a rocky journey. Two steps forward, one step back. But some of those steps back can be large and painful. Tidal waves of AI slop will be one of them. Hugely impactful non-AI breakthroughs drowned out by the AI hype will be another.

Despite all the headwinds and disruptions, I wouldn’t trade it back though. Well, maybe a little if we could get Tom back.