Escaping the AI slowdown trap

You know the feeling - you start a new project with AI assisted software development and you're absolutely flying. Knocking down features like they are pins in a bowling alley. Then slowly, surely, your progress starts to slow down. Plans take longer. Features churn for those few extra tens of minutes. Progress no longer feels easy.

Why does this happen? Big picture-wise, it's pretty obvious: as a project grows, the LLM needs to understand more before it strikes. It needs to fill up its context with more information, read more files, and deal with more complexity than it did at the beginning when the project was a blank slate.

Is this inevitable? In some ways, yes - as a project grows, there is more to understand before writing code. Humans have dealt with this forever. To paraphrase Fred Brooks from The Mythical Man Month (1975): "Coding is easy, systems are hard."

But as this has been a problem since nearly the dawn of computing, people have been studying and working to solve this for a long time. The issue was also captured by Brooks in his seminal 1986 essay "No Silver Bullet". He broke the problem down into two types of complexity. Essential complexity, which is inherent to the problem itself, and Accidental Complexity, which comes from the tools and processes used.

AI helps dramatically with the Accidental Complexity - abstracting much away and perhaps even approaching the mythical 10x improvement we're all hoping for. But it struggles with the Essential Complexity, because now not only does that complexity exist, but it must be communicated in a succinct and repeatable way to an LLM that will have its memory wiped in 20 minutes.

That complexity, which is driven by complex requirements, large systems with many interacting parts, cannot be removed - it can only be managed. It is part of the domain of the business and specific to what's being built. This is why there is no silver bullet - as the solutions need to be tailored to the problem.

However, the same solutions people have been applying for a long time - reducing requirement complexity, creating architectural boundaries, reducing state with functional patterns - all things that help reduce the complexity of the problem being worked on right now versus the overall problem, are the key to keeping LLMs fast.

By reducing that mental (contextual) overhead for a project, and keeping strong contracts among various parts of the system, we can reduce the amount of time the LLM needs to spend gathering context before attacking the problem.

Some people have pushed this to a logical conclusion and ask "Is the answer just microservices for everything?", but this ignores the fact that microservices aren't perfect abstractions - they go down, have state, add complexity around migrations and versioning, and bring the complexity of their own data stores and scaling problems - which dramatically adds to the essential complexity of the problem instead of reducing it.

Instead, I'd focus on functional black boxes with strong contracts organized into in-app subsystems, applying ideas like The Clean Architecture paired with strong test suites to keep the blast radius of each change small. This is the opposite of "vibe coding" - don't let the LLM incrementally define a haphazard architecture, define it upfront and stick with it. LLMs love rules and process, so defining explicit layers and the contracts between those layers helps manage the complexity and ensures the project grows in the right ways. This has worked well for me in a number of projects (write ups coming soon).

At the end of the day, AI doesn't address the essential complexity of the project, it only helps smooth over the accidental complexity that surrounds it. The hard parts are still hard, and still beholden to the hand of the developer to shape and mold to fit the problem. The better we manage complexity at its source, the more efficiently those systems will grow in the long run.