Start slow: the counterintuitive approach to better results coding with LLMs

One of the fundamental paradoxes of coding with LLMs is that they go fast when they should go slow and they go slow when they should go fast. Let me explain.

At the very start of a new project when the core architecture and design decisions that will affect every subsequent feature are being made - LLMs are happy to spew out dozens of lines of code a second without a second thought.

Later on when all you need them to do is add in a couple more attributes to a GraphQL type, they will sit, ingest a whole bunch of code that they don’t need to, and spend 20 minutes adding in something for which there are already 30 great examples they could pattern match off of.

A lot of this comes down to how much the LLMs are relying on context - which is built slowly and carefully during an individual session - versus their built-in weights, which are instantly accessible and encoded in their trillion or so parameters.

But because coding harnesses make sure to use this existing context carefully as they build in an existing codebase, the more code there is the slower they go - as they will do their best to code within the confines of the existing architecture and patterns when possible. Fundamentally LLMs are still super smart completion engines so given some existing code they will often match what is already there with the new code that they write.

Why does this matter and what does this mean?

It means that at the beginning of a project, you are defining how another LLM will code with that project for the rest of the project's existence. Any mistakes the LLMs make when it comes to architecture, style or factoring will need to be manually unwound by you down the road. Otherwise the LLM is going to be pretty darn happy just to keep applying those bad patterns ad infinitum.

So this means slowing the LLM down at the start of a project very intentionally. Reading more of generated plan documents. Reading more of the code and file layout and architecture as actually implemented. A good file layout and factoring will pay dividends in the future as less context is needed to make changes. Clearing context and asking for some reasoning and breakdowns of what was built and making sure it maps to your mental representation.

All this is to say go slow when you want to go fast and then you will be able to go faster when the LLM is going slow. Otherwise you run into the infamous vibe code cliff that stops many promising projects in their tracks as work slows to a crawl and regressions start outweighing progress.