AI is Mediocre (Metaphors for AI)

As many folks worry about LLMs racing towards AGI and the eventual machine apocalypse, the more I work with LLMs on coding, the more I'm coming to the opposite conclusion: AIs are actually pretty average. This is actually a fairly wondrous achievement we would have believed was barely possible a few years ago, but it's also not the story we're being sold by OpenAI or Anthropic or the breathless benchmarks we see all the time.

Now, when I say AI is average - this isn't quite the dig you might expect. AI is an average programmer, but it's also an average Doctor, Realtor, Storyteller, Psychologist, Gardener. It's average at everything (or at least everything that's been written about publicly). And before you get angry with me - don't forget that being average at these things means that 50% of professionals practicing these disciplines are worse. About half the human race, by definition, is worse than average. Stepping back, given that it's trained on all the garbage you find on the internet, both good and bad, it's no surprise it's kinda average. Only the last couple steps SFT (Supervised fine-tuning) and RLHF (Reinforcement Learning from Human Feedback) push it above the average, but the scale of those pales in comparison to the overall quantity of data in pre-training.

This is an idea that becomes obvious when AI is writing code in a language or framework I know a lot about, Ruby, Rails, and React, and I often shudder at what I see it write. It sometimes has more up-to-date knowledge than I do, but it's not writing particularly wonderful or beautiful code when I step back and look.

This is a fact, however, I quickly forget about when it does work in areas I have less-than-average experience in. Swift MVVM. Kotlin and Jetpack. When it's working in those areas, I'm amazed at its ability to quickly crank out working code.

There's actually already a similar phenomenon when it comes to journalism that was coined by Michael Crichton (yes, that one) called the Gell-Mann Amnesia effect:

While LLMs aren't quite as bad as the aforementioned journalists, I think this is a useful lens for thinking about LLMs as coders and in general - what do you need to be careful about if you were to give this problem to an average coder, just one that happens to be able to work so fast you can't really monitor the process, just the output. What are the things you need to put in place in your system - the feedback loops, the constraints and checks - that help that average coder achieve excellence, so that they can get their job done even quicker without mucking up the works.

With the "average" lens - the Architecture issues, Non-DRY repetition, tight coupling - all make sense, and when it's designing something, making sure to give it a kick in the right direction (use pattern X, use architecture Y, please do research on Z) all help an "average" AI achieve above average results.

See more Metaphors for AI Development