I’m calling it now, the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history. Agents cannot program, and it’s taking longer and longer to realize that they can’t. They are a highly sophisticated statistical model designed to mimic the distribution of programming. The output is broken, but in a way that’s getting harder and harder to detect. Which is exactly what you’d expect from an increasingly accurate statistical model.

Source: The Eternal Sloptember

geohot is now ringing some alarm bells. This is just the latest in a series of the vibe turning on agents as a panacea.

The Earendil works folks, now also the patreon of Pi have always been clanker-skeptic agent users. They are also doing some great work trying to both embrace but be rigorous on what works and what doesn’t in this world:

The most frustrating failure mode right now is that people submit issues that are not in their own voice. They contain an observed problem somewhere, but it has been thrown into a clanker and the clanker reworded it and made a huge mess of it. Typically, it was prompted so badly that the conclusions produced are more often than not inaccurate but always full of confidence. The result is complete guesswork on root causes, fake-minimal repros, suggested implementation strategies, analogies to adjacent but often the wrong code, and long lists of error classes that might or might not matter.

Source: Building Pi With Pi - Armin Ronacher’s Thoughts and Writings

The folks at Every publication have a long report that suggests something similar - all the people lazily laying off humans because they think that AI is going to take over everything are in for a rude surprise.

AI makes yesterday’s human competence cheap Cheap competence gets rapidly adopted Abundance creates sameness—old expertise becomes commoditized Sameness creates a demand for difference Demand for difference is new demand for expert … However, I am arguing that, regardless of your current job, there is a form of work that stays structurally ahead of the models—using them to address today’s problems as you see them. That’s where knowledge work is headed.

Source: After Automation - Every

So, as I noted above - the vibe is turning.

Slop existed long before AI

As I noted a couple days ago, slop was always present - even before clankers. Slop is a result of a lack of care. What’s increasingly clear is that currently clankers generate a ton of output. Output that humans have to verify and ensure for quality. What happens when you overwork the human / put them in charge of things they don’t actually care about? Quality suffers.

The more you care, the more you realize that AI isn’t quality

This is a correlated learning. The more I care about something, the more I find AI output to not meet my quality bar. This is why I give credence to Armin and geohot’s pieces. It’s worth considering - what if that were true?

This is not writing LLMs off. And neither are they. The question is keeping an eye out for - what are the labs and Google doing? 1) are they recognizing these are the issues? 2) Are they moving results to be better for these cases and how?

Fwiw, at least Demis seems to think that replacing engineers en masse with AI is short sighted.