AI Slop: The Future of Software Engineering

Introduction:

To state the obvious, it feels that in almost every dimension of human life, we’re being bombarded with AI slop. Product reviews, ad copy, code reviews, robo-calls, job applications, resumes, movies, art, music, search results, and every single thing on LinkedIn for the last year. Everything we see or read seems like it could be fake, but there’s not enough time in the world to sort it all out. It’s as if the world’s engineers, data centers, power plants, and investors have joined forces with one goal: absolute overwhelm us with high-volume, low quality information.

Our age carries with it a special variety of malaise and overwhelm. I hope that in this article I can help you with a conceptual framework and vocabulary that you can use to express, and understand, this overwhelm: Cybernetics!

AI Generated Code

I’m going to start with an example problem, and then I’m going to use some tools of cybernetics to describe it:

Reviewing AI-generated code is…nauseating.

Producing AI-generated code is…exciting.

The first time a code assistant spits out a giant Object-Relational Mapping(ORM) for you, it feels pretty magical. Writing these can feel tedious, uninteresting.

The first time somebody sends you a PR and asks you to review thousands of lines of AI generated ORM, however, is not so magical. The mistakes made are incredibly hard to see. Declaring the same variables multiple times, using libraries that are usually imported rather than what is, referencing objects that don’t exist yet, logic bugs in unit tests that make them impossible to fail. Sometimes even rewriting the standard library or overloading standard methods. These mistakes are sometimes valid code, compile, and even work…but are still profoundly illogical and irritating to try to read. When humans reach the edge of their knowledge, we degrade in human ways: we seek help, bang our head on the desk, make elementary mistakes, etc. Other humans are prepared to recognize these failures, and help.

When these LLM-based tools reach that same edge, they degrade in ways that are best described as…1% dissimilar to functioning code? Or, maybe, “the most statistically accurate imitation of the right code”? In other words, the “AI Fingers” problem…but the kind that gets you paged in the middle of the night. The mistakes here are literally engineered to be hard to detect because they are so similar to working code. Humans are, simply put, not ready to recognize these failures. They are the code equivalent to an optical illusion.

Using Cybernetics to Describe This Situation:

Variety in Cybernetics describes the number of potential states that a system can take.

A simple example is a traffic light — it can be red, yellow, green, or unpowered. 4 states. This small number makes them extremely easy to deal with as a driver, they fit in our busy brain almost unconsciously. The systems that control them can be extremely simple.

Ross Ashby, an important early contributor to the field of Cybernetics who focused primarily on self-regulating systems, adaptation, and complexity, coined The Law of Requisite Variety:

“the variety in the control system must be equal to or larger than the variety of the perturbations in order to achieve control”

Often paraphrased as “only variety can attenuate variety”.

Shannon Entropy(named after Claude Shannon), is a fundamental concept in information theory that measures the uncertainty or unpredictability of a system. It quantifies how much information is contained in a message or how uncertain we are about the outcome of a random variable.

Variety and entropy are related — the greater the variety of a system, the greater its entropy. If you’ve ever scaled software products, you’ve experienced the fact that with scale comes…weirdness. If you run 50,000 instances of a given architecture, it’s just a given that somewhere out there, one of them is broken in a way that’s never been seen before.

In the code review example above, the variety that our AI coding assistant is capable of exceeds the variety that any one individual is capable of. After all, they’re an amalgam of all of the available coding styles it was possible to consume, but you and I are an amalgam of only one lifetime. This is why teams employ coding standards — to reduce the variety of the code produced, hoping to scope it to a common set of patterns that align with the variety that reviewers are capable of.

We can also see that the entropy present in the coding assistant’s model is qualitatively different. A system with a high number of possible outputs like an AI coding assistant has very high entropy. Balancing entropy in these models is critical to them becoming useful — but the “safer” they are from entropy, the more likely they are to tell you something like “do a loop”, which just isn’t terribly impressive. These are frothy times in the AI market, so it’s probably safe to say that every coding assistant is going to be tuned to the maximum entropy a buyer will accept for the foreseeable future.

Managing Variety and Risk

Overwhelming a system(or your coworker)with a high variety output might seem like fun — in fact, it’s easily confused by both sides as intelligence. And who doesn’t want to use the magic “be smarter” machine? Doing this deliberately is an act of aggression, used to overwhelm. Just ask Steve Bannon, for whom it is the primary tactic for swamping regulatory and legislative systems. Or observe debates on almost any “controversial” topic, where you’ll see techniques like the “Gish gallop” deployed as a matter of course.

Intelligence and variety might be correlated. The size of a person’s vocabulary, the range of their experiences… these things are associated with intelligence for a good reason. But that’s not the whole story; most of us can point to at least one well-spoken idiot who made our lives worse. Intention matters.

Good engineers make complex tasks simple, specifically with the goal of reducing and attuning their variety to the level that the systems involved can accept. They write code for the receiver — the operators, the reviewers, the customers. They write code for themselves, when they’ve forgotten all the details about the current project, and someone brings them a bug in the distant future. This is because they understand that variety is risk in technical systems.

We have two options for managing the risk that comes from escalating variety:

Increase the variety that the control system is capable of.

This looks like employing AI in code reviews…one which is capable of equal or more amounts of variety. This might work in the coding situation, because the same parties are in control of the system and it’s regulator. It is not hard to imagine how ugly this particular option can get in cases where that’s not true — like job application processes, the legal system, cybersecurity, and others. At those edges, there will be a disastrous arms race of competing AIs.

Strategically decrease the variety of the system under control.

In humans, this looks like our example senior engineer mentioned above. Reducing the number of ways a given thing is accomplished, simplifying aggressively, so that the code provided is maintainable and comprehensible to others. In AI, this looks like ensuring that your model is trained on your coding standards, constrained to smaller changes, heavily supervised, etc.

Conclusion

Even in a quick example like this, it’s not hard to see how much a little bit of vocabulary from cybernetics can help you move the discussion of these topics forward from hype and into reality. Without shorthand for complex concepts, we’re left only with the option of shouting marketing terms at each other, louder and louder. This is the grim reality of our current discourse.

Cybernetics is hard to understand. As an interdisciplinary field, it is guaranteed that you’ll find yourself way out of your comfort zone. But I believe the time has come for it to be more broadly recognized and included in discourse, and the muli-faceted, complex threats to humanity that AI poses makes this study worth your time. If you’re looking for an entry point into the field, you simply won’t find one. Considering picking up Norbert Weiner’s The Human Use of Human Beings, or listening to some of Stafford Beer’s 1973 Massey lectures, Designing Freedom.

Special Context for the Future Reader:

At the time of this writing, it remains unclear whether or not these systems will hit parity with, much less exceed, humans. LLMs have passed the Turing Test, but they’ve achieved this by…stealing almost all of the information ever produced by humanity. We cannot produce another millenia of media in a year, so we’ll have to get better at how we ingest it and make it meaningful. Some of us think that for cost and computational reasons, we just won’t, and that this is the end of the road. Some of us think that AGI is right around the corner and that “the singularity” will usher in a wonderful/terrible new era of humanity. This time in history reminds me a lot of the era in which we called the internet cyberspace and earnestly believed that we’d live a life in Lawnmower Man-grade virtual reality. In other words, it is the season of life for wildly stupid, ambitious ideas. Exciting times!