I had a conversation with a software engineer recently that changed how I think about something.

We were looking at a piece of code that had been implemented in a way neither of us could explain. I’ve come across plenty of these situations over the years: code that seems more complicated than it needs to be, dependencies between parts of a system that have no obvious reason to depend on each other, implementations that appear to go out of their way to make something harder.

I’ve generally assumed one of two explanations.

By far the most common is that there was a perfectly sensible reason. The world looked different when the decision was made, the circumstances have changed, and the reasoning has been lost along the way, but there was a good reason at the time the decision was made. Much less common is that someone simply didn’t know what they were doing.

This particular example had me stumped though. I couldn’t even imagine what the original rationale might have been, and I said as much.

“Maybe Claude wrote it” the engineer replied.

It was a throwaway comment, but it made me realise there’s now a third explanation to consider in these situations: that there wasn’t a reason at all.

A fence with nobody behind it

I’ve always tried to give the original engineers the benefit of the doubt when looking at puzzling solutions. A bit like an archaeologist, I try to reconstruct the circumstances in which something was built and work out why someone might have made the choices they did.

It’s essentially the principle behind Chesterton’s fence. If you come across a fence in the middle of a field and can’t see why it’s there, you shouldn’t take it down until you’ve found out. The assumption is that someone put it there for a reason, and that reason might still matter.

That assumption has served me well. But it relies on someone having thought about how the fence was built, not just deciding to put it there.

AI makes that a less safe assumption.

With AI it’s possible to describe a problem, have it generate a solution, check that it works and ship it, without ever really understanding why it was implemented the way it was.

The code works and the tests pass, but buried in the implementation might be architectural choices, dependencies or unnecessary complexity that nobody consciously considered.

Of course, being able to ship code you don’t fully understand isn’t something AI invented. Copying code from Stack Overflow didn’t guarantee anyone understood it either. But finding, selecting and adapting a solution generally required at least some engagement with its implementation. AI makes it possible to skip much of that process entirely.

Someone still made the decision to ship the code. But that doesn’t necessarily mean anyone made the decisions within it.

I’m not suggesting that’s what happened in our example. We have no evidence that AI was involved.

Nor do I think this makes AI a bad thing. I’m a big advocate for using it to help teams do more, faster, and I’ve used it to help write (and illustrate) this article. But every sentence here is one I’ve chosen to keep, and I could tell you why.

That’s the distinction that matters to me. Verifying that software works isn’t the same as understanding the decisions behind how it works, and AI makes it much easier to do the first without the second.

For most of my career, when I’ve found something in a product that makes no sense, I’ve assumed there’s a story behind it. That somewhere, at some point, someone made a decision, rightly or wrongly, for reasons I might not fully understand.

I’m less sure that will always be true.

And I’m beginning to wonder how much time we’ll spend in the future trying to understand decisions that nobody actually made.