Anti-concretion

I recently re-watched The Only Unbreakable Law which breaks down Conway's Law of organizational structure and communication overhead. What stood out to me this time around was the conflict between optimal behavior and abstraction.

In essence, we humans are limited in how we can solve problems. We really don't have much in our problem-solving tool belt other than to take a problem, break it up into pieces, solve those easier problems, assemble, and refine. We can refine a few sets of concepts or pieces at a time once we've identified them, but the structure will be largely the same as what we started with and heavily influenced by the design we already have. A premise of Conway's Law is that how we decide to slice up the problem in that initial first pass, shapes what allowable forms we take to solve the problem at all. Communication costs immediately following that slicing and maintenance costs longer term cement these problem slices as boundaries and artifacts.

I saw this in practice just this week where I was solving a trivial to describe user-facing feature request. The request was for a CSV of all emails sent to a given recipient with about 10 fields of data for each email. This ought to be straightforward in implementation as well but as tiny requirements unearthed themselves, the architecture underneath needed vast iteration. This difficulty is because of how the existing system was designed. Some tasks are easy and some are hard and much of this had to do with how the system had grown over the past five years.

A lot of what it means to be successful in designing things in a large software company is simply knowing your tool belt for solving a problem. "For this we'll need two database tables, a web service, three cron jobs, logs and metrics here and here, detectors built on those, and this feature flag to control the rollout." This recipe of abstractions becomes so commonplace that the abstractions can start to feel a lot more like concretions, like Lego building blocks.

Lego's really are the perfect metaphor for abstractions. Yep you can stitch them together all you want but it's gonna come out looking blocky & cookie cutter. If you want to make a modification you'll have to break up that big construction and strip many pieces away and then add them back to get something good. Oh but it's so much easier to just add more on top. If you step on abstraction the wrong way it's gonna hurt a lot. Etc, etc. Probably the only place we give abstractions as lego's too much credit is that it assumes they connect together at all. We need glue code to get abstractions to fit.

What's been on my mind lately is anti-abstraction. Can we escape Conway's Law or build something capable of it?

Going back to the problem solving tool belt, instead of dissecting a problem, how can we solve it? Can we really solve a large problem all in one go?

Suppose we say we can't as humans approach the problem any other way (randomness doesn't work either), can we devise an optimizing program to take our abstraction littered MVP and unwind it into the essentials needed to solve a problem?

Conway's Law dominates so much of what I design today. There must be this alternate reality of design approaches that haven't been taken simply because the one tasked with taking them was a human or set of humans. More abstractly, we're unable to design the very best solutions that may exist because there exists communication overhead, a limited capacity in each node, and a history of previous abstractions. A perfect design is physically limited by the speed of light, conservation of energy, and time.

Abstraction is an overloaded term. The abstractions we build today are concretions, a loose representation of a generic solution that is poorly implemented for at least one thing or more – it's not universal. A true abstraction is what's left when everything is boiled away and it can't get any more simple. The only thing that could be left is an abstraction. (This is something you can hear a different way watching Simple Made Easy.)

I'm not suggesting we consider in our day to day how we might consider breaking the laws of physics to achieve new, better designs. Conway's Law to me is a challenge to solving problems with few pieces, fewer problem space slices, to achieve better designs and more fluidity in our solutions. There can be too much decomposition and we need to be mindful of those costs.