A great article came across my desk this morning: Can you know too much about your organization? The TL:DR version is that a bunch of managers were tasked with organizational transformation via reconsidering their processes from first principles. What they found almost invariably shattered their mental model of the firm and their role within it. This caused a crisis within them, resulting in many of them abandoning their position of authority altogether.

This is because they derived a great deal of meaning from their profession. Like rational science had in the early 20th century, they've peeled the onion and discovered the world they thought they lived in was an illusion. Those given the greatest authority in the firm turn out to be the most powerless to effect positive change in the production process. The actual means by which decisions get made in the firm are a rats' nest of bypasses often held up by the force of will in singular individuals.

Many of these individuals (such as staff engineers) also have a crisis of meaningness when and if they realize their vast skills are essentially wasted being a glorified "glue stick" holding together a system which is perverse, and for no real purpose.

This happened to me. Coming out of QA means I was very concerned with catching things as early as possible, and thereby reducing the cost involved. This evolved into a particular interest in shaving off the "sharp corners" of software production processes, as it was time wasted on these largely preventing better early scrutiny. Paul Graham has a great article on the subject called Schlep Blindness, but the concept is well-encapsulated within Kanban.

The poster child for this in modern development organizations is using CODEOWNERS files as a means to prevent howlers from slipping by in large monorepos. Many like monorepos because it theoretically means that less time is wasted by developers hunting down code and making many PRs. Having to impose a CODEOWNERS regime in a monorepo implies that the automated testing corpus is far from adequate for catching bad changes. It instantly negates 100% of the possible advantage one can achieve through usage of a monorepo. In both situations, every second spent chasing people down to approve changesets and splitting changesets into multiple pull requests is time far better spent writing tests. This purported solution only gives the feeling that things are under control while slowly and corrosively making the problem worse.

I took a look at the PR history for one of these monorepos, and sorted it into buckets. It turns out the vast majority of changes required approval by at least 3 groups, and had at least one merge conflict result in seeking approval multiple times from the same people. Even the best case estimate of how much time was wasted here (notwithstanding how many people simply drag feet and become discouraged) was quite troubling. At least 1 man-lifetimes a year were spent on this, and this was a firm with less than a thousand developers. This amounts to human sacrifice to no productive end, and there are many more examples of this and worse lurking in the modern software development organization. Lord knows I've spent unhealthy amounts of my life dealing with bikeshedding masquerading as "standards" over the years.

It is easy to then lose heart when you consider the consequences of actually fixing these problems. Chesterton's Fence comes to mind. The problem that made this feel necessary likely hasn't (and won't) go away anytime soon, and the Lindy Effect is likely in play. This is why the managers in TFA reported huge levels of alienation and many even changed careers once they understood they were dealing with a gordian knot they could not just cut.

Similarly, most individual contributors simply "check out" mentally when they realize there's not only nobody else willing to strike the root, but all attempts to do so will be savagely punished. Like with the Rationalist's crisis of Meaningness, thinking on another level of abstraction is required to actually cut the knot.

Most seemingly intractable problems in production lines are because the approach used does not scale. Like in computer science, you must re-frame the problem. Rather than solve an NP-Hard problem, solve a subset of the problem which can be handled in linear time.

The solution to the particular problem I've used as the example here (unwieldy and untested big repos) involves understanding how they came to be so in the first place. The reality of business is that the incentive to cut corners to meet deadlines will always be present. The larger the organization becomes, the more its decision-making will resemble total acephaly and incoherence. Steps must be taken to reduce the impact of this.

To date the most effective mechanism for this has been Autocephaly. Regardless of how many corners are cut, or doctrinal corruption is tolerated in one bishopric, it cannot fully infect the body. In the modern firm this was first implemented as divisions; Peter Drucker's Concept of the Corporation covered this in 1946! The modern software firm's analog to this is called Service Oriented Archetechure.

Meta-Rational approaches are always like this. They are strong because they recognize the common and intractable failure modes present and contain them rather than attempt to stamp them out. Much of this is why both free markets and political decentralization have proven so durable. For all their faults, they effectively limit the impact of any given group going catastrophically sideways.

Nevertheless, there are always growing pains. The reality of power dynamics means that things subdivided will usually not subsequently subdivide once more until far past the point it is once again necessary. Sometimes subdivision "in name only" such as Scrum Teams occur. This introduces its' own set of pathological behavior for which entire firms base their livelihood upon servicing.

Rather than become alienated and hopeless upon discovering the reality of corporate existence, a re-orientation to not fight this flow re-establishes meaning. The participants in the firm can once again proceed forward taking pride in their corner of the great work. Even in firms which failed to scale and reverted to de-facto acephaly you can do good work when you realize what does and does not work there. Given I've had a lot of experience with the latter, I'll write a follow-up soon on how to be effective in acephalous organizations.