Chaos and Order

The world (and software development) can be chaotic, so we have conventions and rules to bring about order. This enables progress, but we must also recognise the cost.

In The Beginning

In a chaotic world everything is unpredictable and nothing makes sense. The outcomes of your actions are random, which makes it impossible to move towards a goal. Individuals act in their own self interest without any cooperation or collective effort, further stifling progress.

So, since time immemorial, we've developed social norms and conventions to tame the chaos and bring a degree of order. If we have an idea of how others will interact with us then we can make our way in the world, and we can make some progress. We also developed common values, enabling us to agree on important issues and collective goals, accelerating progress further.

For this reason we've been able to organise in greater numbers than any other species, and impacted the world accordingly.

In Groups

As societies continued to evolve some of our norms became rules and laws, and some of our values became morals, further increasing orderliness. However, if left unchecked this process can continue until the role of the collective overwhelms the individual. This can make for highly efficient organisations, but it's disastrous for innovation and creativity.

On the spectrum from order to chaos things can get pretty bleak at each end, but luckily the wisdom of the ages gives us ways to recognise and deal with the problem.

The Taijitu, used in Daoism and other philosophies, symbolises how seemingly contrary forces are often complementary and inter-dependent. Everything is said to have yin and yang, and the goal is to find the right balance between the two.

In Harmony

The particular duality of order and chaos manifests all around us, with interesting examples as diverse as: Sports, with enough rules to makes things comprehensible without stifling individual creativity and flair; Music, where composers blend regular scales, rhythms and harmonies with occasional dissonance, accidentals and free time; Randomized algorithms such as Monte Carlo, which benefit from random input; and modern democratic systems where the left (keen to embrace the unknown) are (in theory) kept in check by the right (emphasising the importance of tradition) and vice versa (in theory).

Finally, what's interesting to note about the Taijitu is the black dot within the white and the white within the black. The former warns us that extreme orderliness can render a particular type of tyrannical chaos, where the individual's actions become overwhelmed by the will of the group, and therefore meaningless. Conversely, the latter represents how order can, sometimes, emerge from a sustained period of chaos. This typically unpleasant process is familiar in its milder forms as benefitting from time spent outside your comfort zone, while in the more severe case it's captured by the adage from the hottest fire comes the strongest steel.

In Software Development

As interesting as this may be it's fair to question whether it's applicable in the workplace or, more specifically, to software development teams. It turns out to be remarkably easy to find examples where the general ideas expressed above ring true in our everyday experience as software developers.

You'll recognise chaos if:

• You've got tests in CI that fail intermittently.
• You're afraid to merge the latest code from source control in case the app is currently broken.
• You've ran npm install and everything stopped working.
• Your continuous deployment fails intermittently.
• Your TDD feedback cycle is slow due to a lack of CPU or memory and can stall due to background processes.
• You're not sure if anyone else is working on the same feature as you.
• You're not sure if another team is working on something that could negate what your team's working on.
• You're not sure if the customer even needs the feature you're working on.
• You're not sure if your colleagues are on leave or if they'll be in today.
• Domain terminology is used inconsistently throughout code and documentation.
• Daily standup takes anywhere from 5 minutes to an hour and a half.
• You've noticed The Churn.

The list goes on, and it can even strike lower down Maslow's Hierarchy of needs, e.g. you're not sure if you'll be in a stable, comfortable working environment from day to day, or simple HR issues aren't treated consistently.

If you recognise any of the above then you need to find a way to bring about some order. When the things on this list rear their head they can wipe hours from your day, even days from your week, devastating productivity. However, they can all be addressed: some need time, some need money, some need communication, some need leadership, but all require attention. If you don't? Well, it's true that if you spend enough time in the wilderness you may just end up with JQuery, but I wouldn't call that a good strategy.

In Reality

But can you, as history teaches us, take it too far?

• Static analysis tools are great for ensuring consistent coding style, but do you really want to break the build and block the release to UAT because someone missed punctuation in a comment?
• Code reviews are a good way to ensure code quality and share knowledge, but a short list of mandatory reviewers can introduce bottlenecks that hold developers back.
• Sticking with tried and tested frameworks and languages may get every new project up and running quickly, but if you're still writing Visual Basic with Classic ASP you may struggle to deliver modern applications efficiently.
• Fixing version numbers can save headaches in the short term, but you risk becoming tied to a library, which ties you to other libraries, and a language version, which prevents you using new modern development features down the line.
• Company values help to focus everyone on common goals, but they shouldn't be allowed to trump your own personal ethics.

The good news is that the line between order and chaos isn't a razor's edge. You can walk it comfortably if you're aware of the warning signs as one side grows too dominant and you take careful, remedial action when necessary.

Small companies that move quickly, adapt to change and allow individuals to flourish are more likely to allow chaos to get a foothold. They're also more likely to draw order from chaos but beware of survivorship bias in any such anecdotes. If you're in a small company, therefore, consider how some consistent processes and predictability might improve things.

Big companies with layers of bureaucracy and explicit, overt company values are more susceptible to overdo orderliness. In these companies individual freedom is naturally limited so it can be difficult to change things. However, experimenting with technologies and blue sky days could pay dividends. You can develop piecemeal by changing one part of your stack, for example switch to TypeScript on the front end, while keeping the rest of the stack familiar.

In Conclusion

The good news is that we're naturally equipped to know on an emotional level when we're on the right path. To quote Jordan Peterson* of the University of Toronto, who speaks a lot on the ideas expressed here:

Stand with one foot in order, and one foot in chaos... You’re secure enough to be confident, but not so secure that you’re bored. You’re interested enough to be awake, but not so interested that you’re terrified. When you're in a state like that, when you find things interesting and meaningful, time slips by"

It's here that individuals can flourish, groups can co-operate, and organisations can succeed. That's what you should aim for.

* This article was written before the controversy of Peterson's resignation and political views became widely publicised. We've kept the quote in because within the context of this article and the author's intent at the time, we feel it adds relevance. It does not mean that we endorse or support Peterson's wider political views.