Simplicity

One of the key values of the Agile software development mindset is simplicity. Maxims such as “do the simplest thing that can possibly work” are very popular, and developers swear by them.

Unfortunately, simplicity isn’t quite that simple.

Let’s look at an example. At one point last year I was brought in as a consultant to help add some functionality to a project. The project was staffed by a team of relatively young, idealistic software developers who took great pains to always do the simplest thing at any particular point.

One of the components of this project was a back end Rest API, which was written in Python, as that was the simplest possible way of exposing a Rest API for the team at the time. The front end component was a dynamic reporting application written in Ruby and Javascript. It involved rendering interactive charts, which the modern javascript front end (based on d3.js if I recall correctly) handled well.

The project had a requirement for the charts to degrade gracefully for older browsers, browsers with javascript disabled, search engines, screen readers, and so on. The conclusion was that since the chart rendering was all written in javascript, the simplest way to degrade would be the render the charts as jpgs or pngs server side using Node.JS, which could use the same rendering code as the client side engine.

The team continued on, building each new feature to meet a requirement in the simplest possible way, for a period of 6 months or so.

By the time I got there, this “simple application” consisted of over a dozen components, written in Ruby, Python, and two different flavours of Javascript, all held together by an increasingly more complicated and fragile patchwork of infrastructure to coordinate the many components.

Developing each component in its own git repository for maximum simplicity meant that to run the entire application you needed to download more than a dozen git repositories. The sheer number of repositories involved meant that the best way to coordinate all of this interaction was a script, hosted in yet another git repository. The different version requirements for all of the python, ruby, and node libraries used were difficult to manage across all of the components, so to simplify things, development happened inside of a virtual machine. However, some of the things the virtual machine needed (such as standard DNS entries) were complicated to update and maintain, so additional development infrastructure that ran outside of the virtual machine on a developer workstation was created. This infrastructure was written for a specific version of ruby, and had specific libraries which had to be installed, and so on.

The entire team of about 8 developers wasted several person days per month just maintaining all of this development infrastructure.

The interesting thing about this story isn’t just that the project had technical debt. The interesting thing is that this debt was accumulated, relatively quickly, by a team that focused on simplicity at almost every opportunity. The mistake that was made was to focus on the wrong kind of simplicity.

Local simplicity versus global simplicity

Every time this team was given a new requirement, they honestly tried to do the simplest thing that could possibly work. The problem was that they framed the conversation by asking the question “what is the simplest possible way to add [Feature X]?”. What they should have done instead was ask the question “what does the simplest possible overall system that includes [Feature X] look like?”

This small change of focus from local simplicity (what is the simplest way to add the feature) to global simplicity (once we add this feature, what is the simplest way the overall system could be designed) can make all of the difference in the world.

Rather than just build the feature on top of the existing system, instead the practice becomes to refactor the entire system into a form where adding the new feature results in something that is still very simple and straightforward. Sometimes the team’s foresight is lucky, and this refactoring is minimal. Sometimes the team’s foresight is less lucky, and the refactoring required is not minimal. Either way, the “simplest thing that can possibly be done” is to do the work.

It’s not always easy for a team to admit to itself that it needs to radically revisit some work that it has just completed. Sometimes is can be a very humbling experience. But it is much better for both the team and the project to take this in a number of small doses early on than it is to experience it in one very large dose later.

Let’s take another look at the project. While it’s true that the a number of individual features in the project would have been more work if they couldn’t be written in exactly the “right language” and simplest way for that specific job, the overall system would surely have been cheaper to the manage and maintain with only a single component, with only a single codebase. While the speed of development might have slowed in places because an individual feature was more expensive to build, the overall long term pace of development would have been much faster, and that’s the statistic that actually matters in the end.

This is why one of Equal Experts’ technical values is:

“We value overall simplicity over localised simplicity”.