Every successful multi-year software project that I’ve ever seen has had some sort of documentation.

Whether it’s a user manual, deployment guide, sample usage code, wiki, a run book for operations, release notes, javadoc, usage documentation for a REST API, or something else, there’s always some sort of documentation somewhere.

My views on documentation for software projects are roughly in line with the agile manifesto, which states “[we value] working software over comprehensive documentation”. This seems to be a very popular point of view in the agile community, if only because many developers would rather not write documentation in the first place. In general, with documentation (as with code), less is more.

Unfortunately, “less” usually doesn’t mean “none”. The truth is that other developers, operations teams, and architects almost always need at least some documentation. This is especially true for a consultancy like Equal Experts, because people will need this information when we’re gone.

And all forms of documentation have the same key weakness: they need to be maintained and kept up to date. Documentation that isn’t actively maintained is often worse than no documentation at all — many agile developers have had to recite the phrase “documentation lies” so often that it becomes a mantra.

In order to be successful, your project needs documentation, and you don’t want to end up accidentally lying to your users, so this documentation will need to be kept accurate and up to date over time. How do you do this without a painful manual review¹ stage as part of every release? This is especially important when you practice continuous delivery: reviewing documentation for each release when you release several times a day adds so much overhead that it’s basically impossible.

One strategy I like for keeping documentation accurate and up to date is to have the continuous integration process automatically verify as much of that documentation for you as it possibly can:

1. If your project has javadoc, make javadoc generation part of your CI build, so that referencing a parameter that doesn’t exist in javadoc causes your build to fail.
2. Does your documentation have examples or sample code? Execute them as part of your build, and (automatically) check that each one still works, and still emits the expected results.
3. Automatically generate as much of your documentation as possible. For example, generate REST API documentation from source code annotations.
4. Use libraries and patterns (Such as our own opslogger library) that make it easier to automatically generate documentation.

This tactic can help with heavy handed technical change management requirements as well. Need a detailed change request with every release to keep corporate operations happy? Automatically generate it from your git logs and issue management system. I was once on a team where we built a system that automatically compiled the list of JIRA issues that had been fixed since the last deployment. It was really simple (just a text file with JIRA issues and CI build numbers checked into git, really), but it made our operations team happy, and once we had it running it took us almost no effort at all to maintain.

In general, quality assurance for documentation (and make no mistake, that’s what documentation maintenance is) is just like all other forms of agile quality assurance: automate as much of it as you possibly can. You’ll find that the upfront investment in automation pays for itself, and then some.

¹ pun intended.