It’ll never fly

There’s a real challenge for established companies where technology is concerned, and it’s all about getting off the ground. It would appear that projects are like aeroplanes; the larger they are the longer the runway they require, and once they get too big they don’t get off the ground at all. If you’ve been in technology for any time at all, I’m sure you’ve come across at least one project that seems to run and run without end, never quite delivering anything of real value.

Many technologists who work in large organisations bemoan the inevitability of failure at scale. They gaze longingly at the challengers who deliver at blistering speed in trendy start-ups. How easy it would be if they could work in a green-field environment with no constraints and none of the rigorous governance required for large systems. But apparently, that’s a luxury you can’t hope to have in any “serious” company. Even start-ups get slow as they grow, don’t they?

In many cases they do, but not always. Of the ones that actually survive (and of course this is one of the biggest risks of working in such a world), many start-ups turn into average companies and then into lumbering corporations. Many of the same people work there, but they’ve lost the spark and become the very thing they used to mock. Many even buy a suit, and start using Powerpoint!

Staying nimble

However, in some rare cases the agility remains, and new stuff gets done quickly and often. This leads us to ask the question, how can this be possible, and is it something that can be introduced into a company that has long since lost that loving feeling?

The attempts that have been made to achieve this are many and varied and most result in failure. Innovation teams sprout up, only to wither on the vine, starved of funding and lacking any true stakeholder buy-in. Companies strip away governance in the hope that things will progress faster, and then discover the mess they get into when large projects go off down blind alleys.

Some companies have incorporated Agile into their ways of working, but many are still experimenting with it. Sceptical leaders introduce it to an unconvinced workforce, but even when they manage to create evangelists for this iterative development approach, they still end up with something that an old colleague of mine referred to as “water-scrum-fall”. Things go well at first as they try it out on small internal projects, but then as they scale up it all goes wrong. Once more the size, coupled with the weight of existing processes, overwhelms them and the speed disappears. Agile methods get consumed by Waterfall governance. There are even frameworks that encourage this counterproductive approach.

But Agile seems so promising. It reinvents something the older amongst us instinctively knew when we started out in software development; you always start with the user’s need (which is very different to the user’s want). Requirements capture, huge documents, and wall sized Gantt charts felt like something from another world; one populated not by engineers but by accountants and lawyers. Agile felt right because it was creative and dynamic, and that’s what we were doing; creating new things that had never been seen before, using technologies no one really understood yet.

The size problem is still the killer though. With a small number of people, the agile creative approach works beautifully. As the number of people grows, the stand-ups become unwieldy and communication breaks down. The expanding project gets broken down into manageable units, each delivered by a single agile team. Now you find yourself needing to do more up-front design work to ensure that each team knows the boundaries of what it’s delivering.

This is the thinking that leads to “water-scrum-fall”, an uncomfortable union in which agile teams deliver component parts of a much larger solution that in turn has emerged from waterfall style thinking. For some this seems ideal as it keeps all of the best practice of the tried and tested methods, whilst still allowing the development teams to get busy (and wear jeans). Unfortunately it misses the point as those best practices are the problem. They’re definitely best practices, but not for this stage of the development life cycle.

Test and learn

In Agile you do discovery. At first glance it might look a lot like requirements capture, but it’s a fundamentally different activity. You’re not asking the user to describe what they want in concrete terms and quantitative words. You’re observing them and working with them to generate ideas and find new approaches. You try things out and fail fast in order to succeed faster. 

Agile product development teams often refer to Alpha and Beta phases, and these are fundamentally different concepts as well. You don’t design then build then test; you design, build and test as a single activity but in an iterative manner, learning as you go. If you embed this approach inside Waterfall then you’re not doing agile at all. How often have you heard people who’ve adopted agile refer to the introduction of a “fire break” between discovery and alpha, and between alpha and beta to allow them to get the paperwork together and apply the governance processes? This is a sure sign that what is taking place is Waterfall.

In agile, governance is not abandoned. It’s just not done as a separate activity. Governance is done in-flight as an integral part of the iteration process and it’s continuous. At any time during an iteration you can change direction, resolve an issue, or stop completely.

This is the point where I mention the wake-up call regarding the waterfall method. It doesn’t work for new things! Sorry to be the bringer of bad news, but just because something seems to work for many situations doesn’t mean it will work everywhere, and waterfall doesn’t work where innovation is involved. When planning your production schedule, or managing a team of people performing a well-defined and repeatable activity the tools of the Waterfall trade work very well, but therein lies the problem. It is a very rare event in IT when we do the same thing for a second time. 

Each time we build a system we are building it for the first time, and we have no information on which to base a meaningful plan. All the analysis and research in the world will not give you the answers you need; you will only find out anything real once you start building. Waterfall does not allow this, and this is why agile wins every time. There is also the reality problem. The disciplines of waterfall are fine in theory for one change, but as you scale up the number of new things you’re trying to do, the time and resource required scales up linearly; there are no economies of scale. Unless you have infinite resources you aren’t going to get the complete works of Shakespeare and even if you did, no-one has time to read it.

The good news, for lovers of waterfall, is that it isn’t being replaced by Agile or Lean. Agile and Waterfall should be able to live comfortably together in an organisation at opposite ends of the product life cycle, with Lean sandwiched as mediator in between.

Pace, personality and persistence

So what is needed to break out of the big-company thinking patterns that lead to programme bloat? How do we avoid long running programmes that promise the world but never seem to come to fruition? At the start of this post I talked about the delivery mindset that allowed start-ups to work at a pace unmatched in larger organisations, and it is from this model that I draw my solution. 

There is more to the start-up world than just the advantage of small size. There are also the external pressures that are brought to bear. Start-ups are dependent on funding for survival in a way that far exceeds any equivalent pressures in more established companies. They are acutely aware of their own vulnerability and that in turn increases the volume of the ticking of the clock to deafening levels. Every day counts and quite often, every hour counts.

This creates appetite and pace.

There is also the sense of purpose that exists in a start-up. There is of course a sense of purpose in more established organisations but it is never as personal and certainly never as all consuming. In an organisation a bad year for the company is a disappointing year for the employee. In a start-up, a bad month for the company can be disastrous for the individual. A large organisation becomes like a machine whilst a start-up is a living entity with moods and emotions.

 This creates engagement and personality.

Finally, there is something bigger to aim for. Everyone in a start-up is aware of the potential exit or growth strategies. The commitment in time and energy is worth it because firstly at some point it will come to an end and something new will begin, and secondly everyone will benefit from the outcome.

This creates motivation and persistence.

Duplicating these environmental factors might just make large companies deliver benefits at speed, and in the second part of this post I will describe how to foster this start-up mindset and give your organisation pace, personality and persistence.