Four myths of digital transformation. And how event-driven architecture can help

Picture the scenario. You’re a large enterprise—let’s say a bank or a national retail chain—working in a traditional waterfall environment with a monolithic architecture. And you know you need to change.

Why? Things are becoming untenable. You’re getting left behind.

You’re frequently delayed by large-scale, heavy releases. You’re hampered by slow feedback cycles, with teams tied up in manual reviews and sluggish approvals processes.

You’re reactive in the market, unable to rapidly develop new features, functions, or initiatives to support the evolving demands of your customers. In short, you can’t deliver quickly enough. For internal or external stakeholders.

If that’s where you are, it’s obviously not where you want to be.

You want small batches delivered more iteratively. You want a higher frequency of deployments, faster cycle times, and continuous delivery. This way, you can delight customers by delivering more value in the moment or new features far more rapidly than your competitors.

You want real-time feedback and metrics driven by automated workflows. You want to react and respond to customer needs in real time, across complex systems like payment gateways.

Unfortunately, there’s a chasm between what you have and what you want. The bridge across that chasm is the process of digital transformation.

While many organisations identify the need for a digital transformation project, it’s often seen as a daunting proposition. Too complicated; too lengthy; too much work up front; too many unknowns; too much disruption for already-stretched development teams.

In reality? These concerns are unfounded. Let’s look at four myths of digital transformation.

(For the purpose of this piece, we refer to digital transformation in the context of shifting from a monolith, supported by waterfall methodology, to an event-driven architecture supported by an agile team. Of course, there are other strategies you can adopt for digital transformation depending on your organisation’s unique requirements, but many of these same concepts and benefits still apply.)

Myth 1: “Everything must be planned out and perfect, well ahead of time.”

When you embark on a full digital transformation, it’s tempting to scope everything out prior to delivery: architecture, technologies, team structures, development approach and more.

That’s a natural impulse, especially for teams and organisations that already work in a traditional model. You’re culturally predisposed to consider solutions in a holistic end-to-end approach, in exhausting detail. You’re used to the concept of: establish all requirements > build everything > test everything > deploy it all.

If you want faster, more nimble and responsive ways of working—and delivering value—it’s counterproductive to begin with a time-consuming and over-scrupulous architecture design process.

Rather than plan everything in excruciating detail (which is unnecessarily complicated, time-consuming, and often liable to change over time anyway), you can determine the best approach iteratively. You can use emergent design practice to establish your architecture.

With an event-driven architecture, you can evolve your overall architecture as you go. In fact, you should. Whether you choose to begin with a singular business process like an event-driven payment gateway, or a unique user journey like processing an online payment, there’s huge value in building your architecture step by step.

Think big by starting small. By gradually atomizing organisational complexity into smaller, more manageable pieces—using events and microservices—you’ll enjoy greater flexibility and control in the future.

Speaking of flexibility…

Myth 2: “There’s only one path to success.”

While you gradually establish your architecture through emergent design practice, you also have multiple options in how you manage the transition from ‘old world’ to your new environment.

If you intend to embrace an event-driven architecture, supported by an agile delivery approach, you have some options:

• You can strangle the existing architecture. In other words, leave everything in its current state and transition piece-by-piece to a new setup. In this scenario, both environments can run concurrently. As a result, you can train part of your team in new processes to support the new architecture, while maintaining a significant proportion of your current output.
• Alternatively, you can build a fully functional architecture in its entirety, and ‘flip the switch’ to the new environment when your teams and key stakeholders feel confident and supported enough to do so.

Either approach is entirely valid, depending on your preference or requirements.

Myth 3: “Processes and technologies will be reimagined simultaneously, creating a state of upheaval.”

It’s true. Processes and technologies will be reimagined simultaneously. For most companies moving from a traditional waterfall approach to event-driven patterns and agile delivery practice, you’ll have to change:

• The way you deliver, and
• The technical solutions under the hood that support these new delivery practices

If you’re adopting agile, the whole point is to deliver more value, more rapidly, and more frequently; a monolithic architecture won’t have the flexibility to mirror your team’s development cadence.

While holistic change can be necessary, it doesn’t need to happen in one fell swoop.

Like agile, event-driven architecture is about reimagining the complexity of sophisticated business domains as more manageable “increments”. Fittingly, your transition should be equally incremental.

For example, you don’t move to agile by suddenly changing the entire release process. Instead, you’d likely start off with something small and self-contained—like the process of provisioning changes in a test environment—until the team is comfortable and confident enough to move forward. Alternatively, you might start off with a focus on continuous integration. Then you can gradually move to a more continuous delivery model, as you emerge a more flexible event-driven architecture, while your team becomes familiar with event-driven patterns.

You build out your technology iteratively. You enhance your team’s development practice iteratively.

Myth 4: “We won’t derive any value until the full digital transformation is complete.”

Think of your architecture like a cake. Why? If you approach your digital transformation appropriately, you can ‘have your cake and eat it too’.

In other words, you can partly transition to a functional new, event-driven environment—with new, agile ways of working—while the majority of your organisation continues to operate with familiar technologies and delivery methods. And you can establish valuable organisational capability and infrastructure as you go.

Imagine your organisation is a sponge cake. Let’s say each layer represents a siloed business domain:

• icing is the front-end
• sponge is a gateway
• cream is a set of APIs
• jam is a core system
• sponge is a database.

A traditional approach would suggest building the cake one layer at a time. That means someone will design the front-end. Then, connect it to the existing system. Then they’ll build a set of APIs. Eventually, those APIs will connect to the core system.

If you want the most enjoyment (read: value), would you ever cut a cake horizontally—layer by layer—in the same way you ‘build’ it? Probably not.

A vertical slice of cake, with every ingredient, is far more appealing than a horizontal slather of pure jam. Horizontal slicing, or building out systems based on siloed infrastructure, is a bad anti-pattern.

An example of a vertical slice might be ‘e-commerce’. Say you focus exclusively on product listing pages. For those pages, your team will build the front end, the APIs, the core reference data; everything required to make the e-commerce component of your product event-driven and fully functional.

You don’t have to think about the entirety of the architecture or a range of systems or platforms. You don’t have to overcomplicate the flows.

You’ll get value by immediately improving the end-to-end process of your e-commerce journey. At the same time, you establish replicable infrastructure and capability, which you can then easily apply to other domains and user journeys as the digital transformation rolls out across your organisation.

So, there you have it. Four common myths that are easily overcome.

Digital transformation may seem like a daunting prospect. But with more leading corporations investing in event-driven technology, and nimble start-ups able to pivot on the fly to secure ever-more market share, the far more frightening reality is persisting with technology and delivery methods that leave you uncompetitive.