Data Fri 28th June, 2024
Making data mesh real: Transforming existing teams to be domain driven
The benefits of being domain driven
On a recent project we moved to a Domain Driven organisation from an existing team & capability based structure.
We saw benefits immediately:
- Consensus and transparency across the organisation on existing domains, product ownership and teams
- Teams began to understand domains and how they directly tied to their day-to-day activities
- Teams taking ownership of their domains and immediately pushing for change such as boundaries and product ownership
- Formed a baseline organisational map that enabled discussions between business stakeholders, product owners, architects and engineers about:
- Product ownership
- Scaling teams up or down
- Executing large programmes and initiatives without large scale re-organisation
- Reducing cognitive load for busy teams
The following is our experience on a long-running data platform project with well-established teams, but no explicit domain-driven design.
We developed a bottom-up process to extract a domain baseline, which the organisation then iterated on gradually to move towards an ideal domain structure. This was a long, hard fought process – a year for 9 teams – but the end result was an invaluable understanding that was previously missing.
We think this will be useful for anyone wanting to take a domain-driven approach to their data.
Understanding what a domain is
Domains can be initially mystifying – we found people in our teams didn’t initially understand them. What does this abstract thing have to do with my day-to-day work? Isn’t a team a domain?
We ran into this quickly by attempting to explain and define domains right out of the gate. This resulted in frustration in not understanding what we were trying to surface, and why it was different to existing team concepts and language.
By accident, we ran a session that showed a way. In this session, we took known business concepts of an existing team and asked everyone to find ways to group them to split them into sub-teams – a typical scaling problem.
These groupings were really domains, and everyone could create bounded contexts and explain their choices once they had a concrete set of items to work with. They could also name these groupings based on their business knowledge and share their understanding.
We realised everyone knew implicitly how to create domains, they just needed something to get them going.
Extracting a domain baseline
So, here is how we did it. Every team held a workshop, with team members and business stakeholders participating, with a mediator to run it.
Note: For these examples, we are using a fictitious Fintech bank called Bozo who have a department dedicated to fraud prevention & customer risking.
Workshop Steps
Step 1: Common language
First, every team member fills in prompting sentences to describe the team’s current working context. This quickly generates terms that will be used for the grouping exercise. The different coloured post-its are used in the next step.
Example prompt questions
Step 2: Define team context
We then take the answer post-its and group them into domain properties by colour. This forms our first view – a super grouping of everything the team currently does – and our first attempt at a domain definition. It’s also the first chance for everyone involved to observe how others think about their team context, and to begin building consensus.
To do this, we needed to define what a domain is and its properties. Each organisation should find this definition for themselves. There’s no right answer, and we arrived at this one after several iterations.
We took the domain definition from the Data Mesh book and extended it with User Value (we felt it was missing) and Boundary (we had a lot of boundary disagreements between teams).
In our organisation a domain uniquely owns the following:
- Services (Purple post-it): Digital services & data products that influence how business actions are carried out.
- Outcomes (Blue post-it): Targets and criteria which measure a domain’s success and failure. Can be measured e.g KPIs, metrics=
- Activities (Yellow post-it): The tasks this domain requires its owners to carry out that aren’t digital services. e.g. user support, raising product awareness, legal compliance documents.
- Knowledge (Orange post-it): The detailed information and rules this domain specialises in.
- User Value (Green post-it): Which users derive value from this domain, and what is that value?
- Boundary (Pink post-it) : Important things to call out as NOT being part of this domain
The questions in Step 1 surface these different domain properties without the participant needing to understand the full domain definition. Each Post-it colour corresponds to a property.
A grouping exercise now takes place for each domain property. The example below shows grouping together all answers for the Services domain property.
This is repeated for each property and colour e.g. Outcomes, Activities etc
This was always the longest part of the exercise – as the property post-its were being grouped, participants would discuss and agree on the group boundaries, and on a single name for each group.
They were forming a common language for describing their domain – and for many it was the first time they’d had to be specific. This agreement was essential to ensuring concise communication throughout the rest of the exercise.
Step 3: Split team context into domains
A grouping using the agreed names from the previous step takes place. This shifts thinking up from the individual element level (products, activities etc) to common verticals (domains). Every participant attempts their own grouping – this produces several experiments with different boundaries.
Whenever we did this step, common approaches to grouping would surface. A final agreed grouping – the first domain draft – is driven by the mediator through review of all experiments. Domain names are suggested, iterated on and agreed upon.
An example grouping using the Services property into three domains. The pink post-its are generated from the group name headings in the previous domain property exercise.
Note: This exercise will include group names from all properties (Activities, Knowledge etc) but has been kept simple for this example.
Final output
Congrats – we have a first draft of our domains!
Example domain baseline across Bozo Bank fraud prevention platform. Colours indicate team ownership of domains. Squares are the identified services from Step 2 – products – owned by the surrounding domain.
The baseline draft of domains showed team & product ownership and highlighted some key issues that teams had already been raising. Some of these domains have issues or are antipatterns, but as a baseline now exists, the organisation can begin to iterate and improve them.
By going through the exercise, teams began to understand domains and could see how the output directly tied into their day-to-day. They immediately wanted to make changes to their boundaries and ownership to reduce cognitive load and duplication. It also meant we had a map that could be used to talk through organisational changes – and a realistic baseline of where the organisation currently stands.
Most importantly the team, business and stakeholders had been on the journey together, built consensus and a common language, and were now invested in the next steps.
Many thanks to Simon Case, Paul Brabban, Nathan Carney and Dennis Robinson for their contributions to this blog.