Want to know more?
Are you interested in this project? Or do you have one just like it? Get in touch. We’d love to tell you more about it.
From small beginnings to big visions with data
HMRC CIP
How Equal Experts helped HMRC turn vast, unstructured data into a powerful, scalable platform that drives fraud prevention, customer insights, and smarter decision-making for UK government services.
One of the responsibilities of HMRC, the UK tax authority, is to audit taxpayers’ visits to its websites, specifically for the purposes of fraud prevention. As a key digital transformation partner since 2017 on HMRC’s Multi-channel Digital Tax Platform (MDTP), Equal Experts was involved in building a single tool to store this data, streamed from all the different services taxpayers use. As we iterated on the tool – and supported by our customer’s vision – what began life as a basic transaction monitoring tool eventually became a fully-fledged Customer Insights Platform (CIP) with the capacity to help guide smart business decisions thanks to the accessibility of the rich data stores handled by the platform.
This is the story of how facilitating a basic legal requirement led to the creation of a valuable, flexible and scaled data platform filled with customer behavioural insights that could support diverse government business use cases with a level of efficiency that has never before been possible.
Outcomes
30% YOY
capacity for growth in data stores
Best Public Sector IT Project
UKIT Awards 2021
Millions
of Covid-19 related claims from taxpayers automatically assessed preventing fraud and enabling fast payout on genuine claims
About the client
HMRC is a non-ministerial department of government. It is the UK’s tax, payments and customs authority, and has a vital purpose: gathering the approximately £740b tax revenue that pays for the UK’s public services (hospitals, schools, etc.), and helping families and individuals with targeted financial support.
Industry
Government
Organisation size
66k+
Location
UK
Length of project
7 years
Challenge
Turning vast data stores into meaningful pictures
As compliance data started to accumulate, HMRC realised how valuable it was to have a better understanding of what customers were doing on its websites – what they needed, how they used the services, and whether they were getting what they came for. Looking at the data in this light, the team knew that there were almost endless use cases it could serve, from simply optimising a digital service for the customer, to using it to improve HMRC’s call centres as well as to help prevent fraud.
There was one key problem. Despite datasets continuing to grow (illustrating billions of interactions every month) the data was costly to accommodate and wasn’t easy to share. In other words, there was a disconnect between what HMRC understood about its customers, and its ability to leverage that information. The task was to figure out how to coordinate and connect all this data cost-effectively, and in a way that would enable useful searches and analysis to inform decisions about the business.
Solution
Platform automation that joins the dots for a clearer picture
In the beginning, Equal Experts was required to help create a tool for effective transaction monitoring. However as more and more data was stored, and as world events – including Covid-19 – changed the environment for HMRC, we knew that the tool could do more.
CIP is a production platform that collects, aggregates and helps evaluate all transaction data, including the metadata used to submit those transactions. We created a repeatable and extensible machine learning framework with easy access to very large datasets (9B+ events) and provided machine learning tooling for HMRC’s data scientists.
CIP uses shared libraries to tag customer interactions with identifying metadata across thousands of HMRC websites and services, e.g. completing an online tax return. It captures device ID, email addresses and other data which – as well as identifying potential fraud – helps HMRC keep its customers’ information safe. Because a common set of library codes is used across the whole of HMRC, we are able to identify entire journeys taken by the customer to have a better understanding of what constitutes regular behaviour, and to spot anomalies more easily.
A modern data platform
But capturing the data in a solution that integrated data storage and tooling wasn’t enough when data volumes were increasing by 30% year on year. To transform, we started streaming data to cloud storage so we could take advantage of the flexibility, scaling and interoperability that cloud services offer. As we got more and more requests for meaningful visualisations of that data, we realised that the existing data analytics tooling couldn’t handle the data volume either and that one tool wasn’t sufficient for all the different business users of that data. We knew there was more that could be done to accelerate outcomes.
In considering how to modernise the tooling we were using, we started to look at data mesh and the option to decompose the monolithic architecture in favour of a modern data platform that would eventually become CIP. Using this domain driven design approach, we built a platform which supports data products and enables scale.
A modern application platform
We also realised we needed to modernise the team’s application platform to be able to leverage the operational efficiency and security of modern cloud services. The vision was to create a cloud-first application platform so that HMRC could automate the process of deploying, monitoring and storing data at the lowest cost possible, and to distribute risk upstream to the cloud provider so that the platform team could focus on the value of democratising the data, instead of a lot of manual engineering management. This model of security design and monitoring garnered interest across other HMRC teams as examples of cloud best practices.
Overall, modernising both the data and application platforms underpinning CIP allowed us to accelerate our use of machine learning for both in-journey and post-submission fraud risking. CIP allows HMRC to validate ownership of bank accounts and devices to protect customers and government services, joining the dots for a more transparent, evidence-based picture of what’s happening in the UK’s digital tax services.
Results
A secure place to understand what UK taxpayers need
As we ended our work on CIP, we left behind a set of core data products, a data product pipeline and comprehensive data tooling supporting multiple use cases, all of which is fully decoupled from legacy tooling. Examples include:
- A bespoke case-working tool we created for use by fraud teams
- Tools that enable users to complete advanced searches on vast data stores that would otherwise be impossible to search through meaningfully
- APIs providing upfront fraud prevention for different customer journeys.
CIP has proved its value countless times, but never more so than during Covid-19, when – before the country was even aware of the full extent of the crisis about to happen – the team was able to proactively pull together proposals in anticipation of being asked to prevent the widespread fraud of Covid-response schemes. Millions of claims submitted by taxpayers were automatically assessed for fraud detection by the service, accelerating the speed and accuracy of the fraud protection process and of payouts to genuine claimants.
“What the CIP team achieved at the beginning of the Covid-19 pandemic was nothing short of remarkable. They responded to possible fraud threats to the COVID-19 schemes and rose to meet the needs of 2020 in a way that no UK government had ever responded before, by supporting workers on an unprecedented scale.” Caitlin Smith, Delivery Lead, Equal Experts.
The work was deemed so impressive it won Best Public Sector IT Project at the UKIT awards in 2021.
Our work on CIP has far-reaching potential too, with the team now able to make its data available for valuable services outside of HMRC, including third parties such as accountancy firms, who are now able to comply with the same fraud prevention measures through their own platforms. A cross-government insight pilot is already underway to provide a bank account checking facility which will enable other government departments to check the validity of UK bank account details submitted by customers. This has the potential to speed up processes and identify possible criminal activity far beyond HMRC.
Conclusion
In CIP, HMRC has a complete data platform housing more than a petabyte of rich data, which can be used to automatically detect criminal behaviour patterns and prevent common attacks to the UK tax system as well as to inform UK government business decisions. We believe data volumes will continue to grow, doubling in size every 2 years, and we know that CIP is more than able to handle this.
To find out more about how HMRC collects and uses your personal information for transaction monitoring purposes, check the HMRC transaction monitoring privacy notice.