At 101 Ways we appreciate that business leaders are still learning about data mesh and the benefits it can provide. That’s why we decided to put data mesh on trial to determine if it’s just the latest buzzword or a new long-term approach to business data architecture.
As Chief-of-Staff at 101 Ways, I was given the honour of presiding over the trial (in full costume, of course). Our jury convened last Tuesday at the Nationwide Building Society Offices to hear from four expert witnesses:
- Simone Steel, Chief Data & Analytics Office at Nationwide, who has 30 years of experience in data architecture and is an active speaker on data and analytics strategy and sustainability at global events.
- Grant Smith, Technical Director at 101 Ways. Grant has created and led high-performance engineering teams in some of the largest UK companies.
- Finbarr Murphy, Data Product Owner at Modular Data, a start-up founder and specialist in decentralised data, data products and electronic exchanges.
- Dominic Messenger, CTO for Modular Data, a technologist specialising in automated marketplaces and originator of the UK’s automated mortgage underwriting benchmark.
Read on to find out how the trial of data mesh unfolded…
Data mesh, data warehouses, and data lakes
Before we dive in, let me briefly explain why this trial was necessary.
Having emerged as a data management platform that allows end-users to access data without the support of expert teams, there has been much debate around the viability of data mesh. Whilst many see it as a long term solution to data management, some are sceptical of its adoption, preferring established solutions like data lakes and warehouses.
Without further ado, let’s get started with the testimony of our expert witnesses.
Further reading: Learn more about the pros and cons of these strategies on our blog — Data Strategy Head to Head: Which solutions are best for your business?
Why data mesh should be adopted
The case for data mesh started strongly as all four experts were in favour of adoption — mostly down to how data mesh benefits businesses.
The first of our expert witnesses to be sworn in was Finbarr — who was keen to discuss how data mesh makes it possible to “look at data from a product perspective.” For him, bringing “product thinking together with data” and “putting business value first” are key factors not addressed by data lakes or warehouses. This means data isn’t considered a “business valuable object” that organisations can track return on investment.
Finbarr also stressed the importance of data warehouses, noting examples of good implementations. However, he also talked about how these tend to fall down when you “start looking at insights” and try “to make sense at a wider business level.” Essentially, people need data built for them when they need it, something data mesh can provide by serving as a “socio-technical paradigm,” not just a tech implementation.
Next, I swore in Simone, who built on Finbarr’s arguments. She was quick to point out that a data warehouse approach tries to control the uncontrollable, taking too long to respond to urgent business questions. In her view:
- Things need to change with data acceleration not aligning with business needs
- There is still a lag in the way we consume data
- Warehouses can often act as an anchor which ultimately drags a business down
Simone sees data mesh as a way of solving this problem because it “accepts this is real life” and that businesses “need answers this second.” For her, data mesh is serving as “a solution trying to address a paradigm shift” — providing new techniques to manage the exponential growth of data and help real people make crucial decisions.
Next, after promising to tell the truth, the whole truth and nothing but the truth, Dominic talked about the importance of decomposing “large business problems into small, easily digestible and manageable problems.” This is an approach that has already been successful, but not one that has been implemented within the context of data strategy.
That’s part of the reason why data is still seen as quite monolithic when it shouldn’t be. We should instead look to align data with businesses more than with tech, having “less concern with data movement” and focusing more on “data value”.
For Dominic, data mesh can help achieve this by introducing more outside-of-the-box thinking, something he explained with a useful sea shipping container analogy. What this amounts to, is the fact that warehouses are equipped to solve highly curated, in-the-box problems, whereas anything that requires next-level modelling needs data mesh.
Our final expert witness was Grant, who, building on the points made by the other witnesses, put forward several compelling arguments in favour of the adoption of data mesh, including:
- Allowing teams with purpose skills sets to quickly extract insights from data
- Helping businesses get started with data management quickly
- Enabling focus on small yet significant sections of data
- Spreading data literacy throughout an organisation
Grant’s testimony also highlighted the deficiencies of data warehouses. Whilst warehouses are still crucial for validating insights gathered in the data mesh, that doesn’t make them suitable for quick insights. Ultimately a combination of both is required, with data mesh providing quick, product-led insights while warehouses validate insights at board level.
To wrap up his testimony, Grant posed a question to anyone who would argue against data mesh — “What is so special about data that it should buck the trend of everything we’ve been doing since 1940?” That being “the constant journey to distribute computing.” Something worth considering.
The challenges associated with data mesh
While each of our expert witnesses had some extremely complimentary things to say about data mesh, there was some discussion around the potential challenges that could come up from adopting it.
For example, giving teams who aren’t data literate access to an organisation’s critical data can create problems. Grant was quick to point out that these teams will give an answer based on said data, but it will be hard to determine if it is the right answer.
Essentially, data mesh has the potential to create nervousness due to a perceived lack of governance. So businesses will need to be proactive in addressing those anxieties in their teams to avoid both chaos and untraceable decision-making.
Other concerns our experts acknowledged that can arise due to the adoption of data mesh, include:
- Difficulty implementing the changes that data mesh represents across an organisation
- User roles not being equipped for this new solution to data management
- The effective communication between domains (that data mesh relies on) can be hard to build in time for its adoption
Find the right solution for your business
After all the evidence had been given, the jury was given some time to interrogate our expert witnesses in small groups. Following much deliberation, a verdict was returned, and the result of putting data mesh on trial was… a hung jury!
Grant summed things up when he concluded there is “nothing” wrong with data warehouses. A warehouse gives you a place to “do experiments” and “validate insights”, providing an overall view of a business. However, a warehouse is inefficient when it comes to product insight, which is why it’s crucial to consider implementing data mesh alongside a data warehouse — rather than replacing it.
As a result, the key takeaways from the trial of data mesh are clear:
- There needs to be a bespoke approach to data management across organisations to ensure successful outcomes
- Businesses must always consider the impact any adoption will have on their teams
Working with outside partners ensures your data management solution can actually support your organisation’s goals and objectives. With our industry-leading expertise, at 101 Ways we help customers overcome challenging issues and take the uncertainty out of technological change.
There are 101 Ways to go about devising and developing a data management solution that works for you — get in touch to start your journey today.