“As long as we persevere and endure, we can get anything we want” – Mike Tyson
Human beings are terrible at seeing the bigger picture, strategizing, taking responsibility for it and overcoming huge amounts of opposition. Add to that a management with an innate proposition towards short-termism, risk aversion and lack of trust and you have yourself a rather explosive cocktail that results in chaotic organisations.
Chaotic organisations will have chaotic data…
The quality of the data in organisations is terrible. Lets not beat around the bush, it is. Michael Stonebreaker, winner of the A.M.Turing Award in 2015, confirmed it. According to Stonebreaker the average organisation owns 5000 silos of data, bigger organisations estimated to have 10.000. It boggles the mind why organisations keep on persisting in cleaning this data downstream. Stonebreaker – founder of Tamr – wonders why you don’t clean your data before it enters your downstream systems. If you don’t, he continues: “systems like Tamr will consume all your profits”.
I love this guy. Blatantly honest. But the fact is that the visualisation and “data muddling” technology is thriving. And I get that…
Cleaning up the garbage is ‘easier’ when the actual garbage can be seen. You can hire cleaners, buy machines/software to do the cleaning, etc.. It falls perfectly in line with a managements innate proposition towards short-termism, risk aversion and lack of trust.
Prevention of (data-) garbage requires a long-term view, vision, strategy, trust (!), perseverance. It requires an informational perspective on your organisation. A perspective where data (and process) – the raw ingredients – are at the center of the information strategy. Applications, functions and technology are derived from it, not vice versa. Organisations keep on buying or building applications/technology where the data is a by-product and subsequently keep investing in downstream tools and people to clean the garbage. Crazy.
It is what I call ‘high heels architecture’ – It looks sexy, but walking hurts like hell.
We need to look at data with an holistic perspective – across the divisions and often even across organisations. We need to conceptualize the information, logically model the data and choose the appropriate technology.
Very old school? Hell yes!! We need to educate our young people again in the craft of information, the ability to think in (logical) abstractions and separations of concerns. It is a craft! Nowadays, our education systems are hugely biased towards programming languages, applications and technology, etc..
Finally, the biggest problem in my view, is how we run organisations. How we educate our managers. Appreciate bad news from your employees, trust the craftsmanship, take responsibility for the path chosen, trust in teams to organize themselves, radical transparency (!), etc..
Back on the subject. Many organisations seem to have given up on data. In a future where datafication will only increase, huge amounts of their profits will indeed be spend on getting the data right. Risks in terms of operational failures but also privacy breaches will increase ever-more. Ultimately, the chances of delighting your customers, patients, tax-payers or students will diminish drastically.
Lets not give up on data….