Data Management & Decision Support

Wendbaar wetgeven; en nu de data.......

Komende vrijdag 22 november 2019 geef ik een presentatie voor de Expertisegroep Regelbeheer, dit is een kennisuitwisselingsverband tussen Belastingdienst, SVB, DUO, IND en Rijkswaterstaat op het gebied van bedrijfsregelbeheer. Onderstaand een abstract van de sessie die ik aldaar
ga geven. 

Voor een meer generieke achtergrond verwijs ik graag naar een vorige blog (engels)

Data is helaas, in de wereld van nu, vaak een bijproduct van een applicatie of een te ondersteunen proces. Belangen die de applicatie of het proces overstijgen worden onder het tapijt geveegd en 'opgelost' in kunstmatige en gebrekkige opwerkingen zoals datawarehouses, datalakes of andere (vaak informele) kopieermechanismen. Deze 'dataschuld' zorgt voor enorme uitdagingen t.a.v. wendbaarheid van wetsveranderingen op het digitale landschap, actualiteit en kwaliteit in dienstverlening naar burger en bedrijf en risico's t.a.v. ongecoördineerde verspreiding van data. We moeten data & logica scheiden van proces & applicatie. Dit wordt ook wel de scheiding tussen 'know' en 'flow' genoemd. Op het gebied van logica, en deze netjes afleiden uit de wet- regelgeving, heeft de Belastingdienst reeds ruime ervaring. Het is tijd dat nu ook het grote 'databeest'  zich hierop aansluit.

Daartoe heeft de Belastingdienst een Verticale Datadefinitie Architectuur  vastgesteld waarbij elke vorm van vastlegging is afgeleid uit wet- en regelgeving en los staat van proces- of
applicatiebelangen. Waarmee de impact van wetswijzigingen transparant wordt en - vice versa - de data altijd terug kan worden herleid tot wet- en regelgeving. Kennis-, informatie en datamodelleren is het vakmanschap dat (weer) nieuw leven in moet worden geblazen. Denk aan cognitatie, feitmodelleren, natuurlijke taal, regelspraak, logisch modelleren, anchor-style modellering, modeltransformaties en ordentelijke  'separation of concerns'. En nee, dit is allesbehalve een IV of IT taak, dit is per definitie een innige samenwerking tussen wetschrijver, wetsanalist. modelleur en engineer. Dat deze datadefinitie architectuur een paradigmaverschuiving is in de wijze waarop informatiesystemen tot stand komen, begint langzaam door te dringen. De grote uitdaging zit hem dan ook in de transitie, de fundamenteel andere manier van denken, werken en opleiden en hoe e.e.a. realistisch kan kantelen in de uitvoering.  

Data en regels horen bij elkaar, ze zijn als het ware 'brothers in arms'. Gezamenlijk definiëren zij een kennisdomein die gescheiden moet zijn van uitvoering.  De nieuwe Referentiearchitectuur Gegevenshuishouding van de Belastingdienst is gebaseerd op drie ontwikkelingen; (1) Wendbaar wetgeven, (2) Common Ground en (3) Regie op Gegevens. Een van de belangrijkste strategische thema's, de Verticale Datadefinitie Architectuur, wordt in deze sessie toegelicht.

The Yin & Yang of data: 'Science of data' & Datascience

People familiar with my thinking know that I am a bit of a 'fundamentalist' when it comes to 'data'. I am the guy that pushes the non-sexy part of data; data quality, data governance, metadata, data protection, data integration, semantics, rules, etc..

It is hard to stand your ground in a time where shortermism, technology-fetish and data-populism is thriving. I see ‘data architectures’ in my industry that boils down to superdooper databases, ultrafast massively parallel hardware and of course huge amounts of software that seem to glorify ‘coding’ your way to the promised kingdom.

Call me old school, but I want (data) architectures to separate concerns on various levels (conceptual, logical and technical), dimensions (process, data, interaction) and aspects (law & regulation, people, organisation, governance, culture). Architectures should enable businesses to reach certain goals that (preferably) serve the customer, civilian, patient, student, etc..

Lately I have been studying the ‘datascience’ community, attempting to comprehend how they think, act & serve the common goals of an organisation. I have abandoned (just a bit :-)) my declarative nature of data modelling, semantics, dataquality or governance and I have drowned myself in coursera courses, learning to code in Python, Julia and R. Dusting off my old Statistics books I learned in Uni, installed Anaconda, Jupyter notebook, Rstudio, Git, etc..

And oh my, it is soooo cool. Give me some data, I don’t care what, where it comes from and what it exactly means, but I can do something cool with it. Promise!

Now my problem…

• (1) It seems to me that the ‘science’ in ‘datascience’ is on average extremely low to non-existent. Example; I have heard of ‘data science’ labs/environments where the code is not versioned at all & data is not temporal freezed, ergo; reproducibility is next to zero. Discovering any relationship between variables does not mean it is a proven fact, more is needed. Datasciene is not equal to data-analysis with R (or whatever), is it?  
• (2) There seems to be a huge trust in relevance and quality of data wherever it comes from, whatever its context and however it is tortured. Information sits at the fabric of our  universe¹, it’s life, it’s the real world. Data is the ‘retarded little brother’ of this ‘information’, it is an attempt of humankind to capture information in a very poor way. Huge amounts of contexts are lost in this capturing. Attempting to retrofit ‘information’ from this ‘retarded brother’ called ‘data’ is dangerous and should be done with great care. Having these conversations with data scientists is hard and we simply seem to completely disconnect.
• (3) There seems to be little business-focus, bottom-line focus. Datascientists love to ‘play’, they call it experiment or innovate. I call it ‘play’ (if you are lucky they call their environment ‘sandbox’, wtf?). Playing on company resources should be killed. Experiments (or innovations) start with a hypothesis, something you wanna proof or investigate. You can fail, you can succeed, but you serve the bottom-line (and yes, failing is serving the bottom-line!) and the purpose/mission of an organisation. Datascientists seem to think they are done when they’ve made some fascinating machine learning-, predictive- or whatever-model in their sandbox or other lab-kind-of-environment. Getting this model deployed on scale in a production environment for ‘everyone’ to use, affecting the real world…..that is where the bottom-line value really shines, you are not done until this is achieved.
• (4) There seems to be little regard for data protection aspects. The new GDPR (General Data Protection Regulation) is also highly relevant for datascience. Your ‘sandbox’ or your separated research environment needs to be compliant as well! The penalties for non-compliance are huge.

There is huge value in datascience, its potential is staggering and it is soo much fun. But please, stop fooling around. This is serious business with serious consequences and opportunities for everyone and for probably every domain you can think of, whether it be automotive, banking, healthcare, poverty, climate control, energy, education, etc…

The ‘science of data’ and ‘datascience’ are the yin & yang of fulfilling the promise of being truly datadriven. Both are needed.

For my Data Quadrant Model followers; it is the marriage  between quadrant I & II versus quadrant III & IV.

¹ Increasingly, there is more and more 'evidence' originating from theoretical physics that this statement hold some truth to it, link [Dutch]. I would also like to give attention to this blogpost by John O'Gorman, very insightful, putting DIKW to rest....

 

 

Make data management a live issue for discussion in your organization

Last month an interview with me was published in the SAS Future Bright Magazine. There quarterly theme was 'A Data Driven Reality' [Link to Dutch version] - right up my alley. This magazine was published in French, English and Dutch.

At the event that launched this quarterly magazine I had the honour to speak about the core of my interview; The Data Quadrant Model.

Thanks SAS Institute for having me, for publishing a great magazine and for giving data management the attention it so desperately needs. 

For those interested in the interview that attempts to explain the Data Quadrant Model in layman's terms - here are the downloads:

Download Dutch - Het Data Kwadranten Model, Interview Ronald Damhof

Download English - The Data Quadrant Model, interview Ronald Damhof

Download French - modèle de quadrant de données, Ronald Damhof

 

 

Data Quadrant Model: decreasing entropy

With regard to 'order' and the 2^nd law of thermodynamics I have discussed the four quadrants of my Data Quadrant Model:

1. The order of quadrant I
2. The chaos of quadrant III
3. The un-order of quadrant IV
4. The "order" of quadrant II

Entropy in the Data Quadrant Model is lowest in quadrant I, higher in II, even higher in IV and highest in III. If we do not actively decrease it, we tend to loose the value of (the) data(-platform) and we will find ourselves investing huge amounts of Euro's to do the same thing over and over again (like Groundhog Day) or spending huge amounts of Euro's to control an 'out-of-control-beast'. Unfortunately, I have seen and still see this a lot.

In an ultimate open system¹, entropy can not decrease. In the universe the entropy will only increase (and eventually we all die). However, the Data Quadrant Model is a closed system. And in a closed system we can decrease entropy. How?

There are roughly three important directions in which entropy is to be decreased actively: 

1. Decreasing entropy from III to I
2. Decreasing entropy from IV to II
3. Decreasing entropy from II to I
   (I describe these in the details of this post. Warning it is not for the faint-hearted)

Important message: like in physics, decreasing entropy costs energy. The higher the difference in entropy between two systems/quadrants, the higher the energy needed. And yes, you can replace 'energy' with 'costs'. 

We now enter the field of Data Management. A prime directive of Data Management is to reduce entropy in data and to keep the data-platform in a sustainable modus where it serves the data-driven and data-centric organisation. It is hard, not cheap and still mostly unknown territory, but if you can make it work, the rewards - in the era of datafication - are huge. 

------------------------------

Data Quadrant II: the battle against entropy

A word of warning; after writing this post I realise that truly understanding it, requires a more than basic understanding of the Data Quadrant Model. And of course; if you still don't understand where I am going with it, then it's save to say that I failed you, my apologies. :-)

A while ago I wrote a blog about the Data Quadrant Model I developed. I use this model in my consultancy and speaking engagements. Increasingly I receive great feedback from organisations that are applying it, which is great.

A week ago I wrote about the un-order of quadrant IV. A few days ago I posted the blog about the order of quadrant I as well as the chaos of quadrant III. It's time to describe the battle against the ever increasing entropy of quadrant II.

Imagine quadrant I as data where the molecules are set in a fixed position, neatly managed, ordered & governed. Now, in quadrant II the molecules are beginning to move. They can be rearranged in a vast amount of different positions, the number of positions can be huge, infinite even. Now replace these molecules with data and take into account that data can be copied and re-used over and over again. There is a vast array of possible positions for data.....

Back to the Data Quadrant Model...

Quadrant II reflects the contexts¹ in which data is used, like I said before - in quadrant II we have multiple versions of the truth.

Lets, for argument's sake, assume that the number of contexts/truths is potentially large. It is save to say that the number of ways in which data can (and probably must) be re-arranged, aggregated, calculated, inferred or otherwise manipulated to serve a vast array of contexts, will equally be (exponentially) large. 

What quadrant II aims to achieve is to control the entropy as long as possible against justifiable costs. It is known that entropy, by definition, will only increase. But there all kinds of tools, technologies and mitigation measures to extend the time before entropy reaches an un-manageable (or non-cost-effective) state. If the latter happens we can always return to the 'zero-state' - still present in quadrant I - and start again. 

That's the cool thing about data. Entropy in the universe is ever increasing and we can never reset (unless we break some fundamental rules of physics or with divine intervention), with data we can....

Summarise
Quadrant II is trying to maintain order (same as quadrant I). The entropy makes it harder though and subsequently more energy (=costs) is needed. Where the systems in quadrant I are characterised by its obvious state, the systems in quadrant II are considered to be complicated².

 

¹ It would be interesting to research what the independent variables for # of contexts could be. Size of the organisation, # of internal & external stakeholders, # differentiation of products and services, dynamics of the (world)market, complexity of business processes, managerial effectiveness, organisation hierarchy...?

² A system can be very complicated but not complex at all. A system is complex when it has emergent behaviour (quadrant IV)

The chaos of Data Quadrant III

A while ago I wrote a blog about the Data Quadrant Model I developed. I use this model in my consultancy and speaking engagements. Increasingly I receive great feedback from organisations that are applying it, which is great.

A week ago I wrote about the un-order of quadrant IV. A few days ago I posted the blog about the order of quadrant I. Now it's time to describe the most misunderstood quadrant, quadrant III. 

So far, quadrant III has received little mention, even though it is incredibly important. It is the quadrant of data sources which are not under governance, like an ad hoc download which you obtain from an open data provider, a list in Excel that you want to use, or a set of verification data which you have received on a CD. But you might also use it to dump huge amounts of data that you want to explore or experiment on. 

Quadrant III exists (implicitly) in every organisation. Only think of the huge amounts of Excel sheets or MS Access databases currently stored on your fileservers. 

Entropy in quadrant III is at its height. There is data chaos. As Stephen Brobst (chief technology officer of Teradata) so eloquently put it when he described a data lake - slightly paraphrased; 'the only people that can use this data are the people that  have put it in'.

And yes - the popular 'data lake' is an artefact that is situated in quadrant III. 

Governance, control, checks and balances in quadrant III are virtually non-existent. People using this data (the peeps in quadrant IV) need to discover the structure/schema, make inferences and be extremely careful in drawing conclusions. The field of statistics is important in this regard. 

Let me be absolutely clear; I am not against a data lake. I am just against the notion that a data lake is viable as a quadrant I artefact (and that is how the industry is trying to sell it). As a quadrant III artefact the notion of a data lake could be valuable. Is is a cost-effective artefact that has the potential to drive innovation.

Although I described quadrant III as data chaos which is unmanaged, ungoverned, etc... It is possible to manage and govern the infrastructure. Current technology is promising in offering a managed and governed infrastructure on top of which one can innovate with data in an unmanaged and ungoverned manner.

The cool thing of this technology is that quadrant III is being made explicit, from the shadows into the light. In quadrant III an organisation might want to offer its users an infrastructure-as-a-service. 

Oh, one more thing; if you consciously architect/configure a quadrant III you are engaged with innovation. If you do not, you are just messing about. ;-)

The big danger of quadrant III (and quadrant IV) is that promising discoveries are never productised (I unfortunately see that a lot!). They remain to be high potentials that never ever see the light of day (promotion to quadrant I and II). The high level of entropy enforces this risks evenmore, since discoveries are by definition often related to the brilliance of individuals. Governing mechanisms are need to counter this risk, otherwise you are just playing with data.....a waste I'd say.

I have now discussed three quadrants (I, III and IV) in terms of various levels of order. I have one quadrant to go. Before I move on, let me again stress the following:

1. None of the four quadrants is more desirable that any other; there are no implied value axes. All quadrants are needed in every organisation that wants to be more data-centric, data-driven, adaptable to changes, innovative, etc..
2. The Data Quadrant Model is NOT a blueprint implementation model. It can however be used as a guide in how to organise, how to manage, how to govern, what technology choices to make, what people to hire, etc.. 

 

 

The order of quadrant I

A while ago I wrote a blog about the Data Quadrant Model I developed. I use this model in my consultancy and speaking engagements. Increasingly I receive great feedback from organisations that are applying it, which is great.

A week ago I wrote about the un-order of quadrant IV, this post will be about it's (diagonal) mirror image, quadrant I. The quadrant of Facts. 

It is in all its aspects a mirror image of quadrant IV. It is the quadrant with the lowest entropy. There is order, governance, rules, standards and automation. Quadrant I is constantly searching for efficiency or optimisation gains. The 'systems' (I use the term loosely here) in this quadrant are characterised by (mostly linear) cause and effect relations that are repeatable, perceivable and predictable. Checks and balances are strict in terms of the adherence to standard operating procedures.

Quadrant I is the domain of methodology (seeks to identify cause-effect relationships through the study of properties which appear to be associated with qualities). Methodology can be translated to architecture as in (information) system development. Put it In other words; if you have no architecture for quadrant I.....

There is a tremendous effort necessary to keep the 'systems' in quadrant I low in terms of entropy, it needs to be controlled and it needs to be governed fiercely. Organisational-wise, quadrant I benefits hugely of organisational centralisation (as opposed to quadrant IV that benefits a lot from decentralisation).

What it all boils down to is that in quadrant I there is no fooling around with data. People working in quadrant I have a data-mindset combined with an engineering heart. Data Architecture, Data & Information modeling, Databases (using the terms loosely), Data processing (sets & transactions), etc.. They are infused with the will to improve the consistency, availability, reliability and traceability of data. If there is a science of data, these people are living it and doing it. If anyone wants to know the pedigree of a certain data element....

And no - they are not 'IT' - they are the glue that binds business with IT¹ and they are scarce. You either got to buy them or train them - there is no middle ground, especially since universities are dropping the ball here big time. They seem to be pre-occupied by offering 'Data Science' masters now, riding the waves of opportunity and short-termism.

I want 'Science of Data' masters.....

Ah well, at least there is job security for quadrant I peeps. ;-)

 ^{1 In terms of Business- and IT alignment I am loosely referring to the generic framework for Information Management - the middle column of the Information management model. Warning: Organisations are using this framework as a blueprint for organizational structure. I can not begin to explain how utterly wrong that is. It is a framework that assists in thinking and designing, nothing more.} 

The un-order of Data Quadrant IV

A while ago I wrote a blog about the Data Quadrant Model I developed. I use this model in my consultancy and speaking engagements. Increasingly I receive great feedback from organisations that are applying it, which is great. 

In this post I want to elaborate a bit more on quadrant IV (see figure).
This quadrant is often present implicitly in most organisations.Just think of all the Excel users in your organisation! Nowadays, quadrant IV is supercharged by all the stuff that is offered under the flag of 'Data Science'. And yes, I put quotes around it....the word 'science' bugs me a lot...

But that is another blogpost.

Quadrant IV is often misunderstood, especially by my fellow architects and management. They have an inert need to establish order in this quadrant. And there is no order....there is also no dis-order.

Huh?

Dave Snowden is actually describing it correctly; there is un-order: "Un-order is not the lack of order, but a different kind of order, one not often considered but just as legitimate in its own way". He uses a great story to exemplify un-order:

"....a group of West Point graduates were asked to manage the playtime of a kindergarten as a final year assignment. The cruel thing is that they were given time to prepare. They planned; they rationally identified objectives; they determined backup and response plans. They then tried to “order” children’s play based on rational design principles, and, in consequence, achieved chaos. They then observed what teachers do. Experienced teachers allow a degree of freedom at the start of the session, then intervene to stabilize desirable patterns and destabilize undesirable ones; and, when they are very clever, they seed the space so that the patterns they want are more likely to emerge. "

Order might eventually emerge.....

Now, lets take this notion of un-order a bit further into quadrant IV. If something is ordered, management (and architects alike) like to focus on efficiency (e.g. control, standards, predictable/repetitiveness, best practices etc..) a lot. In an un-ordered system (like quadrant IV) this behaviour is killing. Management and architect need to allow a lot of sub-optimal behaviour in order to reap the benefits of quadrant IV. 

Put it more popular (and scientifically wrong, but what the hell); In quadrant IV we need to 'architect' a bit of chaos. We need to un-manage.....un-control....experiment & learn...Trust

So, we gotta free the experts/knowledge workers (yep, not calling them "data-scientists") in quadrant IV.

Will order eventually emerge? Yes and no. Order and un-order will exists both in the same timeframe in quadrant IV. 

 

 

[Dutch] Full circle; ‘geef mij de data, de software en de machine’

De afgelopen 15 jaar hebben veel organisatie veel geïnvesteerd in de uitnutting van gegevens. Enerzijds door middel van de aanschaf van technologie, zowel hardware als software, anderzijds door middel van projecten met concreet op te leveren informatieproducten en de daarbij behorende training van ontwikkelaar tot gebruiker.

Dit zijn significante investeringen die vanzelfsprekend om een gezonde ROI vragen. En devraag die ik hier stel is of we de ROI op al gedane investeringen kunnen verhogen?

Het antwoord van een goede consultant zou moeten zijn ‘dat hangt van uw context af’. En daar heeft die consultant helemaal gelijk in, maar met dit schrijven wil ik een scenario schetsen dat al dan niet bij uw organisatie past. Ik zie dit scenario in toenemende mate langskomen – aan de lezer de vraag of hij dit scenario herkend in haar eigen context.

Organisatie X heeft veel geïnvesteerd in het voortbrengen van gegevens; ofwel gegevens geschikt maken voor velerlei gebruik; marketing automation, risk management, financieel management,  in-externe verantwoording en sturing, etc.. De vorm van dit gebruik uit zich vaak in de vorm van rapportages, dashboards of geleide analyses.

Om bovenstaande daadwerkelijk uit te nutten heeft organisatie X ook de kwaliteit van haar gegevens moeten verbeteren, processen moeten aanpassen, rollen en verantwoordelijkheden moeten herdefinieren en belanghebbenden moeten trainen.

De stijl van ‘ontwikkeling’ wordt nog vaak gekenmerkt door een hoge mate van systeem ontwikkeling. Ofwel; er moet bekend zijn wat er gemaakt moet worden, hoe het eruit ziet, welk gegevens nodig zijn, hoe die gegevens aangepast moeten worden, wie er toegang moet hebben etc.. In deze stijl van ontwikkelen is er een sterk onderscheid tussen diegenen die ‘maken’ en diegenen die ‘gebruiken’.  Bovendien krijg je te maken met de typische kenmerken van applicatie ontwikkeling zoals de gevreesde ‘voortbrengingsstraat’ (ontwikkel, test, acceptatie, productie) en versiebeheer.

Organisatie X heeft deze stijl van systematische ontwikkeling goed onder controle, maar ondervond enkele jaren terug toch dat de gebruiker niet echt heel tevreden was en dat de voorzieningen maar mager werden gebruikt. Eén van de hoofd oorzaken die werd genoemd is dat de gebruiker maar moeilijk in staat was om zijn wensen snel te kunnen realiseren.  De gebruiker had eigenlijk de behoefte om zelf voor een deel de ontwikkelaar te zijn. Organisatie X besloot (achteraf bezien) om een andere ontwikkelstijl te introduceren, namelijk die van de ‘gedelegeerde ontwikkeling’. Een ander type gebruiker ontstond; de gebruiker die vanuit zijn domein zelf rapportjes in elkaar ging klikken en zijn achterban voorzag.

Al klinkt dit eenvoudig, deze nieuwe stijl van ontwikkeling bracht met name voor de IT-afdeling grote hoofdbrekens; hoe zit het met versiebeheer? Wie test de rapportages? Wie beheert de rapportages? Wie zorgt ervoor dat de omgeving niet omver wordt getrokken? Maar ook daar is organisatie X goed uitgekomen.

Op dit moment voelt organisatie X nog steeds dat het rendement op de investeringen te laag zijn, dit wordt nog eens versterkt door de steeds toenemende dynamiek in de omgeving die vereist dat medewerkers van organisatie X sneller moeten kunnen reageren, meer vooruit moeten kunnen kijken en schaarse middelen steeds beter moeten inzetten.

Er ontstaat een groep gebruikers met eisen die helemaal in lijn zijn met bovenstaande dynamiek:

‘Ik weet nog niet wat ik nodig heb om mijn probleem op te lossen. Ik moet exploreren, experimenteren en leren. Dus geef mij data, geef mij software, geef mij machines’.

Wederom een nieuwe ontwikkelstijl, of toch niet? Het is namelijk de ontwikkelstijl waarbij de gebruiker gelijk is aan de ontwikkelaar. En waar hebben we dat eerder gezien? Juist – in de glorietijd van spreadsheets en lokale databases. Ook toen had de gebruiker zijn eigen data, zijn eigen software en zijn eigen machine. De cirkel is rond? Deze ‘opportunistische ontwikkelstijl’ vraagt een kanteling in denken van vele partijen binnen organisatie X.

Zo is er besloten om, naast de omgeving die systematisch (en gedelegeerd) informatie producten kan voortbrengen, ook een aparte omgeving te creëren:

• Waar snel gegevens ter beschikking gesteld kunnen worden die qua volume niet of nauwelijks grenzen kent;
• Waar gegevens overal vandaan kunnen komen;of het nu een data warehouse is, gegevens van een ketenpartner of een twitterfeed;
• Waar geavanceerde visualisaties op uitgevoerd kunnen worden;
• Waar geavanceerde analytische mogelijkheden zijn;
• Waar de resultaten van deze analyses eenvoudig (mobiel) gedeeld kunnen worden;
• Waar de omgeving op kosten-efficiënte en transparante wijze opgeschaald kan worden.

Organisatie X beseft ter dege dat deze opportunistische ontwikkelstijl governance uitdagingen kent, veel vragen van de vaardigheden van de gebruiker en zeker niet de systematische en gedelegeerde ontwikkelstijlen vervangen, laat staan het data warehouse – ze zijn allemaal nodig.

Heel bewust is er voor gekozen om hier achter te komen gedurende de reis waaraan men is begonnen. Soms kan de technologie helpen om te fungeren als breekijzer om veranderingen teweeg te brengen of te leren aan de hand van experimenten.

 

The democracy of data

There is something going on for some time now, decades even. It all started with the arrival of the Internet where people voluntarily contributed data to, well, everyone who was interested. Data about themselves, their relationships, their adventures, their careers. This data was shared with consent of the owner of the data - although not everyone knew what the data was used for. So one might say that there was consent, but not informed consent.

Lets take it a step further and imagine....

What if our data  - generated by others - was given back to us and we could consent in an informed manner, that we wish to share this data for the greater good? For example; my tax information is my data, it is about me and I want to decide whether or not others can use this data. Or, suppose I can get a hold of my location/GPS data, showing all my movements. Or my point of sale data from the grocery store, showing my eating patterns. Suppose I can even get a hold of the data of the last MRI I took, my genome data or the data of my last blood test or even the data of a clinical test I was in?

Imagine....

What if I could decide to contribute this data (consensually) for the public good, where my privacy was still being honoured? What if dozens of people would decide that? What if millions of people would decide that? Clinical research would never be the same again. We would be able to scan for patterns in seas of data consisting of environmental data and healthcare data. No more clinical trials with just 2000 people and ever-increasing smarter statistics. In this setting the healthcare specialists, the quants, the sociologists and the behavioural scientists would have an unprecedented test bed of data. Is there a correlation (or even causality) between aspects of travelling, career, eating patterns, social status and cancer? Suppose even several generations would contribute their data; what would that mean for clinical research? Mindblowing.... 

In the above I discussed data that was about myself and so I should be the one who should decide whether or not to share. But what about data that is ours? The government heavily sponsors research in many countries. Research on biology, behavioural science, economic science, climate science etc.. Shouldn’t the data generated by this research be public domain? I think it should...

What about data created by government - which is us. Data about im- and export movements, data regarding employment, schooling, law enforcement, crime, etc.. 

Imagine....

What would this democratization of data mean with regard to innovation? I think it would truly ignite a burst of possibilities and a huge potential for our general wellbeing. And no - I am not referring to the challenge of marketing handbags to middle age ladies (quote somewhat paraphrased from Neil Raden).

No, set the data free to go for the real challenges we face; decreasing poverty, climate control, improving healthcare, scarcity of resources, economic stability and decreasing crime.

This blogpost is hugely inspired by John Wilbanks - google the guy (!) -, all the Open Data initiatives of the world where governmental agencies free up their data, the technological possibilities of data storage, data deployment, data enrichment, data visualization and advanced analytics and finally...this blog is inspired by a deeply felt wish and conviction that our field of knowledge (data management and data utilisation) can make a contribution to a better place for us to live in.