Taking sustainability and agility in data modelling to the next level
Both authors (Frederik Hofstra and myself) of this whitepaper are involved in a data warehouse project with a university in the Netherlands. This university implemented a new SAP module; Student LifeCycle Management. A comprehensive module covering all processes from enrolling students, to credit scoring, to progress reports, to eventually graduating. This module is also the main source for master data like the academic structure of programmes and modules, as well as the organisational structure (faculties, divisions, subdivisions, etc.) and the student information.
The strategy of the university is to deliver a service to their organisation that is capable of delivering data in various ‘truths’ to people or information systems and with an array of functionality (e.g.: reporting, analysis, etc.).
The solution architecture consists of two storage layers. The first one is based on Data Vault principles and heuristics and the second one is a dataset layer. The latter is the object of discussion in this paper. This solution architecture is currently available in production mode and is - as such - a proven solution.
This paper is all about delivering quality data products. However, quality is by definition a relative construct. Its requirements can only be fixed by the time the user needs the data. Context - as we use the word in this paper - is a function of the person involved at a specific moment in time with the quality attributes he desires. And these contexts change all the time. Accommodating these constant changes is partly covered by continuous innovation in technology. However, the use of innovative technologies does not free us from proper software design. This paper addresses a software design perspective to offer the required agility.
It is a continuing effort in trying to evolve the work of giants like Dr. R. Kimball, W.H. Imnon, B. Devlin, C.Imhoff, D.Linstedt and many many more.