The SFA has recently made the move from waterfall to agile. So how do you adapt data modelling for a Agile project?
What one produces and why one produces it doesn’t change, but how it gets produced does. Agile data modelling still adheres to the organisation’s data modelling framework and most definitely to its data modelling standards and notation. This is not data model anarchism.
The ‘how’ is about approach and mindset.
The Waterfall approach allows one the luxury of time to produce the complete data model – although one never really knows when a data model is complete, so surely it’ll be struggle to know when it is just good enough. Well maybe, but the ‘just good enough’ actually gives the data modeller a degree of freedom not so evident in the Waterfall approach. Rather than going around in ever decreasing circles trying to squeeze the last drop out of a data model, the modeller has licence to declare the model incomplete, but still fit for purpose.
Architecture is carved into domains of Technology, Infrastructure, Security, Business, Data and Application – some or all of which appear in various frameworks. The on-the-ground reality of an Agile project is that the Business Architects/Analysts and the Data Architects/Analysts had better start getting very friendly because they should be asking the business remarkably similar questions. This also means they should be spending a lot of time in the same meetings, not least to avoiding irritating the business folk by separately asking them the same questions. Collaboration isn’t a dirty word after all.
Data models buried inside some software which only the data modellers has access to is hardly a lesson in collaboration. ‘You really do your data modelling in Visio/Excel?’ comes the cry from the Data Model police. Well they can cry whilst the rest of us share easily understood material in widely available software. Oh, and it’s cheap. Make sure those data models see the light of day - produce data model wallpaper if you have to.
Like any methodology, Agile isn’t perfect. Its iterative nature can make it difficult for the data modeller as they try to keep the various types of models in line with each other. There is an obvious tendency to minimise the number of data models, thus reducing rework. The challenge with mixing Logical and Physical data models into one, is the loss of information. Each type of model gives the reader a particular set of knowledge and it’s hard to maintain both sets of knowledge in a single model. Guess this is why projects have Data Architects/Analysts.
So, we may have Logical and Physical data models, be working within a framework, to standards and adopting a common notation, but this shouldn’t limit imagination. A picture saves a thousand words (well at least two thousand if it’s a data related picture) so create whatever data related models are of value and, conversely, only produce data related models that are of value (a bit of Lean thinking).
Just in case anyone thinks data doesn’t matter. It does. Whilst the costs of buying storage might get ever lower, the cost of managing storage doesn’t and managing a data estate that is n times larger than it needs to be cripples budgets. Projects must give data the consideration it deserves.