Implementation and automation using Data Vault (Day 3)

Roelant Vos, Allianz Worldwide Partners

This full day session is relevant for anyone seeking to learn the intricacies of Data Vault implementation and leverage ‘model-driven-design’ and ‘pattern-based code-generation’ techniques to accelerate development. As advanced modelling and implementation techniques for Data Vault are also covered, this applies to a wide range of data professionals including Business Intelligence and Data Warehouse professionals, data modelers and architects as well as DBAs and ETL specialists.

Data Vault has emerged as the leader of contemporary data modelling techniques specialized for Data Warehouse design. The associated methodology provides some elegant handles to develop your Data Warehouse – the various required Data Warehouse mechanics are organised in a way that allows for a flexible solution.

But you’re still delivering an Enterprise Data Warehouse and the associated complexities will need to be addressed somewhere.

This session intends to briefly revisit the fundamental Hub, Link and Satellite concepts but will quickly move to the various other considerations required to define a truly resilient and flexible Data Warehouse solution.

When all these necessary considerations have been incorporated, the ETL patterns start to look a fair bit more complicated. The reasoning for this, and the modelling and implementation choices you have to make along the way – as well as their consequences – will be discussed in this day of Data Vault implementation.

Practical development is further considered by discussing the patterns (and their various nuances) in the context of how metadata can be organized to support automation.

At the end of the session you will understand how an ETL generation metadata model can be defined and configured to suit specific needs, but also which exception cases need to be supported and how to value existing available metadata-driven approaches.

Contents:

  • Overarching principles: what concepts should a solution support?
  • Data Vault implementation patterns, what kind of considerations are there?
  • Loading multiple changes in on go using record condensing and change merging for Satellites. When is a change a change?
  • What prerequisites need to be in place? (ETL framework, conventions, patterns)
  • How do database-level configurations support your Data Vault?
  • Understand the difference between Raw and Business Data Vault
  • ETL generation – how does this fit in and how do I get started?
  • What metadata do you need and where do you store it?
  • How to balance performance issues using helper constructs (e.g. PIT, Bridge)?
  • Getting data out again – ‘time flattening’ in PIT and Dimension tables. How can information from a Data Vault be exposed through (virtual) Data Marts?
  • The role of the ETL control framework and Referential Integrity.

Roelant Vos has been active in Data Warehousing and Business Intelligence for more almost 20 years and is working for Allianz Worldwide Partners as the General Manager for Business & Customer Insights in Brisbane, Australia. In a role that is highly focused on analytics, Roelant is working on collecting, integrating, improving and interpreting data to support various business improvement initiatives. Passionate about improving quality and speed of delivery through model-driven design and development automation, he has been at the forefront of contemporary modeling and development techniques for many years. Whenever there is some time updates on these topics are published on www.roelantvos.com/blog.

Comments are closed.