Mark Madsen, Third Nature
The problem we designers need to solve isn’t “big data” or “small data”—it’s all data. The data warehouse is sufficient for a portion of the data we manage, but not for all of it.
The requirements we have today are to accept any data, not just rigidly structured data in rows and columns; to accept that data at any speed, not just what the database can keep up with; to deliver via any means, not just SQL-based BI tools; and to support any process—not just queries but also algorithms and transformations.
The technology that we use is problematic because it constrains and sometimes prevents necessary activities. We don’t need more technology and bigger machines. We need different technology that does different things. More product features from the same vendors won’t solve the problem.
The big data market has set itself up as an alternative to the data warehouse, not realizing the new technologies solve different problems and aren’t appropriate for some of the original problems. This is really a confusion of technology with architecture.
Architecture is more than just software. Architecture starts from use, and includes the data, methods of building and maintaining, organization of people, as well as the software. We are also in an emerging technology space when it comes to data. This requires exploratory design practices, something we’ve largely discarded over the last 10 years as data warehousing and BI matured.
You will Learn:
- Data architecture alternatives to those of the past that are able to adapt to today’s data realities
- New technologies that can be applied to address new problems inherent to the scope and scale of data today
- Methods and techniques to migrate from old data architecture of the past to new data architectures that resolve today’s problems and prepare for the future.
Mark Madsen is president of Third Nature, a technology consulting and market research firm focused on business intelligence, data integration, and data management. Mark is an award-winning architect and former CTO whose work has been featured in numerous industry publications. He is a principal author of Clickstream Data Warehousing (John Wiley & Sons, 2002) and frequently speaks at conferences and writes about business intelligence and emerging technology.