Over the last few weeks, Mathias Brink and I have worked hard on the topic of Data Vault on EXASOL.
Our (simple) question: How does EXASOL perform with Data Vault?
First, we had to decide what kind of data to run performance tests against in order to get a feeling for the power of this combination. And we decided to use the well-known TPC-H benchmark created by the non-profit organisation TPC.
Second, we built a (simple) Data Vault model and loaded 500 GB of data into the installed model. And to be honest, it was not the best model. On top of it we built a virtual TPC-H data model to execute the TPC-H SQLs in order to analyse performance.
Our goal: To compare the performance of executing TPC-H SQLs on the original persistent data model and the new Data Vault model with a virtual TPC-H data model on top.
Our result: What can I say? At first, it was disappointing when compared to the performance of the classic TPC-H result that EXASOL had achieved.
But we overcame our disappointment and delved deeper into the details. Mathias optimized the view statements and together we undertook a little physical optimization of the Data Vault tables. Now, performance was much improved. Next, I tuned the Data Vault data model, still not using any PIT or Bridge tables. As a result, we had infinitely better performance. Incredible! The TPC-Hs on Data Vault and on the classic model are almost the same.
We’re still working on optimization. So I hope I can publish some detailed facts soon and some more news of EXASOL’s roadmap.
Read ITGAIN and EXASOL’s press release results from our work (in German):
ITGAIN und EXASOL bringen Agile Modellierung und Analytische In-Memory-Datenbank zusammen
Es ist kein Geheimnis mehr, dass Data Vault als Modellierungsmethode auf dem Vormarsch ist. Allerdings werden Einführungs- und Umsetzungskomplexität, sowie technologische Anforderungen im Hinblick auf die Query-Performance des zugrunde liegenden Datenhaltungssystems meist unterschätzt.
ITGAIN Consulting und EXASOL entwickeln aus diesem Grund ein Best Practice Modell für die Umsetzung von Business Intelligence Anforderungen mittels Data Vault Modellierungsansatz und der auf intelligentem In-Memory Design und skalierbarer Shared-Nothing Architektur basierenden High-Performance Datenbanktechnologie von EXASOL.
[…]
Read more on ITGAIN.
So long,
Dirk