Leveraging Open-Source Tools for Building a Quality Data Warehouse

In recent years the modern data stack has seen a great increase in the number of tools available on the market, making the landscape overwhelming and difficult to navigate. In this presentation, we will give an overview of how Kognic built a data warehouse with quality as its core value. We will present our architecture and explain how we leverage some popular open-source tools in our setup, highlighting their main features and shortcomings. Particular emphasis will be put on how data is transformed for the data mart using a test-driven approach that aims to stay true to software-development best practices.