Data Warehouse Architecture in Microservices world

In this talk I will cover some issues that arise during movement from monolith to microservices. Examples from our projects in fintech will be employed. 

Main one, definitely, is about ultimate knowledge – after moving to microservices, usually there is no specific component responsible for any particular process in general that can be used as a main data source. Instead, there are many of them appeared, and, suddenly, Data Warehouse become the first place where all data about specific processes is being collected.

Another important subject here is data transfer - while it's more or less essential for monolithic age's ETL to use API for batch loading of data, in microservices more ofter event-based model is used. So it is important to change the approach of loading and cleaning the data.

We will cover some other issues like architect requirements for building DWH in microservices, platform-specific issues, and so on.
At the last section we will briefly cover issues with a querying microservices data – what king of exploratory tools is needed, like lineage, catalog, dashboards, quality checks, and so on is suitable with a microservice architecture.

All this issues and some other, along with our experience in dealing with them, will be presented.