Make it Streaming: Highs and Lows on the Way to Real-time Risk

In a world where trades are done on iPads and cars park themselves in garages, banks need to be quicker at decision making and have to calculate risks on the fly to keep up with customers, markets and regulators. It is not good enough to produce risk figures overnight – it needs to be done here and now, fast, precise, reliable and also reactive to changes as they happen

I am going to tell you a story of how we (re)designed our systems to streamline risk calculations and move from an overnight 8 hour batch to near-real-time display. How we came from huge relational databases with lots of clunky procedures to decoupled microservices feeding in-memory cubes. How event sourcing helps and why a data log really matters. But where the two seas of batch and streaming meet, there is turbulence. We will see how to deal with thorny consistency issues arising from streaming systems.