Big data serving: The last frontier. Processing and inference at scale in real time.

The big data world has mature technologies for offline analysis and learning from data, but have lacked options for making data-driven decisions in real time. When it is sufficient to consider a single data point model servers such as TensorFlow serving can be used but in many cases you want to consider many data points to make decisions.

This is a difficult engineering problem combining state, distributed algorithms and low latency, but solving it often makes it possible to make far superior solutions when applying machine learning. This talk will explain why this is a hard problem, show the advantages of solving it, and introduce the vespa.ai platform which is used to implement such solutions in some of the largest scale problems in the world including the world's third largest ad serving system.

This talk introduces vespa.ai - a mature open source platform for storing, selecting processing and making model inferences over large data sets at end user request time.