Predicting Response Time in a Large-Scale Information Retrieval System

Meltwater’s media intelligence platform executes millions of search queries every day. An Elasticsearch based platform contains 40+ billion documents and provides search results and analytics.Queries can be interactive (a human waiting on the result) and non-interactive (batch-like queries for asynchronous reports). While most queries take in the order of milliseconds, some queries can be expensive and take seconds or minutes to execute. These ‘slow’ queries have a detrimental effect on the user experience, by hogging machine resources and increasing wait times. A 10-millisecond query that waits 30 seconds creates a bad quality of service.Accurately predicting the execution time of queries makes it possible to segregate slow queries into a separate machine pool, thus mitigating the negative impact of these queries. Historical query logs contain an abundance of data and features to train a machine learning model on. However, the problem domain is complex and the data is noisy. With thousands of queries running concurrently across more than 400 machines causing interactions that affect query runtime.This talk will describe the problem and the challenges it poses from a machine learning perspective. It will describe the attempts to solve it, with a particular focus on techniques and algorithms that are useful when working with large-scale noisy data.