To enhance user experience, datacentres monitor millions of resource usage series, resulting in big data to gather useful insights. Dapprox derives methods and tools to predict performance anomalies in real time by selecting a key subset of data and proposing solutions to better manage resources.

Lay summary

Dapprox is a set of methods and software tools for fast and approximate analyses of resource usage series in real time. The goal of Dapprox is to predict potential anomalies (and propose solutions) by simultaneously taking into account accuracy requirements, maximum delays and available resources. Dapprox first looks for characteristics that are common across servers over time, and then processes only subsets of “key” data in a way that does not sacrifice the accuracy of the results. Particularly, Dapprox can dynamically select and process the optimal amount of data, based on common structures that change over time. Dapprox comprises three work packages: dependency-aware predictive analytics for forecasting, approximate streaming analytics for live data and datacentre anomaly management.

To ensure quality of service and system reliability, datacentres monitor and collect performance logs from many virtual and physical computing resources. The sheer quantity of data generated is so large that it is nearly impossible to always correctly analyse it in real time. Existing analyses tend to be unsophisticated and slow, which leads to delays in addressing performance anomalies and significantly degrades end-user experience.

Our goal is to analyse performance data to better manage computing resources in cloud datacentres and thus to enhance user experience. But rather than analysing all of the data, we will develop approximate analytics – i.e. methods and tools based on subsets of data – to predict complex patterns of resource usage series and so-called critical states. We will also create tools for real-time processing and anomaly analysis. Finally, we will propose anomaly management policies for cloud datacentres.