ExtremeXP

Extreme data characteristics (volume, speed, heterogeneity, distribution, diverse quality, etc.) challenge the state-of-the-art data-driven analytics and decision-making approaches in many critical domains such as crisis management, predictive maintenance, mobility, public safety, and cyber-security. At the same time, data-driven insights need to be extremely timely, accurate, precise, fit-for-purpose, and trustworthy, so that they can be useful. ExtremeXP will handle the complexity of matching extreme needs with complex analytics processes (i.e., processes that involve and combine ML, data analysis, simulation and visualization components) by placing the end user at the centre of complex analytics processes and relying on user intents and running experiments (i.e., trial and error) to prune the vast solution space of possible analytics workflows and configurations i.e., “variants”. Its main goal is to create a next generation decision support system that integrates novel research results from the domains of data integration, machine learning, visual analytics, explainable AI, decentralised trust, knowledge engineering, and model-driven engineering into a common framework. The overarching idea of the framework is to optimise the properties of a complex analytics process that the end user cares about (e.g., accuracy, time-to-answer, specificity, recall, precision, resource consumption) by associating user profiles to computation variants. The framework is envisioned as modular and extensible, orchestrating different services around an Experimentation Engine: Analysis-aware Data Integration, Extreme Data & Knowledge Management, User-driven AutoML, Transparent & Interactive Decision Making, and User-driven Optimization of Complex Analytics. The framework will be validated in five pilot demonstrators.