ESG4Stars

ESG4Stars, Sustainable Scientific Workflows for Research Infrastructures (RIs), aims to reduce the environmental footprint of the entire scientific computing and data analysis lifecycle on European RIs. From data transfer through compute requirements to data storage, the project will achieve quantifiable reductions for each component of that footprint. Building on a pan-European federation of Galaxy servers established under the EOSC EuroScienceGateway (ESG) project, ESG4Stars will track energy and resource usage of scientific compute jobs across thousands of compute tools run by tens of thousands of users from multiple scientific disciplines and will publish these metrics in machine-readable FAIR format. Domain experts within the project will use this data to compile a hotspots catalogue of analysis tools, workflows and used file formats with the biggest potential for sustainability improvements. Corresponding optimisations, implemented by tools and workflow developers in the project, will realise this potential without sacrificing scientific throughput or reproducibility. The data will also provide the basis for a reusable job-resource-calibrator service for tailoring resource allocations to new compute jobs to avoid wasteful hardware resource overprovisioning. In parallel, platform optimisations like improved support for efficient, next-generation file formats, a compute-local data cache, shared reference data and ML/AI models, upload deduplication and promotion of non-persistent data storage through the user interface will further reduce data transfer and/or storage needs, while a global job-cache will avoid redundant computation. Combined with training, awareness building among users, and a green-ready checklist for infrastructure operators, the project will provide sustainable solutions that are transferable across Science Clusters for curbing the energy consumption of European RIs despite ever-growing data volumes to be analysed.