AI-ECC

How can AI be reliably used to coordinate critical workloads in the edge-cloud continuum despite the risks? To solve this challenge, we are developing a planning model that optimizes CPU, GPU, bandwidth, and memory across the edge-cloud continuum, enables controlled resource over-provisioning, and integrates these transparently into the Fraunhofer Edge Cloud (FEC). The benefits of the model are being evaluated at Fraunhofer IPT. We are also creating the technical basis with appropriate monitoring, dashboards, and automated energy data collection, as well as a clear implementation process that ensures the reliability, explainability, and policy compliance of the solution.

The challenge

The growing use of the Fraunhofer Edge Cloud (FEC) is leading to capacity bottlenecks, unused instances, and high complexity in resource management. AI workloads in particular require accurate resource allocation, avoidance of bottlenecks, and reduction of error rates—without compromising performance or sovereignty. Heterogeneous platforms and the need for transparency in terms of efficiency, energy, and CO2 emissions are increasing the pressure to introduce a secure, traceable process that goes beyond ad hoc optimizations and is suitable for institutional use.

Our service

We develop and integrate an AI model that intelligently plans critical workloads, optimally allocates resources, and allows controlled resource oversubscription to increase efficiency and minimize error rates. Technically, we integrate the model into the FEC—with end-to-end monitoring, dashboards, and automated collection and evaluation of energy and usage data for data-driven, auditable decisions.

An LLM-supported multi-agent system plans and implements changes in a HITL (Human in the Loop) controlled manner; calibrated models with power distribution unit (PDU) balancing (≤ 5–10% deviation) and uncertainty bands that define upper and lower limits for measured values ensure quality. Regular performance reviews, reports on resource utilization, efficiency, and potential bottlenecks, as well as recommendations for time- and location-specific placement in the edge-cloud continuum ensure an orderly introduction with clear responsibilities. At the same time, we address energy and CO2 impacts and lay the foundation for economic decisions.

The result

AI-ECC reduces energy consumption by around 63 percent without any observed deterioration in SLO (service level objective) and increases both utilization and stability. In the FEC, infrastructure and measurement and evaluation systems were set up productively, energy data was collected automatically and visualized in meaningful dashboards, making consumption, cost levers, and optimization opportunities transparent. The combination of secure AI planning, controlled resource over-allocation, and continuous evaluation reduces error rates, simplifies resource management, and provides a measurable basis for FinOps and Sustainability. The results are evaluated at Fraunhofer IPT, made transferable to other Fraunhofer locations, and presented in reports and demonstrations.

The partners

• Fraunhofer IPT