To achieve carbon neutrality by 2050 and limit global warming to below 1.5°C, the energy sector must shift to more renewable energy sources, moving away from fossil fuels. This shift will lead to a dramatic transformation of the entire energy landscape, including the electricity system. However, renewable energy sources such as solar and wind are inherently intermittent and unpredictable, posing substantial challenges to grid stability. Tackling these issues will require enhanced energy flexibility, which means having better capabilities in dealing with supply and consumption imbalances. While the overall energy demand is expected to decline, electricity demand is projected to rise due to the higher levels of electrification. Moreover, the traditional electricity grid architecture is evolving from a hierarchical structure, where power is distributed from large central plants to consumers, towards a decentralized framework, where consumers also act as producers. This shift makes power flows bidirectional and integrates a vast amount of small, geographically distributed energy resources (DER). All these recent changes bring new challenges to the electricity system.

A promising solution to manage this growing number of DERs is aggregation, where multiple assets are grouped to function as a single entity in the electricity system, called a VPP, and are managed by a controller to provide energy flexibility. Finding such a control strategy comes with some challenges. Firstly, the system must cope with variability and unpredictability on both the demand and supply side. Secondly, the assets under control, for example, an EV charging station, chillers, etc. are often very hard to model due to their complexity, unknown physical relations or parameters. Third, there is an important safety concern that naturally comes with providing flexibility to the grid. Lastly, the possible assets under control are very diverse, have different characteristics, and work on different timescales. These challenges make the process of scheduling and controlling various DERs under uncertainty with conventional techniques very difficult. For instance, current approaches using Model Predictive Control (MPC) are not able to guarantee an effective ’anytime’ solution for real-time control and scheduling. Primarily because of challenges related to computational scalability. Additionally, MPC lacks adaptability in stochastic conditions, limiting its effectiveness in real-time, unpredictable situations. However, MPC excels at handling hard constraints, making it very useful for safety-critical situations such as energy systems.

A second family of techniques that are being proposed in literature for making decisions under uncertainty emerges from the field of Artificial Intelligence (AI), called reinforcement learning. RL, a data-driven technique, does not necessarily require a physical model. This is also the case for other data-driven techniques. RL for instance can learn its control policy through interactions with the environment. Despite this, RL’s exploratory nature poses safety risks, and it struggles to generalize its learned policies to new or unforeseen circumstances, making it less adaptive in certain cases.

This PhD research aims to enhance energy flexibility through advanced and smart coordinated control systems that can address the challenges of aggregating and coordinating DERs, by leveraging the advantages of both MPC and learning-based approaches. These developed methodologies will be trained with historical, real-world data and tested in real-world simulations to demonstrate their value. In addition, a benchmark study will be conducted to compare the new developed methodologies with other state-of-the-art techniques. The main challenges with which the controller needs to cope are: the inherent uncertainty and variability of load- and supply-side in the energy system, ensuring safety by respecting the system’s operation limits, the uncertainty about the physical systems under control and being able to adapt to new circumstances. To realize this, three sub-objectives are established: (1) Enhance energy flexibility by a safe, adaptive hybrid learning-based control strategies. (2) Research to multi-controller, multi-level and decentralized control approaches. (3) Evaluation and benchmarking of methods for impact.