Find in Library

A key to ensuring the security of smart electrical power grids is to devise and deploy effective defense strategies against cyberattacks. To achieve this goal, an essential task is to simulate and understand the dynamic interplay between the attacker and defender, for which stochastic game theory and reinforcement learning stand out as a powerful mathematical and computational framework. Existing works are based on conventional Q-learning to find the critical sections of a power grid to choose an effective defense strategy, but the methodology is only applicable to small systems. Additional issues with Q-learning are the difficulty in considering the timings of cascading failures in the reward function and deterministic modeling of the game, while attack success depends on various parameters and typically has a stochastic nature. Our solution for overcoming these difficulties is to develop a deep Q-learning-based stochastic zero-sum Nash strategy solution. We demonstrate the workings of our deep Q-learning solution using the benchmark Wood and Wollenberg 6-bus and the IEEE 30-bus systems; the latter is a relatively large-scale power-grid system that defies the conventional Q-learning approach. Comparison with alternative reinforcement learning methods provides further support for the general applicability of our deep Q-learning framework in ensuring secure operation of modern power-grid systems.