Novel swap-based representations enable robust automatic balancing of two-player game levels
Florian Rupp, Manuel Eberhardinger, and Kai Eckert presented “Balancing of competitive two-player Game Levels with Reinforcement Learning” at the 2023 IEEE Conference on Games (CoG).
The Manual Balancing Challenge
In competitive two-player games, level balance is crucial for ensuring fair and engaging gameplay. When one player has a systematic advantage due to level layout—such as better starting positions, more accessible resources, or favorable terrain—the game becomes frustrating for the disadvantaged player. This is especially problematic in non-symmetrical game levels, where ensuring balance requires extensive manual work and repeated testing cycles.
Automated Architecture
The research proposes an innovative architecture for automated balancing of tile-based levels within the PCGRL framework. The architecture comprises three integrated components:
- Level Generator: Provides initial level layouts that serve as starting points
- Balancing Agent: Uses reinforcement learning to iteratively modify levels
- Reward Modeling Simulation: Plays the level repeatedly to assess balance
The Power of Simulation
By simulating gameplay repeatedly, the system can accurately assess whether a level is balanced. The balancing agent receives rewards for modifications that move the level toward equal win rates for both players. This simulation-driven approach eliminates the need for expensive human playtesting during the optimization process.
Swap-Based Representations: A Key Innovation
A critical contribution of this work is the introduction of a novel family of swap-based representations. Unlike traditional PCGRL approaches that replace selected tiles, swap-based representations exchange existing level elements, which offers several advantages:
- Playability Preservation: Swapping maintains overall level structure and connectivity
- Robustness: Less likely to create unplayable or broken level configurations
- Efficiency: Faster convergence to balanced solutions
- Interpretability: Changes are more understandable to human designers
Targeting Equal Win Rates
The system explicitly optimizes for equal win rates, a clear and measurable balance objective. This focus allows the balancing agent to make principled decisions about which modifications improve balance. While win rate is not the only aspect of balance, it provides a solid foundation that other objectives can build upon.
Practical Application
This research demonstrates practical applicability to actual game development scenarios. The tile-based representation is common in many game genres, including strategy games, tactical RPGs, and puzzle games. Game developers can potentially integrate this approach into their development pipelines to reduce manual balancing effort.
Foundation for Future Work
This 2023 paper laid the groundwork for subsequent research by the team, including the IEEE Transactions on Games publication that expanded on these concepts and the empirical evaluation study that validated the approach with human players. It represents an important early step in demonstrating the viability of automated game balancing through reinforcement learning.
Citation: Florian Rupp, Manuel Eberhardinger, Kai Eckert (2023): Balancing of competitive two-player Game Levels with Reinforcement Learning. In 2023 IEEE Conference on Games (CoG).