CompetEvo: Towards Morphological Evolution from Competition

Kangyao Huang, Di Guo, Xinyu Zhang, Xiangyang Ji, and Huaping Liu 📧

The key insight of CompetEvo: agent in its original morph is at a disadvantage in competitive confrontations with the opponent. However, after undergoing N generations of co-evolution in both morphology and tactics, agent with new morphology and combat tactics can overcome the original opponent in competition. Using spider and ant as an example.

Abstract

Training an agent to adapt to specific tasks through co-optimization of morphology and control has gradually attracted attention. However, whether there exists an optimal configuration and tactics for agents in a multiagent competition scenario is still an issue that is challenging to definitively conclude. In this context, we propose competitive evolution (CompetEvo), which co-evolves agents' designs and tactics in confrontation. We build arenas consisting of three animals and their evolved derivatives, placing agents with different morphologies in direct competition with each other. The results reveal that our method enables agents to evolve a more suitable design and strategy for fighting compared to fixed-morph agents, allowing them to obtain advantages in combat scenarios. Moreover, we demonstrate the amazing and impressive behaviors that emerge when confrontations are conducted under asymmetrical morphs.

Method

To incorporate morphology into consideration, we construct a network composed of morphology sub-networks and adversarial tactics sub-networks. At every beginning of the episode, each player utilizes the morphology sub-network and a set of initial parameters to generate a new morphological parameter vector, forming the corresponding morphology, and subsequently engages in combat. Then, players fight and learn their skills under the current morphology. Three kinds of observations are used for training a tactics sub-policy: the current morphology, states of agent itself, and observations of opponent. Therefore, after sufficient training, morphology sub-policy tends to converge to a relatively fixed morph; tactics sub-policy can also converge to a stable strategy that adepts to the current fixed morphology.