A Virtual Character Learns to Defend Himself in Sword Fighting Based on Q-Network

In this paper, we create a virtual character that interacts with human player in a sword-fighting task. The data taken from a human player waving a stick as the 'sword' is mapped into the virtual environment, and the result of the collision detection during the virtual interaction is used as a reward for the reinforcement learning. We train a Q-network that animates the virtual character by rotating its joints toward the blocking the incoming attacks in each individual frame base on the current state. The trained virtual character makes quick and accurate blockings to 'sword' attacks coming in various forms. Further, a human-body dynamic model is used to predict the movements of the task irrelevant joints of the virtual character to create a more human like behavior. Experiment results demonstrate that this method can inspirit a virtual character with an automatic, human-like animation that interacts with human player in real time.