From Hypothesis to Reality: Designing a Superhuman Racing AI Agent Using a Deep Reinforcement Learning

This work started out as a grand challenge to create an AI agent that could beat the world’s best Gran Turismo (GT) drivers. In order to develop an agent capable of competing against the world’s best drivers, GT Sophy was trained to master the following driving skills: race car control, racing tactics, and racing etiquette. This talk will tell you a story about the technical evolution of GT Sophy from a research outcome to an in-game feature that could be introduced as part of the Gran Turismo 7 (GT7) PlayStation racing game. Sony AI, in collaboration with Polyphony Digital and Sony Interactive Entertainment, designed novel deep reinforcement learning approaches with unique training and evaluation methods on a modern cloud computing platform to accommodate this project. In less than two years since GT Sophy appeared on the cover of Nature, the breakthrough AI agent has now become a permanent in-game feature of GT7.