A gaming table, a puck hurtling across an air hockey table, and a robotic arm that plays like a pro. Three UBC engineering physics students (University of British Columbia) have succeeded in a feat that could potentially redefine how robots are trained. The three young researchers created an air hockey robot driven by an AI capable of challenging and beating humans, despite having learned to play exclusively within a virtual simulation.
Generally, to teach a robotic system to perform complex gestures in the physical world, the method of trial and error is used directly in the field. This classic approach, although valid, involves an enormous waste of time and the real risk of wearing out or breaking the mechanical components due to continuous initial failures. Canadian researchers instead got around the problem by transferring the entire learning phase into virtual space, developing a “digital twin”, that is, a highly detailed digital replica of the real table, within which the algorithm was able to play millions of games and make infinite errors without suffering any material damage.
Once this simulated study path was completed, the virtual brain was copied and transferred to the mechanical body of the robot. The result is interesting: from the first moment, the machine proved to be immediately ready to compete, demonstrating extraordinary effectiveness against real opponents and opening new frontiers for the training of autonomous systems of the future.
How the robot was trained
But how do you prepare software to handle the chaotic reality of an air hockey game without ever having it touch a real air hockey table? This sport is a real challenge for robots, since the disc moves at very high speeds, bounces unpredictably and is affected by minimal variations caused by impacts with the banks or with the game knob. Furthermore, in physical reality complex technical obstacles come into play such as latency (i.e. the time delay with which data passes from the camera to the engine), micro-vibrations of the structure and slight drops in electrical voltage.
To overcome these barriers, the UBC team adopted a counterintuitive strategy: they designed a purposefully imperfect virtual environment. Through a technique called “domain randomization”, the students inserted disturbing elements such as irregular sides, slightly deformed tables and bouncing anomalies into the simulation. This allowed the artificial intelligence not to rely on rigid and perfect geometric calculations, but to learn to predict an approximate trajectory of the puck, preparing to manage the unexpected exactly as a human player would.
To make learning fast and efficient, the researchers avoided classic commercial graphics engines and implemented an advanced algorithm based on the “soft actor-critic” mechanism. The system belongs to the family of reinforcement learninga technique in which the algorithm learns by receiving rewards when it makes good decisions and was designed to maximize performance without giving up experimentation with new strategies.
To materialize all this in the physical world, the real table was equipped with a camera positioned from above and a disk covered with retro-reflective tape, a material capable of sending light directly back towards the source that emits it, allowing the electronic eye to track movements at 120 frames per second.
You can appreciate the result of using this system from the following video.
The possible future implications of the experiment
The real victory of this experiment, originally born within the university laboratory to create a teaching platform for future automation courses, goes far beyond simple fun. The students overcame months of technical challenges to integrate the robot’s mechanics with a control model feedforwarda predictive system that anticipates errors before they occur and applies corrections in real time.
The success of this transfer from virtual to real bodes well for the future, opening up scenarios in which safely and quickly training drones, autonomous vehicles, industrial robots, etc., within realistic simulations will not be utopian at all.
