2023 Author: Bryan Walter | [email protected]. Last modified: 2023-05-21 22:24
Researchers in Canada and Singapore have developed an algorithm that allows bipedal virtual robots to learn how to walk and run through trial and error, much like humans learn the same skills. Scientists believe that later this algorithm can be used to train real robots, as well as to create computer animation in games and films. The algorithm was presented at the computer graphics conference SIGGRAPH 2017, and its detailed description is available on the University of British Columbia website.
Previously, to teach computer programs or robots to do something, engineers had to "manually" write the behavior and response to certain conditions in the program codes. In recent decades, a different approach has been increasingly used - machine learning. It allows the trained algorithms not only to follow predetermined algorithms, but also to independently search for the most optimal, in their opinion, method for solving the problem.
Canadian engineers decided to use this strategy to create computer characters and robots that walk efficiently and realistically on two legs. To do this, they used deep reinforcement learning. This type of machine learning implies that the learning algorithm, when interacting with the environment, receives an answer - a reward or a penalty. The implementation of the algorithm presented by the researchers consists of two main components - low-level and high-level controllers-schedulers. The low-level component is responsible for planning specific steps, walking style, and takes into account the parameters of the nearby relief. The high-level controller was responsible for longer-term planning - for example, allowing the robot to plan its route taking into account obstacles.
Training takes place in a virtual environment with variable parameters. So, the robot can be on a narrow path in the mountains or on ice. In addition, the environment changed dynamically. For example, flat and stationary surfaces were replaced by a moving surface like a travelator, and cubic blocks of different sizes also periodically fell on the robot.
Thanks to machine learning, the robot learned to move dexterously and quickly in different conditions and even kick the ball towards the target. The researchers believe that in the future, the algorithm can be adapted for a variety of tasks, not only related to robotics. For example, it will be able to create anatomically accurate animations of people in games and films using computer graphics to replace the cameras and motion capture sensors used today.
Despite the fact that there are other systems for teaching algorithms in virtual spaces, transferring skills to the real world or between robots of different designs is a serious problem. Recently, experts from the Massachusetts Institute of Technology said that they partially solved this problem and created a system that facilitates the transfer of skills between robots of different designs.
