2023 Author: Bryan Walter | [email protected]. Last modified: 2023-05-21 22:24
American engineers have created a robot that can cut vegetables into slices. The peculiarity of the approach chosen by the developers is that initially the robot learned a different task, which helped to learn the basic skill - first, the robot was trained to predict the thickness of slices, and only after that it learned how to cut them qualitatively, say the authors of the article published on arXiv.org.
Object manipulation is one of the main tasks in robotics. To perform even simple actions with objects, the robot must have many skills: be able to recognize objects, calculate the optimal grip or other action, plan the trajectory of the manipulator and predict the properties of objects. The latter skill is extremely important when interacting with soft objects, the shape and other properties of which can change directly during interaction. One of the model tasks that allows you to work out the skills necessary for interacting with various objects is cutting vegetables. During this seemingly simple action, the robot is forced to work with objects that deform when interacting with the knife, which, for example, leads to a change in the cutting path and the final shape of the slice.
Oliver Kroemer and his colleagues at Carnegie Mellon University have taken an unusual approach to solving this problem. First, they trained the algorithm to perform an intermediate task - predicting the thickness of a slice and the remaining vegetable from a single 2D photograph. To do this, they created a dataset consisting of pairs of images taken before and after cutting. During the creation of the dataset, engineers created a random slicing plan (several slices of a given thickness), and then sliced the vegetable and took pictures of it. Thus, the authors of the work collected 50 demos (several cuts each) for cucumbers and 25 for tomatoes. In addition, engineers trained the neural network to detect vegetables using a dataset of about 4,000 cucumbers and tomatoes.
While training a neural network to predict the thickness of a slice, engineers taught it to translate the raw data in the form of a snapshot into a vector representation. The authors of the work note that this made it possible to obtain a trained part of the neural network capable of linking pictures of vegetables with their properties. After training the intermediate algorithm, the developers started training the main one. Its task is to draw up the trajectory of the manipulator with the knife. To do this, the engineers used the method of simulation training, in which the authors controlled the movement of the manipulator with their hands while cutting the slices, and the algorithm subsequently tried to recreate similar movements as accurately as possible.
The scheme of training and operation of the algorithm
Engineers experimented with a robot with a camera and two manipulators. One of them is responsible for fixing the vegetable on the board, while the other holds a knife and cuts off the slices. During slicing, the manipulator moves the knife towards the vegetable and stops during contact. After that, he lifts the knife, moves it at a pre-calculated distance that provides the desired thickness of the slice, and proceeds to cut, during which he imitates the movements of people during a similar action.
Earlier, another group of American engineers taught a robot with a manipulator to interact with vegetables and fruits in order to feed people with a fork. The developers have trained the robot to handle different types of food in different ways. For example, since a piece of banana can slide off a fork, the manipulator does not prick it vertically, but at an angle.
