2023 Author: Bryan Walter | [email protected]. Last modified: 2023-05-21 22:24
Indian developers have created a neural network that searches video recordings for people of a certain height, gender, and in clothes of a certain color. This can significantly reduce the amount of data viewed by a person when searching for people in videos, say the authors of the report, which will be presented at the AVSS 2018 conference.
Algorithms for recognizing people in videos and photographs are already quite well developed and are actually used to search for criminals or identify passengers. Typically, these algorithms identify a person only by their face, and more advanced implementations also use additional factors, such as height, or even more unusual, for example, behavioral characteristics. Some programs already know how to select only a person with a certain set of parameters in frames, but they often encounter problems when separating a person from the background, as well as incorrect determination of height due to a changed perspective and other obstacles.
A group of developers led by Mehul Raval from the University of Ahmedabad in India has created a neural network capable of recognizing people in video and filtering them qualitatively based on several features. The algorithm works in several stages. First, the frames are given to the Mask R-CNN convolutional neural network, which performs semantic segmentation of frames and selects only areas in which people are located.
The scheme of the algorithm
After that, the algorithm starts working only with these areas. First, it calculates the person's height, taking into account the perspective in different frames, and calculates the average. After that, he breaks the body of the remaining people into three parts and determines the color of the clothes on the torso, and he determines two colors - the main and the additional one, which is used when there are two or more people with a similar main color. If these parameters were not enough, the algorithm can also filter the remaining people by gender.
For training, the developers used the COCO and SoftBioSearch datasets. The images were randomly adjusted from −5 to 5 degrees in order to improve the performance of the trained model. In addition, the authors of the work modified the lighting on the frames for better color determination by the algorithm. To search for a specific person, the developers provided the neural network with marked frames with him. The algorithm correctly recognized 28 out of 41 people, for 19 of them the proportion of frames with correct recognition was more than 60 percent.
Recently, another group of developers from India and the UK have created a drone-based system that can detect violent activities among people, such as fights. It also works with a machine learning algorithm that has been trained to identify violent postures.
