A person can be perfectly visible to the naked eye and, at the same time, become almost “invisible” for a camera equipped with artificial intelligence. This is the effect sought by Bill Swearingen, an American cybersecurity expert who created noRecognition, a project that aims to create motifs printed on clothes and objects with geometric designs capable, in certain cases, of confusing the artificial vision systems used in video surveillance.
How noRecognition technology works that deceives facial recognition
To understand how it is possible to deceive a recognition system, however, we must start from a basic concept by briefly explaining what the difference is between detection and recognition. An AI system, using a camera, can first of all try to establish whether there is a person in the image or not by detecting human presence; subsequently it can identify the face of the detected subject and, only then, compare it with a database to try to understand who it is. Facial recognition, therefore, is only one link in a much, much longer chain.
At the Def Con conference in Las Vegas, an annual event dedicated to hackers, Swearingen showed just how effective blocking the first “ring” can be. When in front of a camera, the system indicated with a probability greater than 0.75 that there was a person in the frame. When Swearingen held up a panel covered in a strange black and white pattern in front of his body, the score progressively dropped to 0.21. At that point the software stopped considering the image as containing a person.
But through it all Swearingen had never moved from the frame at all. Very simply he had managed to deceive the automatic recognition system, which had lost the confidence necessary to classify what he saw as a human being. These algorithms, in fact, do not “see” as we see: they analyze an image, formulate various hypotheses about what it contains, assign a confidence score and only report those that exceed a certain threshold. If that threshold is not reached, the detection does not start.
How textures and patterns were produced that make us invisible to AI
Swearingen didn’t design the anti-facial detection patterns to appeal to his imagination, of course. Rather, it has developed an automatic system that generates these motifs using ad hoc software. The approach used is reminiscent of that of the so-called “fuzzers”, those programs used in computer security to subject software to an enormous amount of inputs and identify errors and anomalous behavior. In this case, the target to hit was computer vision systems.
The program creates a pattern, digitally overlays it on an image of a person, and tries to understand how the various AI systems involved react. If the reason significantly decreases the detection score and causes anomalous results, it is selected for further testing. The most effective models are then modified and combined to generate new ones.
Keep in mind that the system ended up comparing each new pattern with 11 different patterns: five dedicated to detecting people, four to finding faces and two to recognizing faces. Swearingen said that as of June, 31.7 million tests had been performed and only a small portion of these (to give you a number, around 534,600) had produced significant anomalies. Of all these, only 85 results were classified as “extreme”, therefore capable of defeating at least one person detector and one face detector in the same test.
The limits of anti-detection technology
However, the numbers show some still evident limitations in this technology, which is far from infallible. Also because it has become clear that an effective pattern does not necessarily work on every person or face. A model that had lowered the confidence level for four people used during development did not work once on the eight “guinea pigs” it was subsequently tested with. This is because the effectiveness and success of the test can depend both on the person and on the detection system used.
The location of the motif also matters. It was found that a larger surface does not automatically mean a better result: patterns on the torso had a greater effect on people detectors, while those close to the head had a greater effect on face detection systems. In some tests, a small pattern on the collar even proved more effective than a much larger print on the torso.
Then there is a fundamental difference between the demonstration and a real t-shirt worn on the street. The test done at Def Con was done with the pattern on hardboard, not fabric. As a t-shirt bends, stretches and creases as it is worn, it is exposed to constant distortions and changes in lighting, which do not replicate the ideal conditions seen during the demonstration. For this reason, Swearingen still considers the effectiveness of the garment worn to be unproven.
