Fooling a Face Recognition System with a Marker-Free Label-Consistent Backdoor Attack

Modern face recognition systems are mostly based on deep learning models. These models need a large amount of data and high computational power to be trained. Often, a feature extraction network is pretrained on large datasets, and a classifier is finetuned on a smaller private dataset to recognise the identities from the features. Unfortunately deep learning models are exposed to malicious attacks both during training and inference phases. In backdoor attacks, the dataset used for training is poisoned by the attacker. A network trained with the poisoned dataset performs normally with generic data, but misbehave with some specific trigger data. These attacks are particularly difficult to detect, since the misbehaviour occurs only with the trigger images. In these paper we present a novel marker-free backdoor attack for face recognition systems. We generate a label-consistent poisoned dataset, where the poisoned images matches their labels and are difficult to spot by a quick visual inspection. The poisoned dataset is used to attack an Inception Resnet v1. We show that the network finetuned on the poisoned dataset is successfully fooled, identifying one of the author as a specific target identity.