Discovering Fingerprints for Deepfake Detection and Multimedia-Enhanced Forensic Investigations

Forensic Science, which concerns the application of technical and scientific methods to justice, investigation and evidence discovery, has evolved over the years to the birth of several fields such as Multimedia Forensics, which involves the analysis of digital images, video and audio contents. Multimedia data was (and still is), altered using common editing tools such as Photoshop and GIMP. Rapid advances in Deep Learning have opened up the possibility of creating sophisticated algorithms capable of manipulating images, video and audio in a “simple” manner causing the emergence of a powerful yet frightening new phenomenon called deepfake: synthetic multimedia data created and/or altered using generative models. A great discovery made by forensic researchers over the years concerns the possibility of extracting a unique fingerprint that can determine the devices and software used to create the data itself. Unfortunately, extracting these traces turns out to be a complicated task. A fingerprint can be extracted not only in multimedia data in order to determine the devices used in the acquisition phase, or the social networks where the file was uploaded, or recently define the generative models used to create deepfakes, but, in general, this trace can be extracted from evidences recovered in a crime scene as shells or projectiles to determine the model of gun that have fired (Forensic Firearms Ballistics Comparison). Forensic Analysis of Handwritten Documents is another field of Forensic Science that can determine the authors of a manuscript by extracting a fingerprint defined by a careful analysis of the text style in the document. Developing new algorithms for Deepfake Detection, Forensic Firearms Ballistics Comparison, and Forensic Handwritten Document Analysis was the main focus of this Ph.D. thesis. These three macro areas of Forensic Science have a common element, namely a unique fingerprint present in the data itself that can be extracted in order to solve the various tasks. Therefore, for each of these topics a preliminary analysis will be performed and new detection techniques will be presented obtaining promising results in all these domains.