Multimedia Forensics: From Image manipulation to the Deep Fake.New Threats in the Social Media Era.

This dissertation reports the research activities carried on during the Ph.D. program in Computer Science I attended at the University of Catania – Departments of Mathematics and Computer Science. My research area is mainly concerned with Social and Multimedia Forensics, starting from a survey and a forensic analysis conduct on 250 Social Network, passing through a detection of false multimedia content and manipulated images (focusing particularly on the double compressed JPEG images), until arriving to the emerging phenomenon of the DeepFakes. Nowadays the social media ecosystem includes hundreds of applications, based on web and mobile technologies and able to allow people to communicate easily and/or to share information, resources, images and videos. However it also hides a serious risk of disclosure of personal data that can lead to privacy issues and crimes. So it is very important to study social network applications also from a forensic point of view. This work aims to explore the Social Media ecosystems formed by 250 web and mobile applications, analyzing everyone to exploit and provide a complete survey on the forensic methods that can be applied to recover useful information (ID user, ID post, . . . ) to be used in front of a court. In the second part of our research, we focus the attention on the multimedia content and particularly on JPEG images. One of the most common problem in the image forensic field is the reconstruction of the history of an image. Image Forensics analysis is the process that aims to understand the history of a digital image. In the last ten years Image Forensics, whose goal is exploit the knowledge from the science of Image Processing to answer questions that arise in the forensic scenario, has developed at a growing rhythm in order to efficiently check the originality of images and videos from the all-day context. The reconstruction data of the further processing are very important information that can help us to have some useful hints about the originality of the image under analysis. If an image has been subjected to more than one JPEG compression the considered image is not the exact bitstream generated by the camera at the time of shooting. The originality of an image can be established by recovering coefficients of the JPEG quantization matrix used to compress an image at the time of shooting (i.e., when the image has been created), when, for some reasons, this information is no more available in the Exif metadata. This scenario may include the primary quantization coefficients of an image that has been doubly JPEG compressed, or the retrieval of the compression matrix of an uncompressed image previously JPEG compressed, since in both these cases the values of the primary compression steps are lost. So it is clear that Double Quantization Detection (DQD) and Quantization Step Estimation (QSE) are two fundamental steps in the overall process. This research deals with also the Double JPEG compression detection algorithm. Starting from the paper of Galvan et al in this work is presented a Matlab implementation of the algorithm by adopting a different approach based on a block splitting methodologies and the results of this algorithm applied on different contexts: high resolution images, minor block number… This PhD thesis investigate finally one of the most terrifying emerging phenomenon in the digital world: the Deepfake. It presents a brief overview of the technologies able to automatically replace a person’s face in images and videos by exploiting algorithm based on deep learning and also several techniques of creation and detection of the so-called Deepfakes with the related social and legal problems. A forensic analysis of those images with standard method will be presented and will show that not surprisingly state of the art techniques are not completely able to detect the fakeness. It is very important to be able to counter this phenomenon by creating new forensic methods. Moreover, this work shows an idea on how to fight Deepfake images by analyzing anomalies in the frequency domain and evaluating details and traces of underlying generation process of the image (e.g. in the Fourier domain). This Phd thesis provides a good study of the state of the art in Social and Multimedia Forensics and shows some results obtained through an in-depth analysis.