Presentation of the InVID tools for image forensics analysis, at the Media Informatics Lab meeting on detection and verification of socially shared videos.

1. Detecting malicious tampering in
digital images
Markos Zampoglou - markzampoglou@iti.gr
Information Technologies Institute (ITI)
Centre for Research and Technology Hellas (CERTH)
Workshop on Tools for Video Discovery & Verification in Social Media
Dec 14, 2017 @ Thessaloniki, Greece

2. Image forgeries in the news
https://www.snopes.com/photos/animals/puertorico.asp

3. Image forgeries in the news
http://www.imperfectedblog.com/2015/11/crimes-of-retouching-and-the-
importance-of-community/

4. Image forgeries in the news
https://thelede.blogs.nytimes.com/2008/07/10/in-an-iranian-image-a-missile-
too-many/

5. The promise of image forensics

6. Image verification: tools of the trade
• Metadata analysis
• Photoshop? Dates/locations? Copyrights?
• Reverse image search
• Content analysis / tampering localization

7. Tampering localization algorithms
Splicing
localization
Copy-move
detection
Block
matching
Keypoint
matching
High-frequency
noise
CFA
patterns
JPEG compression
traces
Filtering and
analysis
Camera-
specific (PRNU)
Quantization
artefacts
Compression grid
misalignment
JPEG ghosts,
ELA

8. REVEAL Media Verification Assistant (1/2)
http://reveal-mklab.iti.gr/
• REVEAL (FP7, 2013-2016)
• Second round of testing as part of InVID
• Collaboration between CERTH and DW
• Features:
• Metadata: listing, GPS geolocation, EXIF thumbnails
• Reverse image search: Google Images integration
• Tampering detection: Eight state-of-the-art algorithms
• Aims:
• a comprehensive, self-contained verification toolset
• an evaluation framework for verification tools and protocols

9. REVEAL Media Verification Assistant (2/2)
Zampoglou, M., Papadopoulos, S., Kompatsiaris, Y., Bouwmeester, R., & Spangenberg, J. (2016, April). Web
and Social Media Image Forensics for News Professionals. In SMN@ ICWSM.

10. Comparison of verification tools
FotoForensics1 Forensically2 Ghiro3 REVEAL4
ELA X X X X
Ghost X
DW Noise X
Median Noise X X
Block Artifact X
Double Quantization X
Copy-move X*
Thumbnail X X
Metadata X X X X
Geotagging X X X X
Reverse search X
1http://fotoforensics.com
2http://29a.ch/photo-forensics/
3http://www.imageforensic.org/
4http://reveal-mklab.iti.gr/reveal/verify.html
*Forensically implements a very simple block-
matching algorithm with low robustness

11. Further features
• Open source1
• Public demo version
• Real-world examples of detections and non-
detections
• Tutorials and algorithm explanations
• “Magnifying glass” for examining fine details
• Scrolling image/map overlay
• Export personalized analysis as PDF
1https://github.com/MKLab-ITI/image-forensics

12. Usage distribution
France 401
Netherlands 262
Germany 214
UK 181
US 153
Argentina 96
Egypt 52

13. Current challenges (1/2)
• Absence of automatic methods to analyse metadata
• High computational requirements
• Interpretation of results not always intuitive

14. Current challenges (2/2)
• When applied in real-world cases, algorithms
perform significantly worse than in research
datasets
• This could partly be attributed to the effects of Social
Media dissemination

15. Identifying Untampered Images (1/2)
• Interpreting the results (especially for non-
detections) can be an issue
Untampered:
Tampered:
Algorithm: ADQ1 (Lin et al, 2009)

16. Identifying Untampered Images (2/2)
• Not all algorithms are created equal
• Still, some training is usually necessary
Untampered:
Tampered:
Algorithm: ADQ2 (Bianchi et al, 2011)

17. The pitfalls of image forensics
“MH17 – Forensic Analysis of Satellite Images
Released by the Russian Ministry of Defence”
• A report by Bellingcat1 using ELA to prove that the
Russian MoD had forged the images
• N. Krawetz: Bellingcat “misinterpreted the results”2
1https://www.bellingcat.com
2http://www.hackerfactor.com/blog/index.php?/archives/676-Continuing-Education.html

18. Future steps
• Add a tampering probability score
• Generate a “merged” tampering heat map
• Test new approaches based on deep learning
• Keep improving stability, speed, GUI responsiveness

19. Acknowledgements
• Dr. Chryssanthi Iakovidou (CAGI algorithm)
• Olga Papadopoulou (back-end development)
• Lazaros Apostolidis (front-end development)
• Dr. Symeon Papadopoulos (Overview & design)
• Ruben Bouwmeester, DW (Design and evaluation)
• Yiannis Kompatsiaris (Overview)

20. Thank you!
http://reveal-mklab.iti.gr/reveal
Get in touch!
Markos Zampoglou markzampoglou@iti.gr
Symeon Papadopoulos papadop@iti.gr

21. References
• Bianchi, Tiziano, Alessia De Rosa, and Alessandro Piva. "Improved DCT coefficient analysis
for forgery localization in JPEG images." In Acoustics, Speech and Signal Processing
(ICASSP), 2011 IEEE International Conference on, pp. 2444-2447. IEEE, 2011.
• Ferrara, Pasquale, Tiziano Bianchi, Alessia De Rosa, and Alessandro Piva. "Image forgery
localization via fine-grained analysis of cfa artifacts." Information Forensics and Security,
IEEE Transactions on 7, no. 5 (2012): 1566-1577.
• Farid, Hany. "Exposing digital forgeries from JPEG ghosts." Information Forensics and
Security, IEEE Transactions on 4, no. 1 (2009): 154-160.
• Fontani, Marco, Tiziano Bianchi, Alessia De Rosa, Alessandro Piva, and Mauro Barni. "A
framework for decision fusion in image forensics based on dempster–shafer theory of
evidence." Information Forensics and Security, IEEE Transactions on 8, no. 4 (2013): 593-
607.
• Lin, Zhouchen, Junfeng He, Xiaoou Tang, and Chi-Keung Tang. "Fast, automatic and fine-
grained tampered JPEG image detection via DCT coefficient analysis." Pattern Recognition
42, no. 11 (2009): 2492-2501.
• Mahdian, Babak and Stanislav Saic, “Using noise inconsistencies for blind image forensics,”
Image and Vision Computing, vol. 27, no. 10, pp. 1497–1503, 2009.
• Zampoglou, Markos, Symeon Papadopoulos, and Yiannis Kompatsiaris. "Detecting image
splicing in the wild (WEB)." In Multimedia & Expo Workshops (ICMEW), 2015 IEEE
International Conference on, pp. 1-6. IEEE, 2015.

22. Evaluation methodology
• Quantitative
• Six reference datasets (images + binary masks of tampering
= “ground truth”)
• Measures capturing the matching between ground truth
mask and algorithm output
• Comparison of 14 algorithms, “best” six plus a newly
proposed one ended up in the tool
• Qualitative
• Informal feedback has been received by end users
• Pertains to both usability and quality of results
Zampoglou, M., Papadopoulos, S., & Kompatsiaris, Y. (2017). Large-scale evaluation of splicing localization
algorithms for web images. Multimedia Tools and Applications, 76(4), 4801-4834.

23. Quantitative evaluations (1/3)
• To assess the performance of the implemented algorithms,
we evaluated their performance on a large number of
images
• Benchmark datasets:
• Columbia Uncompressed (Hsu and Chang 2006, “COLUMB”), First
Image Forensics Challenge (“CHAL”)1, Fontani et al 2013 (“FON”)
• Additional datasets
• The Wild Web Dataset (Zampoglou et al, 2015)
• The Deutsche Welle Image Forensics Dataset
1http://ifc.recod.ic.unicamp.br/fc

24. Quantitative evaluations (2/3)
• Evaluations on benchmark datasets:
• Robustness to resaves:

25. Quantitative evaluations (3/3)
• Performance on the Wild Web Dataset:
• Performance on the Deutsche Welle Image
Forensics Dataset:
• Successful detection for 2 out of 6 cases (4 out of 12
images)
Algorithm DQ GHO BLK ELA DWHF MED
Detections
(out of 80)
3 12 3 2 3 1
Time (sec) 0.27 6.12 13.40 1.29 122.13 0.54

26. Image verification software:
requirements
• Easy to use; requires little expertise and training
• Integrates easily in established workflows and IT
systems
• Provides all common functionalities
• Metadata analysis
• Reverse image search
• Tampering localization
• Clear, interpretable and transparent output
• No “black box” results
• Intuitive and simple interface
• Allows sharing and archiving of reports and
conclusions
• Comparable or better results than existing solutions