DEEPAKE

1. Research Scholar Supervisor
Name: S. ABDUL YUNUS BASHA Dr. K. Ulagapriya
UP id: UP23P9570001 Department of CSE
Department of CSE VISTAS
VISTAS
Block Secure Net: A Block chain-Enhanced Deep
Learning Framework for Robust Multimedia
Authentication and Deep fake Detection

2. Block Secure Net : Deep fake detection
6-Feb-24
2
Way to know what's factual : Detect and Compare

3. Abstract
 The rapid evolution of deepfake tech poses serious threats like
misinformation and privacy breaches, eroding trust in digital media.
Generated by advanced deep learning, deepfakes seamlessly blend one
person's likeness onto another's, challenging conventional detection
methods. Traditional rule-based approaches struggle with the continuous
refinement of deepfake techniques. There's a critical need for advanced
methods to identify and mitigate the proliferation of manipulated
multimedia content, addressing the pressing issues of today's digital
landscape.
 To address these challenges, this research proposes Blockchain integration,
assigning a unique identifier (hash) to each content recorded on the
blockchain for a decentralized and tamper-resistant ledger.
 Custom Convolutional Neural Networks (CCNNs), RNN, Additive Attention
Mechanism and Two-Stream Convolutional Networks enhances deep
learning analysis by considering multimedia watermark , This improves
accuracy in binary classification with added confidence based on watermark
validation and audio pattern analysis. The research introduces
BlockSecureNet, a groundbreaking approach that combines blockchain and
deep learning to combat deepfake manipulation.
6-Feb-24
3

4. Abstract
 BlockSecureNet uses blockchain for content authentication via smart
contracts and decentralized nodes. Deep learning identifies temporal
inconsistencies in multimedia, linked to blockchain through verification
anchors for transparency. Targeting deepfake concerns, it offers a solution
for data integrity, decentralization, and model updates, ensuring reliable
deepfake detection.
 The method uses metrics like true positive rate, true negative rate, and AUC,
employing the Unified Deep Learning Algorithm with Custom CNNs, RNN,
LSTM, Two-Stream CNNs, and Blockchain Integration for Multimedia
Authentication Detection.
 It offers a holistic approach to combat deepfake manipulation, considering
blockchain for content authentication and deep learning for temporal
analysis. Block Secure Net effectively addresses challenges in detecting and
mitigating the impact of deepfake content on society.
6-Feb-24
4

5. Introduction
 Deepfakes, also known as synthetic media, are generated using deep learning
techniques to create realistic and deceptive audio, video, and images. These
deepfakes can be used to impersonate individuals, manipulate conversations,
and spread misinformation. The potential for misuse of deepfakes poses a
serious threat to society, as they can be used to damage reputations,
undermine trust in institutions, and even incite violence.
 Various techniques have been proposed for detecting deepfakes. Traditional
methods often rely on hand-crafted features and statistical analysis, which
are limited in their effectiveness against increasingly sophisticated deepfake
technologies. Deep learning-based approaches have emerged as a promising
alternative, demonstrating superior performance in identifying deepfake
 To stay on top of these challenges, it's important to use a mix of advanced
technologies, including Deep Learning Models, machine learning, Audio
Analysis, and Blockchain. This combination of tools helps us keep up with the
always-changing tricks that deepfake creators use.
. 6-Feb-24
5

6. Sample Image 1
A sample of videos from Google’s contribution to the Face Forensics benchmark. To generate these,
pairs of actors were selected randomly and deep neural networks swapped the face of one actor onto
the head of another. 6

7. Sample Image 2
6-Feb-24
7
Actors were filmed in a variety of scenes. Some of these actors are pictured here
(top) with an example deepfake (bottom), which can be a subtle or drastic change,
depending on the other actor used to create them.

8. Sample Videos
 This video explains how Deepfakes are created by using artificial
intelligence.
 Click here to view the video
6-Feb-24
8

9. Problem Statement
 The rapid advancement of artificial intelligence (AI) and machine learning (ML)
techniques has enabled the creation of highly realistic deep fakes, which are
manipulated or fabricated images, videos or audio recordings that appear to show
people saying or doing things they never actually said or did
 Existing deepfake detection methods, such as those based on pixel analysis or audio
waveform analysis, are becoming increasingly ineffective as deepfake technology
advances. This necessitates the development of more robust and sophisticated by
implementing Block Secure Net (deepfake detection methods) that incorporate
blockchain with embedded watermarking, audio analysis, and deep learning
techniques
 Fabricated images, videos or audio recordings are very difficult to identify so there
is a need to create a embedded watermark on video ,image using block chain &
Deep learning
 Development of accurate and efficient deep fake detection models specifically
tailored for audio content which includes identifying synthesized voices,
recognizing anomalies in speech patterns, and distinguishing genuine recordings
from manipulated ones
6-Feb-24
9

10. Objective of my research work
The intention of my research work Block SecureNet is described as follows:
1. Deep fake Detection with Deep learning
A deep learning model , rigorously trained on diverse datasets to identify visual and audio patterns
characteristics of deep fakes ,would analyze videos promptly upon upload or access
2. Watermark Generation & Embedding
 Upon a high probability of a video being a deep fake a water mark generation process would
initiated .The watermarks would be designed to be invisible .
 The watermark would contain “Potential Deepfake Label”, Date and Time of creation, Hash of
Original video, Link to block chain based verification system.
3. Block Chain Integration for Integrity and Transparency
 Distinguishing between authentic and manipulated content we Utilize embedded Watermark The
cryptographic hash functions (e.g., SHA-256) are used to generate a unique hash value for each
multimedia file, & records Hash on the Blockchain
 Implement smart contracts that automate the verification process. Smart contracts can be
programmed to trigger alerts or take specific actions to Check verification nodes independently
which in turn verifies the integrity of multimedia files by recomputing the hash locally and
comparing it with the hash stored on the blockchain
4. Deep Learning based watermark detection & verification
 Use of hyper parameters, training data, transfer learning, ensemble learning, and data
augmentation, is possible to tailor custom CNNs and RNNs to achieve specific tasks for deep fake
detection.
 Short-time Fourier transform (STFT) , wavelet transform is a mathematical function is used in a
variety of applications, including audio signal processing, image compression, and signal denoising.
6-Feb-24
10

11. Literature survey
6-Feb-24
11
Re
f.
no
Year Techniques Objective Limitations Application Algorithm
used
1. 2018 Liveness
detection
Detect deep
fakes by
analyzing facial
movements
and
expressions
Relies on
high-
quality
video
input
Social media
platforms,
security
systems
Support
Vector
Machines
(SVM)
2. 2019 Audio
deep fake
detection
Identify
manipulated
audio
recordings
Limited to
specific
audio
formats
Voice
assistants,
online
communication
Mel-
frequency
cepstral
coefficients
(MFCC)
3 2020 Facial
artifact
detection
Spot
inconsistencies
in facial
features and
skin texture
Requires
training on
a large
dataset of
deepfakes
E-commerce,
online dating
Convolution
al neural
networks
(CNNs)

12. Literature survey
6-Feb-24
12
Re
f.
no
Year Techniques Objective Limitations Application Algorithm
used
4. 2021 Visual
anomaly
detection
Uncover
abnormalitie
s in video
frames
Sensitive to
noise and video
compression
Video
surveillance,
news
verification
Auto
encoders
5. 2022 Multi-
modal deep
fake
detection
Combine
multiple
detection
techniques
for improved
accuracy
Computationally
expensive
Forensic
analysis, law
enforcement
Ensemble
methods

13. Modules
6-Feb-24
13
: Module 1 :- Data Acquisition and Pre processing
 This module focuses on acquiring and preparing the multimedia data for subsequent
analysis it performs Data gathering, Data Labelling, Data Pre processing.
 Preprocessed and normalized multimedia data suitable for feature extraction and
model training
 Statistical methods, outlier detection, noise reduction algorithms used for Data
Cleaning Techniques,
 Min-max scaling, z-score normalization, are used for Data Normalization Methods
Module 2: Block chain Integration for Tamper-Proof Authentication
 This module introduces block chain technology to provide a robust & decentralized
authentication mechanism. It involves tasks such as Blockchain Network Setup,
Content Hashing & Blockchain Embedding. Tamper-proof and verifiable records of
content authenticity stored on the blockchain.
 Algorithms and Techniques:
Blockchain Protocols: Proof-of-Work (PoW), Proof-of-Stake (PoS), Byzantine
Fault Tolerance (BFT) algorithms.
Cryptographic Hashing Functions: SHA-256, hash functions.
Combination of Solidity programming language & Ethereum Virtual Machine (EVM)
for blockchain interactions is used for Smart Contract Development

14. Modules
6-Feb-24
14
Module 3: Deep Learning-Based Embedded Watermarking
 This module utilizes deep learning techniques to embed watermarks into multimedia
content for enhanced security to create Watermark Generation, Embedding, Extraction
 Watermarks are resistant to various content manipulations and attacks.
 Convolutional Neural Networks (CNNs), Generative Adversarial Networks (GANs) ,
singular value decomposition (SVD) Algorithms and Techniques are deep learning-
based extraction used in this module.
Module 4: Attention-Based Deep fake Detection
 This module employs deep learning models with attention mechanisms to detect deep
fakes tasks such as Deep fake Model Training, Classification & Integration will be
implemented here.
 Deep fake architectures such as Inception-v3 models used for image classification
tasks
 additive attention mechanism are used to identify subtle inconsistencies and anomalies
in manipulated audio.
Module 5: Performance Evaluation and Result Analysis
 This module focuses on evaluating the performance of the proposed framework and
analyzing the results.
 Performance Metrics can be calculated by Accuracy, precision, recall, F1-score.

15. Timeline of Deepfake Technology from 2018-2023

16. Taxonomy of Deepfake detection techniques
6-Feb-24
16
The above taxonomy classifies the detection algorithms according to the media (image, video, or image and
video), the features used (among the 12 features), the detection method (DL, ML, Blockchain, or
statistical), and the clue for the detection (facial manipulation of digital media forensics, or other
indications).

17. Techniques applied
6-Feb-24
17
Ref.
no
Year Techniques Algorithm Accuracy
1 2018 MesoNet Convolutional neural network (CNN) 0.92
2 2019 FaceForensics++ CNN and long short-term memory (LSTM) 0.94
3 2020 DFaker Generative adversarial network (GAN) 0.95
4 2021 StyleGAN2 GAN 0.96
5 2022 Imagen Video Transformer 0.97
6 2023 Block Secure Net
BlockChain, GAN, CCNN, Additive Attention
Mechanism
Estimnated (from
97% up to 99%)
The Deepfake is detected by using different Deep learning algorithms by various
researchers.
 The following table shows the technique and algorithm from 2018-2023.

18. System Overview
 Nowadays, information is increasing fast, also availability of processing data
exceed human abilities.
Block chain technology provides tamper-proof data storage and integrity
assurance, safeguarding multimedia content from unauthorized modifications.
 Combines block chain and deep learning for bulletproof multimedia
authentication and deep fake detection.
Hashes multimedia features onto a secure block chain, creating an immutable
record of authenticity.
Deep learning model sniffs out deep fakes based on inconsistencies in the
block chain record.
Tracks the origin and history of any content, exposing manipulation attempts.
Decentralized storage and access control prevent unauthorized modifications.
Empowers users to trust what they see and fight the spread of misinformation.
 Ongoing research holds promise for further advancements, including enhanced
deep learning model performance, optimized consensus mechanisms, real-time
implementation, and robust privacy considerations.
Steps for Deepfake Detection

19. Proposed workflow
The workflow of our proposed model is described as follows:
 In Phase 1: Collect a diverse dataset of multimedia content & Preprocess the
multimedia content ,
 In phase 2 : Develop deep learning models for feature extraction and anomaly
detection & Integrate block chain technology
 In phase 3 : Multimedia Authentication and Deep fake Detection.
 In Phase 4 : Continuous Monitoring and Improvement , enhance the
performance of the deep learning framework
 Finally based on accuracy measures, performance of Deep learning models are
evaluated.
6-Feb-24
19
Acquisition
diverse dataset
& Preprocess
Design and train deep
learning models &
Integrate blockchain
technology
authenticate
multimedia
content and
detect deepfakes.
Monitoring &
Improvement
Application to test
data(Accuracy,
Performance,
Epochs)

20. Deep Fake Detection : Tactics, and Detection by
Block Secure Net
6-Feb-24
20
1
• Collect and Pre process Data
• authentic and manipulate multimedia content for analysis
2
• Embed Watermarks
• extract features from both watermarked authentic content and target
content for comparison.
3
• Develop Deep Learning Models
• authentic multimedia content to serve as unique identifiers
• Analyze and Detect Deep fakes
4
• identify anomalies or inconsistencies indicative of deepfakes.
5
• Verify Authenticity and Store Results

21. Expected outcomes
 BlockSecureNet comprises, Improved Deepfake Detection, Traceable
Authentication History, Resilient to Manipulation & Trustworthy Multimedia
Content
 Metrics such as precision, recall, F1-score, and accuracy are evaluated to
predict Deep learning classification model performance.
 Comprehensive evaluation of the proposed framework's performance in terms
of authentication accuracy, receiver operating characteristic (ROC) curve, area
under the ROC curve (AUC) & computational efficiency leads to high rated
Performance Evaluation
 Analysis of the effectiveness of each module and its contribution leads to the
overall framework's performance.
6-Feb-24
21

22. Work plan
6-Feb-24
22

23. Conclusion
 The proposed blockchain-enhanced deep learning framework
effectively safeguards multimedia content against manipulation and
deepfakes by leveraging the strengths of both blockchain technology
and deep learning techniques. The framework ensures the authenticity
and reliability of multimedia content by providing a tamper-proof and
transparent record of multimedia content, coupled with the ability to
accurately detect deepfakes, even those that are highly sophisticated.
This framework has the potential to significantly improve the security
and reliability of multimedia content.
Hence, this research work focused on detecting specific attack
based on BlockSecure Net along with Deep based CNN optimization
GAN, Additive Attention Mechanism which enhances deep fake
detection.
 In my next DC meeting, I will take up this topic and show the results.

24. References
1. G. Oberoi. Exploring DeepFakes. Accessed: Jan. 4, 2021. [Online].
2. Available: https://goberoi.com/exploring-deepfakes-20c9947c22d9
3. J. Hui. How Deep Learning Fakes Videos (Deepfake) and How to Detect it. Accessed: Jan. 4, 2021.
[Online]. Available: https://medium. com/how-deep-learning-fakes-videos-deepfakes-and-how-to-
detect-it- c0b50fbf7cb9
4. I. Goodfellow, J. P. Abadie, M. Mirza, B. Xu, D. W. Farley, S. Ozair,
5. A. Courville, and Y. Bengio, ‘‘Generative adversarial nets,’’ in Proc. 27th Int. Conf. Neural Inf. Process.
Syst. (NIPS), vol. 2. Cambridge, MA, USA: MIT Press, 2014, pp. 2672–2680.
6. G. Patrini, F. Cavalli, and H. Ajder, ‘‘The state of deepfakes: Reality under attack,’’ Deeptrace B.V.,
Amsterdam, The Nether- lands, Annu. Rep. v.2.3., 2018. [Online]. Available: https://s3.eu-west-
2.amazonaws.com/rep2018/2018-the-state-of-deepfakes.pdf.
7. H. Hasan and K. Salah, ‘‘Combating deepfake videos using blockchain and smart contracts,’’ IEEE
Access, vol. 7, pp. 41596–41606, 2019, doi: 10.1109/ACCESS.2019.2905689.
8. IPFS Powers the Distributed Web. Accessed: Jun. 5, 2020. [Online].
9. Available: https://ipfs.io/
10. C. C. Ki Chan, V. Kumar, S. Delaney, and M. Gochoo, ‘‘Combating deepfakes: Multi-LSTM and
blockchain as proof of authenticity for digital media,’’ in Proc. IEEE/ITU Int. Conf. Artif. Intell. Good
(AI4G), Sep. 2020, pp. 55–62.
11. J. Li, T. Shen, W. Zhang, H. Ren, D. Zeng, and T. Mei, ‘‘Zooming into face forensics: A pixel-level
analysis,’’ 2019, arXiv:1912.05790.
12. T. Thi Nguyen, Q. Viet Hung Nguyen, D. Tien Nguyen, D. Thanh Nguyen, T. Huynh-The, S. Nahavandi,
T. Tam Nguyen, Q.-V. Pham, and
13. C. M. Nguyen, ‘‘Deep learning for deepfakes creation and detection: A survey,’’ 2019,
arXiv:1909.11573.
14. R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and
15. J. Ortega-Garcia, ‘‘Deepfakes and beyond: A survey of face manipulation and fake detection,’’ Inf.
Fusion, vol. 64, pp. 131–148, Dec. 2020, doi: 10.1016/j.inffus.2020.06.014.