In general, fraud is the common painful area in the telecom sector, and detecting fraud is like finding a needle in the haystack due to volume and velocity of data. There are 2 key factors to detect fraud: (1). Speed: If you can’t detect in time, you’re doomed to loose because they’ve already got what they need. Simbox detection is one of the use case for this situation. Frauders use it to bypass interconnection fee. In this use case we’re talking about our real time architecture using Spark SQL to detect simbox within 5 minutes. (2). Accuracy: Frauders changes their method all the time. But our job is finding their behaviour using machine learning algorithms accurately. Anomaly detection is one of the use case for this situation. In this use case we’re talking about data mining architecture to make fraud models using Spark ML within 1 hour. We also discuss some ML algorithm performance on Spark such as K-means, three sigma rule, T-digest and so on. In order to accomplish these factors, we processes 8-10 billion records which size is 4-5 TB every day. Our solution combines end-to-end data ingestion, processing, and mining the high volume data to detect some use cases of fraud in near real time using CDR and IPTDR to save millions, and better user experience.

1. Burak IŞIKLI
Near Real-Time
Fraud Detection in
Telecommunication
Industry

2. Who am I?
Burak IŞIKLI
Senior Software Engineer, Turkcell
burak.isikli at turkcell dot com.tr
@burakisikli, github.com/burakisikli

3. Turkcell
• It is the first and only Turkish company ever to be listed on the NYSE.

4. Fraud Overview
• Roaming Fraud – $6.11B
• Premium Rate Service Fraud – $4.73B
• Abuse or Arbitrage Fraud – $2.2B
• International Revenue Share Fraud (IRSF)- $1.8B
• Domestic Revenue Share Fraud (DRSF) – $0.8B
• Call and SMS Spamming – $0.8B
• Subscription Fraud – $5.2B
• Wangiri Fraud – $2B annually
• SMS Phishing/Pharming – $1.7B
• (V)PBX Fraud – $4.42B

5. Our Fraud Journey
• Wangiri
• Anomaly Detection
• Simbox

6. Wangiri

7. Wangiri
• Wangiri (One ring-cut)
• Premium services
• Both local and
international services

8. Wangiri
Problem: Someone
makes a call, then
subscriber will be curious
about it and calls back but
he/she didn’t know it’s a
premium service so that
next bill become
expensive which is
unexpected for subscriber

9. Wangiri
Solution: Design a system that detect these numbers
to react
• There’s no historical data so it’s an analytical
solution
• Python + Spark = Pyspark
• Send an email to alert

10. Wangiri
• Process CDRs in near real time(nrt) using Spark
• Generate features using these data in NRT
• 1TB data/day
• Wangiri Detection Rate > %50

11. Anomaly Detection

12. Anomaly Detection
Problem: Identify the anomalies of the international
calls and determine if there is a situation where action
should be taken

13. Solution: Designing a system that can detect
unexpected changes. Whenever there is a change,
produce an alarm so that business can quickly examine
and take action.
Anomaly Detection

14. Lambda Architecture
Anomaly Detection - Architecture

15. • 68–95–99.7 (three sigma) Rule
• K-means clustering
• One-class SVM
• Seasonal Hybrid Extreme Studentized Deviates (S-
H-ESD)
• Holt-Winters Method
• Deep Learning
Anomaly Detection - Methods

16. 68-95-99.7 Rule
• Data should be distributed properly

17. 68-95-99.7 Rule
• But data changes in time…

18. 68-95-99.7 Rule
• Implemented in Spark and SQL
• Too much false positive alarm
• Current data should be more weighted than
historical data
• Not enough data to explain anomaly

19. One-Class SVM
• Clustering
• All data in one class
• Too much false positive so not good enough

20. One-Class SVM
• Using Oracle Data Miner(ODM) easier to
implementation

21. One-Class SVM
• Explain how?

22. K-Means Clustering
• Number of cluster is predefined
• It’s impractical to define number of cluster for each
country

23. S-H-ESD
• Global(Seasonally) and Local(trend) Analysis
• R Library Open sourced by Twitter
• Prototype using R
• Ported to python to get production-ready version
• Easier to explain, successful results

24. S-H-ESD
• R Prototype

25. S-H-ESD
• Python Prototype

26. Simbox

27. Simbox
+90 5321112233
Ayşe
+49 5321234567
Ali
Ali
Ayşe
Turkcell gets 0,050 €

28. Simbox
+90 5321112233
Ayşe
+90 5321111111
Ali
Ali
Ayşe
Turkcell gets 0,01 €

29. Simbox - Methods
• Analytical
• Machine Learning
✓ Supervised Learning
✓ Crime Ring Detection (In Progress)

30. Simbox - Machine Learning
• We have simbox numbers that have been blocked
before and their activities(CDR)
• Supervised Learning!

31. Simbox - Machine Learning
• ROC = 0.99
• Good enough?
• What about other metrics such as precision,
recall, confusion matrix and so on?

32. Simbox - Machine Learning
• ROC = 0.99
• Rookie Mistake
▪ 0 predicted simbox!
• Imbalanced data

33. Imbalanced Data

34. Simbox - Machine Learning
• Imbalanced Data
1. Under sampling
2. Oversampling
3. SMOTE(Synthetic Minority Over-sampling
Technique)
4. Different algorithms
▪ Decision Tree, Random Forest
5. Penalized Models
▪ Penalized-SVM, Penalized-LDA

35. Simbox - Machine Learning
Spark:
• Under sampling
▪ Filter&union or sample
• Oversampling
▪ sample(True(withReplacement),
67000.0(fraction), 1234(seed))
• Different algorithms
▪ Decision Tree, Random Forest
• Other models in progress

36. Simbox - Machine Learning
Results:
Accuracy = %88.95
F1 = %88.91
AUROC = %79.87
AUPR = %82.23

37. Simbox - Machine Learning
Results:
• Oversampling or under sampling doesn’t change
the accuracy so much
• Random forest and decision tree handles
imbalanced data set
• The accuracy can be improved

38. Thank You!