SlideShare uma empresa Scribd logo

Local Outlier Factor

•Transferir como ODP, PDF•
•
AMR koura
AMR koura

This document describes Amr Koura's work in implementing and comparing batch and incremental modes of the Local Outlier Factor (LOF) algorithm. The goals were to code LOF in batch and incremental modes, integrate the code into an open source project, and compare the two modes. Incremental LOF was found to have equivalent outlier detection performance to static LOF while requiring less computation time and having lower computational complexity of O(N log N).

Educação
1 de 15
Data Mining Lab, Local Outlier Factor Amr Koura / Page 1 Supervisor: Sebastian Bothe Local Outlier FactorLocal Outlier Factor
Data Mining Lab, Local Outlier Factor Amr Koura / Page 2 Supervisor: Sebastian Bothe LabGoalLabGoal Implement Local Outlier factory Batch Mode. Implement Local Outlier factory Incremental Mode. Comparetwo modes. Integratecodeinto open sourceproject “RealKD”: https://bitbucket.org/realKD/
Data Mining Lab, Local Outlier Factor Amr Koura / Page 3 Supervisor: Sebastian Bothe MotivationMotivation http://www.dbs.ifi.lmu.de/Publikationen/Papers/LOF.pdf
Data Mining Lab, Local Outlier Factor Amr Koura / Page 4 Supervisor: Sebastian Bothe Local Outlier FactorLocal Outlier Factor reach−distk ( A , B)=max(d (B , A),k−distance(B)) lrd (A)= 1 ∑ B∈KNN (A) reach−distk ( A , B)/k LOF (A)= 1 k ∑ B∈KNN (A) lrd (B) lrd ( A) https://en.wikipedia.org/wiki/Local_outlier_factor
Data Mining Lab, Local Outlier Factor Amr Koura / Page 5 Supervisor: Sebastian Bothe DemoDemo
Data Mining Lab, Local Outlier Factor Amr Koura / Page 6 Supervisor: Sebastian Bothe
Data Mining Lab, Local Outlier Factor Amr Koura / Page 7 Supervisor: Sebastian Bothe Incremental Outlier FactorIncremental Outlier Factor Motivation: - infinitestream makesmemory constraints. - computational constraint for processing each stream item. Goal: - Sameperformanceasiterated static LOF algorithm. - efficient algorithm: insertion/Deletion should effect only limited number of nearest neighbours http://www-ai.cs.uni-dortmund.de/LEHRE/FACHPROJEKT/SS12/paper/outlier/pokrajac2007.pdf
Data Mining Lab, Local Outlier Factor Amr Koura / Page 8 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition Berlin;52.520;13.380 Hamburg;53.550;10.000 Munchen;48.140;11.580 Bonn;50.730;7.100 Koln;50.950;6.970 Frankfurt;50.120;8.680 Dortmund;51.510;7.480 Stuttgart;48.790;9.190 Essen;51.470;7.000 Cairo;30.3;31.14 Hurghada;27.15;33.50
Data Mining Lab, Local Outlier Factor Amr Koura / Page 9 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition
Data Mining Lab, Local Outlier Factor Amr Koura / Page 10 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition 1 1.1909475617292364 2 1.1956830856346556 3 0.9645631106850818 4 0.8029601477829005 5 0.7577540135599361 6 0.7377495644370516 7 0.7509608512974867 8 0.99956101138198 9 0.6943310060958396 10 3.7497548217312158 11 3.928514077815152 Now , lets add new Point="Alexandria;31.13;29.58"
Data Mining Lab, Local Outlier Factor Amr Koura / Page 11 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition 0 1 2 6 5 8 7 4 3 9 10 1 0 6 5 8 7 4 3 2 9 10 2 7 5 0 1 3 4 6 8 9 10 3 4 8 6 5 7 1 2 0 9 10 4 3 8 6 5 7 1 2 0 9 10 5 7 3 4 6 8 1 2 0 9 10 6 8 4 3 5 7 1 0 2 9 10 7 5 2 3 4 6 8 1 0 9 10 8 6 4 3 5 7 1 0 2 9 10 9 10 2 0 7 5 1 3 4 6 8 10 9 2 0 7 5 1 3 4 6 8 0 1 2 6 5 8 7 4 3 11 9 10 1 0 6 5 8 7 4 3 2 11 9 10 2 7 5 0 1 3 4 6 8 11 9 10 3 4 8 6 5 7 1 2 0 11 9 10 4 3 8 6 5 7 1 2 0 11 9 10 5 7 3 4 6 8 1 2 0 11 9 10 6 8 4 3 5 7 1 0 2 11 9 10 7 5 2 3 4 6 8 1 0 11 9 10 8 6 4 3 5 7 1 0 2 11 9 10 9 11 10 2 0 7 5 1 3 4 6 8 10 9 11 2 0 7 5 1 3 4 6 8 11 9 10 2 0 7 5 1 3 4 6 8
Data Mining Lab, Local Outlier Factor Amr Koura / Page 12 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition Cities9,10 haschangein their K-distance. According to: TheLRD for citiesexistsin K-NN of cities(9,10) should updated LRD List={9,10,2} According to , all citesthat hasany of cities {9,10,2} in their new nearest neighbour should updatethier LOF value. LOF List={9,10,2,0,7} lrd (A)= 1 ∑ B∈KNN (A) reach−distk ( A , B)/k LOF ( A)= 1 k ∑ B∈KNN ( A) lrd (B) lrd (A)
Data Mining Lab, Local Outlier Factor Amr Koura / Page 13 Supervisor: Sebastian Bothe Comparison between staticandincremental LOFComparison between staticandincremental LOF Running static LOF output: 1.1909475617292364 1.1956830856346556 0.9645631106850818 0.8029601477829005 0.7577540135599361 0.7377495644370516 0.7509608512974867 0.99956101138198 0.6943310060958396 2.3423102537190847 2.342310253719085 2.342310253719085 Running incremental LOF and addition output: 1.1909475617292364 1.1956830856346556 0.9645631106850818 0.8029601477829005 0.7577540135599361 0.7377495644370516 0.7509608512974867 0.99956101138198 0.6943310060958396 2.3423102537190847 2.342310253719085 2.342310253719085
Data Mining Lab, Local Outlier Factor Amr Koura / Page 14 Supervisor: Sebastian Bothe ConclusionConclusion Implementation of Batch incremental modehasdone. Batch modecodeisintegrated into theproject repository while pull request hasmadeto integrateit. Incremental LOF hasequivalent detection performanceas static LOF. Incremental LOF requireslesscomputation timethan time. Incremental LOF complexity isO(N log N)
Data Mining Lab, Local Outlier Factor Amr Koura / Page 15 Supervisor: Sebastian Bothe Thank you

Recomendados

Feature selection concepts and methods
Feature selection concepts and methodsFeature selection concepts and methods
Feature selection concepts and methodsReza Ramezani
 
Naive Bayes Presentation
Naive Bayes PresentationNaive Bayes Presentation
Naive Bayes PresentationMd. Enamul Haque Chowdhury
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature EngineeringHJ van Veen
 
Presentation on Text Classification
Presentation on Text ClassificationPresentation on Text Classification
Presentation on Text ClassificationSai Srinivas Kotni
 
Introduction to Linear Discriminant Analysis
Introduction to Linear Discriminant AnalysisIntroduction to Linear Discriminant Analysis
Introduction to Linear Discriminant AnalysisJaclyn Kokx
 
Introduction to Statistical Machine Learning
Introduction to Statistical Machine LearningIntroduction to Statistical Machine Learning
Introduction to Statistical Machine Learningmahutte
 
Text Classification/Categorization
Text Classification/CategorizationText Classification/Categorization
Text Classification/CategorizationOswal Abhishek
 
Dbscan algorithom
Dbscan algorithomDbscan algorithom
Dbscan algorithomMahbubur Rahman Shimul
 

Recomendados

Feature selection concepts and methods
Feature selection concepts and methodsFeature selection concepts and methods
Feature selection concepts and methodsReza Ramezani
 
Naive Bayes Presentation
Naive Bayes PresentationNaive Bayes Presentation
Naive Bayes PresentationMd. Enamul Haque Chowdhury
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature EngineeringHJ van Veen
 
Presentation on Text Classification
Presentation on Text ClassificationPresentation on Text Classification
Presentation on Text ClassificationSai Srinivas Kotni
 
Introduction to Linear Discriminant Analysis
Introduction to Linear Discriminant AnalysisIntroduction to Linear Discriminant Analysis
Introduction to Linear Discriminant AnalysisJaclyn Kokx
 
Introduction to Statistical Machine Learning
Introduction to Statistical Machine LearningIntroduction to Statistical Machine Learning
Introduction to Statistical Machine Learningmahutte
 
Text Classification/Categorization
Text Classification/CategorizationText Classification/Categorization
Text Classification/CategorizationOswal Abhishek
 
Dbscan algorithom
Dbscan algorithomDbscan algorithom
Dbscan algorithomMahbubur Rahman Shimul
 
Matrix factorization
Matrix factorizationMatrix factorization
Matrix factorizationLuis Serrano
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent methodProf. Neeta Awasthy
 
Beginners Guide to Non-Negative Matrix Factorization
Beginners Guide to Non-Negative Matrix FactorizationBeginners Guide to Non-Negative Matrix Factorization
Beginners Guide to Non-Negative Matrix FactorizationBenjamin Bengfort
 
DBSCAN : A Clustering Algorithm
DBSCAN : A Clustering AlgorithmDBSCAN : A Clustering Algorithm
DBSCAN : A Clustering AlgorithmPınar Yahşi
 
Text Classification
Text ClassificationText Classification
Text ClassificationRAX Automation Suite
 
Activation function
Activation functionActivation function
Activation functionAstha Jain
 
linear classification
linear classificationlinear classification
linear classificationnep_test_account
 
Linear discriminant analysis
Linear discriminant analysisLinear discriminant analysis
Linear discriminant analysisBangalore
 
NAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERNAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERKnoldus Inc.
 
Knn 160904075605-converted
Knn 160904075605-convertedKnn 160904075605-converted
Knn 160904075605-convertedrameswara reddy venkat
 
Introduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersIntroduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersFunctional Imperative
 
Wrapper feature selection method
Wrapper feature selection methodWrapper feature selection method
Wrapper feature selection methodAmir Razmjou
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent methodSanghyuk Chun
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep LearningSebastian Ruder
 
Chapter 09 classification advanced
Chapter 09 classification advancedChapter 09 classification advanced
Chapter 09 classification advancedHouw Liong The
 
Nonnegative Matrix Factorization
Nonnegative Matrix FactorizationNonnegative Matrix Factorization
Nonnegative Matrix FactorizationTatsuya Yokota
 
Support Vector machine
Support Vector machineSupport Vector machine
Support Vector machineAnandha L Ranganathan
 
Evaluation metrics: Precision, Recall, F-Measure, ROC
Evaluation metrics: Precision, Recall, F-Measure, ROCEvaluation metrics: Precision, Recall, F-Measure, ROC
Evaluation metrics: Precision, Recall, F-Measure, ROCBig Data Engineering, Faculty of Engineering, Dhurakij Pundit University
 
Classification
ClassificationClassification
ClassificationCloudxLab
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reductionmrizwan969
 
Data Mining: Outlier analysis
Data Mining: Outlier analysisData Mining: Outlier analysis
Data Mining: Outlier analysisDataminingTools Inc
 
Outliers
OutliersOutliers
OutliersNikesh Patra
 

Mais conteúdo relacionado

Mais procurados

Matrix factorization
Matrix factorizationMatrix factorization
Matrix factorizationLuis Serrano
 
Beginners Guide to Non-Negative Matrix Factorization
Beginners Guide to Non-Negative Matrix FactorizationBeginners Guide to Non-Negative Matrix Factorization
Beginners Guide to Non-Negative Matrix FactorizationBenjamin Bengfort
 
DBSCAN : A Clustering Algorithm
DBSCAN : A Clustering AlgorithmDBSCAN : A Clustering Algorithm
DBSCAN : A Clustering AlgorithmPınar Yahşi
 
Activation function
Activation functionActivation function
Activation functionAstha Jain
 
Linear discriminant analysis
Linear discriminant analysisLinear discriminant analysis
Linear discriminant analysisBangalore
 
NAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERNAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERKnoldus Inc.
 
Introduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersIntroduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersFunctional Imperative
 
Wrapper feature selection method
Wrapper feature selection methodWrapper feature selection method
Wrapper feature selection methodAmir Razmjou
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent methodSanghyuk Chun
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep LearningSebastian Ruder
 
Chapter 09 classification advanced
Chapter 09 classification advancedChapter 09 classification advanced
Chapter 09 classification advancedHouw Liong The
 
Nonnegative Matrix Factorization
Nonnegative Matrix FactorizationNonnegative Matrix Factorization
Nonnegative Matrix FactorizationTatsuya Yokota
 
Classification
ClassificationClassification
ClassificationCloudxLab
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reductionmrizwan969
 

Mais procurados (20)

Matrix factorization
Matrix factorizationMatrix factorization
Matrix factorization
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent method
 
Beginners Guide to Non-Negative Matrix Factorization
Beginners Guide to Non-Negative Matrix FactorizationBeginners Guide to Non-Negative Matrix Factorization
Beginners Guide to Non-Negative Matrix Factorization
 
DBSCAN : A Clustering Algorithm
DBSCAN : A Clustering AlgorithmDBSCAN : A Clustering Algorithm
DBSCAN : A Clustering Algorithm
 
Text Classification
Text ClassificationText Classification
Text Classification
 
Activation function
Activation functionActivation function
Activation function
 
linear classification
linear classificationlinear classification
linear classification
 
Linear discriminant analysis
Linear discriminant analysisLinear discriminant analysis
Linear discriminant analysis
 
NAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERNAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIER
 
Knn 160904075605-converted
Knn 160904075605-convertedKnn 160904075605-converted
Knn 160904075605-converted
 
Introduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersIntroduction to Machine Learning Classifiers
Introduction to Machine Learning Classifiers
 
Wrapper feature selection method
Wrapper feature selection methodWrapper feature selection method
Wrapper feature selection method
 
Gradient descent method
Gradient descent methodGradient descent method
Gradient descent method
 
Optimization for Deep Learning
Optimization for Deep LearningOptimization for Deep Learning
Optimization for Deep Learning
 
Chapter 09 classification advanced
Chapter 09 classification advancedChapter 09 classification advanced
Chapter 09 classification advanced
 
Nonnegative Matrix Factorization
Nonnegative Matrix FactorizationNonnegative Matrix Factorization
Nonnegative Matrix Factorization
 
Support Vector machine
Support Vector machineSupport Vector machine
Support Vector machine
 
Evaluation metrics: Precision, Recall, F-Measure, ROC
Evaluation metrics: Precision, Recall, F-Measure, ROCEvaluation metrics: Precision, Recall, F-Measure, ROC
Evaluation metrics: Precision, Recall, F-Measure, ROC
 
Classification
ClassificationClassification
Classification
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reduction
 

Destaque

3.7 outlier analysis
3.7 outlier analysis3.7 outlier analysis
3.7 outlier analysisKrish_ver2
 
Class Outlier Mining
Class Outlier MiningClass Outlier Mining
Class Outlier MiningMotaz Saad
 
Outliers -Story of Success by Malcolm Gladwell
Outliers -Story of Success by Malcolm Gladwell Outliers -Story of Success by Malcolm Gladwell
Outliers -Story of Success by Malcolm GladwellMa . Josefa Magbanua
 
"Outliers" - Malcolm Gladwell Book Review
"Outliers" - Malcolm Gladwell Book Review"Outliers" - Malcolm Gladwell Book Review
"Outliers" - Malcolm Gladwell Book ReviewArchit Rathi
 
Data Mining: Outlier analysis
Data Mining: Outlier analysisData Mining: Outlier analysis
Data Mining: Outlier analysisDatamining Tools
 
Anomaly detection Meetup Slides
Anomaly detection Meetup SlidesAnomaly detection Meetup Slides
Anomaly detection Meetup SlidesQuantUniversity
 
Multivariate data analysis
Multivariate data analysisMultivariate data analysis
Multivariate data analysisSetia Pramana
 
Data mining: Concepts and Techniques, Chapter12 outlier Analysis
Data mining: Concepts and Techniques, Chapter12 outlier Analysis Data mining: Concepts and Techniques, Chapter12 outlier Analysis
Data mining: Concepts and Techniques, Chapter12 outlier Analysis Salah Amean
 
Mining Methods
Mining MethodsMining Methods
Mining MethodsVR M
 

Destaque (20)

Data Mining: Outlier analysis
Data Mining: Outlier analysisData Mining: Outlier analysis
Data Mining: Outlier analysis
 
Outliers
OutliersOutliers
Outliers
 
Outliers
OutliersOutliers
Outliers
 
3.7 outlier analysis
3.7 outlier analysis3.7 outlier analysis
3.7 outlier analysis
 
Svm V SVC
Svm V SVCSvm V SVC
Svm V SVC
 
Depth based app
Depth based appDepth based app
Depth based app
 
Outliers
OutliersOutliers
Outliers
 
Class Outlier Mining
Class Outlier MiningClass Outlier Mining
Class Outlier Mining
 
Outliers, the story of success
Outliers, the story of successOutliers, the story of success
Outliers, the story of success
 
Outliers -Story of Success by Malcolm Gladwell
Outliers -Story of Success by Malcolm Gladwell Outliers -Story of Success by Malcolm Gladwell
Outliers -Story of Success by Malcolm Gladwell
 
"Outliers" - Malcolm Gladwell Book Review
"Outliers" - Malcolm Gladwell Book Review"Outliers" - Malcolm Gladwell Book Review
"Outliers" - Malcolm Gladwell Book Review
 
Outliers
OutliersOutliers
Outliers
 
Chapter 12 outlier
Chapter 12 outlierChapter 12 outlier
Chapter 12 outlier
 
Data Mining: Outlier analysis
Data Mining: Outlier analysisData Mining: Outlier analysis
Data Mining: Outlier analysis
 
Anomaly detection Meetup Slides
Anomaly detection Meetup SlidesAnomaly detection Meetup Slides
Anomaly detection Meetup Slides
 
Cluster
ClusterCluster
Cluster
 
Multivariate data analysis
Multivariate data analysisMultivariate data analysis
Multivariate data analysis
 
Data mining: Concepts and Techniques, Chapter12 outlier Analysis
Data mining: Concepts and Techniques, Chapter12 outlier Analysis Data mining: Concepts and Techniques, Chapter12 outlier Analysis
Data mining: Concepts and Techniques, Chapter12 outlier Analysis
 
Deep learning - Part I
Deep learning - Part IDeep learning - Part I
Deep learning - Part I
 
Mining Methods
Mining MethodsMining Methods
Mining Methods
 

Semelhante a Local Outlier Factor

Financial Management Slides Ch 14
Financial Management Slides Ch 14Financial Management Slides Ch 14
Financial Management Slides Ch 14Sayyed Naveed Ali
 
地域版IoT共創ラボで地方を元気に！関西企業コラボによる働く現場のDX事例
地域版IoT共創ラボで地方を元気に！関西企業コラボによる働く現場のDX事例地域版IoT共創ラボで地方を元気に！関西企業コラボによる働く現場のDX事例
地域版IoT共創ラボで地方を元気に！関西企業コラボによる働く現場のDX事例IoTビジネス共創ラボ
 
Greyhound - Powerful Pure Functional Kafka Library
Greyhound - Powerful Pure Functional Kafka LibraryGreyhound - Powerful Pure Functional Kafka Library
Greyhound - Powerful Pure Functional Kafka LibraryNatan Silnitsky
 
4. Pipeline Hazards.pptx
4. Pipeline Hazards.pptx4. Pipeline Hazards.pptx
4. Pipeline Hazards.pptxKarthikeyanC53
 
Buildinga billionuserloadbalancer may2015-sre-con15europe-shuff
Buildinga billionuserloadbalancer may2015-sre-con15europe-shuffBuildinga billionuserloadbalancer may2015-sre-con15europe-shuff
Buildinga billionuserloadbalancer may2015-sre-con15europe-shuffPatrick Shuff
 
ON TIME | ON TARGET: Leasing for Army Transportation Battalion
ON TIME | ON TARGET: Leasing for Army Transportation BattalionON TIME | ON TARGET: Leasing for Army Transportation Battalion
ON TIME | ON TARGET: Leasing for Army Transportation BattalionCurt Cooper
 
Rough run Oblivious Bloom Intersection
Rough run Oblivious Bloom IntersectionRough run Oblivious Bloom Intersection
Rough run Oblivious Bloom IntersectionAditya Mehta
 
Regulating Oil and Gas Production Emissions
Regulating Oil and Gas Production EmissionsRegulating Oil and Gas Production Emissions
Regulating Oil and Gas Production EmissionsBret Gallo
 
BlaBlaCar et la mise en place d'une fonctionnalité FlagFeature
BlaBlaCar et la mise en place d'une fonctionnalité FlagFeatureBlaBlaCar et la mise en place d'une fonctionnalité FlagFeature
BlaBlaCar et la mise en place d'une fonctionnalité FlagFeatureCocoaHeads France
 

Semelhante a Local Outlier Factor (10)

Analisis dinamico de un portico
Analisis dinamico de un porticoAnalisis dinamico de un portico
Analisis dinamico de un portico
 
Financial Management Slides Ch 14
Financial Management Slides Ch 14Financial Management Slides Ch 14
Financial Management Slides Ch 14
 
地域版IoT共創ラボで地方を元気に！関西企業コラボによる働く現場のDX事例
地域版IoT共創ラボで地方を元気に！関西企業コラボによる働く現場のDX事例地域版IoT共創ラボで地方を元気に！関西企業コラボによる働く現場のDX事例
地域版IoT共創ラボで地方を元気に！関西企業コラボによる働く現場のDX事例
 
Greyhound - Powerful Pure Functional Kafka Library
Greyhound - Powerful Pure Functional Kafka LibraryGreyhound - Powerful Pure Functional Kafka Library
Greyhound - Powerful Pure Functional Kafka Library
 
4. Pipeline Hazards.pptx
4. Pipeline Hazards.pptx4. Pipeline Hazards.pptx
4. Pipeline Hazards.pptx
 
Buildinga billionuserloadbalancer may2015-sre-con15europe-shuff
Buildinga billionuserloadbalancer may2015-sre-con15europe-shuffBuildinga billionuserloadbalancer may2015-sre-con15europe-shuff
Buildinga billionuserloadbalancer may2015-sre-con15europe-shuff
 
ON TIME | ON TARGET: Leasing for Army Transportation Battalion
ON TIME | ON TARGET: Leasing for Army Transportation BattalionON TIME | ON TARGET: Leasing for Army Transportation Battalion
ON TIME | ON TARGET: Leasing for Army Transportation Battalion
 
Rough run Oblivious Bloom Intersection
Rough run Oblivious Bloom IntersectionRough run Oblivious Bloom Intersection
Rough run Oblivious Bloom Intersection
 
Regulating Oil and Gas Production Emissions
Regulating Oil and Gas Production EmissionsRegulating Oil and Gas Production Emissions
Regulating Oil and Gas Production Emissions
 
BlaBlaCar et la mise en place d'une fonctionnalité FlagFeature
BlaBlaCar et la mise en place d'une fonctionnalité FlagFeatureBlaBlaCar et la mise en place d'une fonctionnalité FlagFeature
BlaBlaCar et la mise en place d'une fonctionnalité FlagFeature
 

Último

HMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptx
HMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptxHMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptx
HMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptxmarlenawright1
 
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptxBasic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptxDenish Jangid
 
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...Nguyen Thanh Tu Collection
 
Understanding Accommodations and Modifications
Understanding Accommodations and ModificationsUnderstanding Accommodations and Modifications
Understanding Accommodations and ModificationsMJDuyan
 
Spellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please PractiseSpellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please PractiseAnaAcapella
 
Food safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdfFood safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdfSherif Taha
 
FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024Elizabeth Walsh
 
Interdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptxInterdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptxPooja Bhuva
 
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptxSKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptxAmanpreet Kaur
 
Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)Jisc
 
General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...Poonam Aher Patil
 
Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)Jisc
 
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...Nguyen Thanh Tu Collection
 
How to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POSHow to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POSCeline George
 
Towards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptxTowards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptxJisc
 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxAreebaZafar22
 
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptxHMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptxEsquimalt MFRC
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.christianmathematics
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxRamakrishna Reddy Bijjam
 
The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxheathfieldcps1
 

Último (20)

HMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptx
HMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptxHMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptx
HMCS Vancouver Pre-Deployment Brief - May 2024 (Web Version).pptx
 
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptxBasic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
 
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
 
Understanding Accommodations and Modifications
Understanding Accommodations and ModificationsUnderstanding Accommodations and Modifications
Understanding Accommodations and Modifications
 
Spellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please PractiseSpellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please Practise
 
Food safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdfFood safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdf
 
FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024
 
Interdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptxInterdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptx
 
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptxSKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
 
Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)
 
General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...
 
Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)
 
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
 
How to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POSHow to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POS
 
Towards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptxTowards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptx
 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptx
 
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptxHMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
HMCS Max Bernays Pre-Deployment Brief (May 2024).pptx
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docx
 
The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptx
 

Local Outlier Factor

  • 1. Data Mining Lab, Local Outlier Factor Amr Koura / Page 1 Supervisor: Sebastian Bothe Local Outlier FactorLocal Outlier Factor
  • 2. Data Mining Lab, Local Outlier Factor Amr Koura / Page 2 Supervisor: Sebastian Bothe LabGoalLabGoal Implement Local Outlier factory Batch Mode. Implement Local Outlier factory Incremental Mode. Comparetwo modes. Integratecodeinto open sourceproject “RealKD”: https://bitbucket.org/realKD/
  • 3. Data Mining Lab, Local Outlier Factor Amr Koura / Page 3 Supervisor: Sebastian Bothe MotivationMotivation http://www.dbs.ifi.lmu.de/Publikationen/Papers/LOF.pdf
  • 4. Data Mining Lab, Local Outlier Factor Amr Koura / Page 4 Supervisor: Sebastian Bothe Local Outlier FactorLocal Outlier Factor reach−distk ( A , B)=max(d (B , A),k−distance(B)) lrd (A)= 1 ∑ B∈KNN (A) reach−distk ( A , B)/k LOF (A)= 1 k ∑ B∈KNN (A) lrd (B) lrd ( A) https://en.wikipedia.org/wiki/Local_outlier_factor
  • 5. Data Mining Lab, Local Outlier Factor Amr Koura / Page 5 Supervisor: Sebastian Bothe DemoDemo
  • 6. Data Mining Lab, Local Outlier Factor Amr Koura / Page 6 Supervisor: Sebastian Bothe
  • 7. Data Mining Lab, Local Outlier Factor Amr Koura / Page 7 Supervisor: Sebastian Bothe Incremental Outlier FactorIncremental Outlier Factor Motivation: - infinitestream makesmemory constraints. - computational constraint for processing each stream item. Goal: - Sameperformanceasiterated static LOF algorithm. - efficient algorithm: insertion/Deletion should effect only limited number of nearest neighbours http://www-ai.cs.uni-dortmund.de/LEHRE/FACHPROJEKT/SS12/paper/outlier/pokrajac2007.pdf
  • 8. Data Mining Lab, Local Outlier Factor Amr Koura / Page 8 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition Berlin;52.520;13.380 Hamburg;53.550;10.000 Munchen;48.140;11.580 Bonn;50.730;7.100 Koln;50.950;6.970 Frankfurt;50.120;8.680 Dortmund;51.510;7.480 Stuttgart;48.790;9.190 Essen;51.470;7.000 Cairo;30.3;31.14 Hurghada;27.15;33.50
  • 9. Data Mining Lab, Local Outlier Factor Amr Koura / Page 9 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition
  • 10. Data Mining Lab, Local Outlier Factor Amr Koura / Page 10 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition 1 1.1909475617292364 2 1.1956830856346556 3 0.9645631106850818 4 0.8029601477829005 5 0.7577540135599361 6 0.7377495644370516 7 0.7509608512974867 8 0.99956101138198 9 0.6943310060958396 10 3.7497548217312158 11 3.928514077815152 Now , lets add new Point="Alexandria;31.13;29.58"
  • 11. Data Mining Lab, Local Outlier Factor Amr Koura / Page 11 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition 0 1 2 6 5 8 7 4 3 9 10 1 0 6 5 8 7 4 3 2 9 10 2 7 5 0 1 3 4 6 8 9 10 3 4 8 6 5 7 1 2 0 9 10 4 3 8 6 5 7 1 2 0 9 10 5 7 3 4 6 8 1 2 0 9 10 6 8 4 3 5 7 1 0 2 9 10 7 5 2 3 4 6 8 1 0 9 10 8 6 4 3 5 7 1 0 2 9 10 9 10 2 0 7 5 1 3 4 6 8 10 9 2 0 7 5 1 3 4 6 8 0 1 2 6 5 8 7 4 3 11 9 10 1 0 6 5 8 7 4 3 2 11 9 10 2 7 5 0 1 3 4 6 8 11 9 10 3 4 8 6 5 7 1 2 0 11 9 10 4 3 8 6 5 7 1 2 0 11 9 10 5 7 3 4 6 8 1 2 0 11 9 10 6 8 4 3 5 7 1 0 2 11 9 10 7 5 2 3 4 6 8 1 0 11 9 10 8 6 4 3 5 7 1 0 2 11 9 10 9 11 10 2 0 7 5 1 3 4 6 8 10 9 11 2 0 7 5 1 3 4 6 8 11 9 10 2 0 7 5 1 3 4 6 8
  • 12. Data Mining Lab, Local Outlier Factor Amr Koura / Page 12 Supervisor: Sebastian Bothe Incremental LOF AdditionIncremental LOF Addition Cities9,10 haschangein their K-distance. According to: TheLRD for citiesexistsin K-NN of cities(9,10) should updated LRD List={9,10,2} According to , all citesthat hasany of cities {9,10,2} in their new nearest neighbour should updatethier LOF value. LOF List={9,10,2,0,7} lrd (A)= 1 ∑ B∈KNN (A) reach−distk ( A , B)/k LOF ( A)= 1 k ∑ B∈KNN ( A) lrd (B) lrd (A)
  • 13. Data Mining Lab, Local Outlier Factor Amr Koura / Page 13 Supervisor: Sebastian Bothe Comparison between staticandincremental LOFComparison between staticandincremental LOF Running static LOF output: 1.1909475617292364 1.1956830856346556 0.9645631106850818 0.8029601477829005 0.7577540135599361 0.7377495644370516 0.7509608512974867 0.99956101138198 0.6943310060958396 2.3423102537190847 2.342310253719085 2.342310253719085 Running incremental LOF and addition output: 1.1909475617292364 1.1956830856346556 0.9645631106850818 0.8029601477829005 0.7577540135599361 0.7377495644370516 0.7509608512974867 0.99956101138198 0.6943310060958396 2.3423102537190847 2.342310253719085 2.342310253719085
  • 14. Data Mining Lab, Local Outlier Factor Amr Koura / Page 14 Supervisor: Sebastian Bothe ConclusionConclusion Implementation of Batch incremental modehasdone. Batch modecodeisintegrated into theproject repository while pull request hasmadeto integrateit. Incremental LOF hasequivalent detection performanceas static LOF. Incremental LOF requireslesscomputation timethan time. Incremental LOF complexity isO(N log N)
  • 15. Data Mining Lab, Local Outlier Factor Amr Koura / Page 15 Supervisor: Sebastian Bothe Thank you