SlideShare uma empresa Scribd logo
1 de 1
Ambit lick Solutions
Mail Id: Ambitlick@gmail.com , Ambitlicksolutions@gmail.Com
4. Information-Theoretic Outlier Detection for Large-Scale Categorical Data
Outlier detection can usually be considered as a pre-processing step for
locating, in a data set, those objects that do not conform to well-defined notions of
expected behavior. It is very important in data mining for discovering novel or rare
events, anomalies, vicious actions, exceptional phenomena, etc. We are investigating
outlier detection for categorical data sets. This problem is especially challenging
because of the difficulty of defining a meaningful similarity measure for categorical
data. In this paper, we propose a formal definition of outliers and an optimization
model of outlier detection, via a new concept of holoentropy that takes both entropy
and total correlation into consideration. Based on this model, we define a function for
the outlier factor of an object which is solely determined by the object itself and can
be updated efficiently. We propose two practical 1-parameter outlier detection
methods, named ITB-SS and ITB-SP, which require no user-defined parameters for
deciding whether an object is an outlier. Users need only provide the number of
outliers they want to detect. Experimental results show that ITB-SS and ITB-SP are
more effective and efficient than mainstream methods and can be used to deal with
both large and high-dimensional data sets where existing algorithms fail.

Mais conteúdo relacionado

Destaque

Tzarouk presentation
Tzarouk presentationTzarouk presentation
Tzarouk presentationtzarouk
 
Safe Screening Flow Chart
Safe Screening Flow ChartSafe Screening Flow Chart
Safe Screening Flow ChartDave Oakes
 
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...zillesubhan
 
English christmas assessment 2012
English christmas assessment 2012English christmas assessment 2012
English christmas assessment 2012AideenG
 
Tutorial de VODPOD
Tutorial de VODPODTutorial de VODPOD
Tutorial de VODPODFernanda Gr
 
Taller delgado-ullaguari_1D
 Taller delgado-ullaguari_1D Taller delgado-ullaguari_1D
Taller delgado-ullaguari_1DJorge Andres
 
Dispdir 131212 05787 evento american cup 2013
Dispdir 131212 05787  evento american cup 2013Dispdir 131212 05787  evento american cup 2013
Dispdir 131212 05787 evento american cup 2013Daniela Petrecca
 

Destaque (14)

Tzarouk presentation
Tzarouk presentationTzarouk presentation
Tzarouk presentation
 
iGuard
iGuard iGuard
iGuard
 
Safe Screening Flow Chart
Safe Screening Flow ChartSafe Screening Flow Chart
Safe Screening Flow Chart
 
Design Gerrit Yourself
Design Gerrit YourselfDesign Gerrit Yourself
Design Gerrit Yourself
 
Factsheet ITIL lite
Factsheet ITIL liteFactsheet ITIL lite
Factsheet ITIL lite
 
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
ORGANIZATIONAL SITUATIONS IN THE CONTEXT OF SOFTWARE DEVELOPMENT FIRMS: PROBL...
 
English christmas assessment 2012
English christmas assessment 2012English christmas assessment 2012
English christmas assessment 2012
 
Tutorial de VODPOD
Tutorial de VODPODTutorial de VODPOD
Tutorial de VODPOD
 
Web20 Bloglab
Web20 BloglabWeb20 Bloglab
Web20 Bloglab
 
Taller delgado-ullaguari_1D
 Taller delgado-ullaguari_1D Taller delgado-ullaguari_1D
Taller delgado-ullaguari_1D
 
Humor is The Health Of the Soul - Ratu Destian
Humor is The Health Of the Soul - Ratu DestianHumor is The Health Of the Soul - Ratu Destian
Humor is The Health Of the Soul - Ratu Destian
 
Dispdir 131212 05787 evento american cup 2013
Dispdir 131212 05787  evento american cup 2013Dispdir 131212 05787  evento american cup 2013
Dispdir 131212 05787 evento american cup 2013
 
Fitxa seguiment candidatura
Fitxa seguiment candidaturaFitxa seguiment candidatura
Fitxa seguiment candidatura
 
Diapositiva subir natali
Diapositiva  subir nataliDiapositiva  subir natali
Diapositiva subir natali
 

Mais de ambitlick

DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...ambitlick
 
Low cost Java 2013 IEEE projects
Low cost Java 2013 IEEE projectsLow cost Java 2013 IEEE projects
Low cost Java 2013 IEEE projectsambitlick
 
Ambitlick ns2 2013
Ambitlick ns2 2013Ambitlick ns2 2013
Ambitlick ns2 2013ambitlick
 
Low cost Java IEEE Projects 2013
Low cost Java IEEE Projects 2013Low cost Java IEEE Projects 2013
Low cost Java IEEE Projects 2013ambitlick
 
Handling selfishness in replica allocation
Handling selfishness in replica allocationHandling selfishness in replica allocation
Handling selfishness in replica allocationambitlick
 
Mutual distance bounding protocols
Mutual distance bounding protocolsMutual distance bounding protocols
Mutual distance bounding protocolsambitlick
 
Moderated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsModerated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsambitlick
 
Efficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsEfficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsambitlick
 
IEEE -2012-13 Projects IN NS2
IEEE -2012-13 Projects IN NS2  IEEE -2012-13 Projects IN NS2
IEEE -2012-13 Projects IN NS2 ambitlick
 
Adaptive weight factor estimation from user review 1
Adaptive weight factor estimation from user   review 1Adaptive weight factor estimation from user   review 1
Adaptive weight factor estimation from user review 1ambitlick
 
Integrated institutional portal
Integrated institutional portalIntegrated institutional portal
Integrated institutional portalambitlick
 
Mutual distance bounding protocols
Mutual distance bounding protocolsMutual distance bounding protocols
Mutual distance bounding protocolsambitlick
 
Moderated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsModerated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsambitlick
 
Efficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsEfficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsambitlick
 
Comments on “mabs multicast authentication based on batch signature”
Comments on “mabs multicast authentication based on batch signature”Comments on “mabs multicast authentication based on batch signature”
Comments on “mabs multicast authentication based on batch signature”ambitlick
 
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...ambitlick
 
Energy efficient protocol for deterministic
Energy efficient protocol for deterministicEnergy efficient protocol for deterministic
Energy efficient protocol for deterministicambitlick
 
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...ambitlick
 

Mais de ambitlick (20)

DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
DCIM: Distributed Cache Invalidation Method for Maintaining Cache Consistency...
 
Low cost Java 2013 IEEE projects
Low cost Java 2013 IEEE projectsLow cost Java 2013 IEEE projects
Low cost Java 2013 IEEE projects
 
Ambitlick ns2 2013
Ambitlick ns2 2013Ambitlick ns2 2013
Ambitlick ns2 2013
 
Low cost Java IEEE Projects 2013
Low cost Java IEEE Projects 2013Low cost Java IEEE Projects 2013
Low cost Java IEEE Projects 2013
 
Handling selfishness in replica allocation
Handling selfishness in replica allocationHandling selfishness in replica allocation
Handling selfishness in replica allocation
 
Mutual distance bounding protocols
Mutual distance bounding protocolsMutual distance bounding protocols
Mutual distance bounding protocols
 
Moderated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsModerated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroups
 
Efficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsEfficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secrets
 
IEEE -2012-13 Projects IN NS2
IEEE -2012-13 Projects IN NS2  IEEE -2012-13 Projects IN NS2
IEEE -2012-13 Projects IN NS2
 
Adaptive weight factor estimation from user review 1
Adaptive weight factor estimation from user   review 1Adaptive weight factor estimation from user   review 1
Adaptive weight factor estimation from user review 1
 
Integrated institutional portal
Integrated institutional portalIntegrated institutional portal
Integrated institutional portal
 
Embassy
EmbassyEmbassy
Embassy
 
Crm
Crm Crm
Crm
 
Mutual distance bounding protocols
Mutual distance bounding protocolsMutual distance bounding protocols
Mutual distance bounding protocols
 
Moderated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroupsModerated group authoring system for campus wide workgroups
Moderated group authoring system for campus wide workgroups
 
Efficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secretsEfficient spread spectrum communication without pre shared secrets
Efficient spread spectrum communication without pre shared secrets
 
Comments on “mabs multicast authentication based on batch signature”
Comments on “mabs multicast authentication based on batch signature”Comments on “mabs multicast authentication based on batch signature”
Comments on “mabs multicast authentication based on batch signature”
 
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
Energy-Efficient Protocol for Deterministic and Probabilistic Coverage In Sen...
 
Energy efficient protocol for deterministic
Energy efficient protocol for deterministicEnergy efficient protocol for deterministic
Energy efficient protocol for deterministic
 
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
Estimating Parameters of Multiple Heterogeneous Target Objects Using Composit...
 

Último

Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)eniolaolutunde
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformChameera Dedduwage
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Krashi Coaching
 
Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3JemimahLaneBuaron
 
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...EduSkills OECD
 
Unit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxUnit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxVishalSingh1417
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxheathfieldcps1
 
social pharmacy d-pharm 1st year by Pragati K. Mahajan
social pharmacy d-pharm 1st year by Pragati K. Mahajansocial pharmacy d-pharm 1st year by Pragati K. Mahajan
social pharmacy d-pharm 1st year by Pragati K. Mahajanpragatimahajan3
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Sapana Sha
 
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in DelhiRussian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhikauryashika82
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13Steve Thomason
 
General AI for Medical Educators April 2024
General AI for Medical Educators April 2024General AI for Medical Educators April 2024
General AI for Medical Educators April 2024Janet Corral
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdfQucHHunhnh
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfJayanti Pande
 
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdfBASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdfSoniaTolstoy
 
Class 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfClass 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfAyushMahapatra5
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphThiyagu K
 
Measures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDMeasures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDThiyagu K
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introductionMaksud Ahmed
 

Último (20)

Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy Reform
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3
 
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
 
Unit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxUnit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptx
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptx
 
social pharmacy d-pharm 1st year by Pragati K. Mahajan
social pharmacy d-pharm 1st year by Pragati K. Mahajansocial pharmacy d-pharm 1st year by Pragati K. Mahajan
social pharmacy d-pharm 1st year by Pragati K. Mahajan
 
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111Call Girls in Dwarka Mor Delhi Contact Us 9654467111
Call Girls in Dwarka Mor Delhi Contact Us 9654467111
 
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in DelhiRussian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
Russian Escort Service in Delhi 11k Hotel Foreigner Russian Call Girls in Delhi
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13
 
General AI for Medical Educators April 2024
General AI for Medical Educators April 2024General AI for Medical Educators April 2024
General AI for Medical Educators April 2024
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdf
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdf
 
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdfBASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
 
Class 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfClass 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdf
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot Graph
 
Measures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SDMeasures of Dispersion and Variability: Range, QD, AD and SD
Measures of Dispersion and Variability: Range, QD, AD and SD
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 

Information theoretic outlier detection for large-scale categorical data

  • 1. Ambit lick Solutions Mail Id: Ambitlick@gmail.com , Ambitlicksolutions@gmail.Com 4. Information-Theoretic Outlier Detection for Large-Scale Categorical Data Outlier detection can usually be considered as a pre-processing step for locating, in a data set, those objects that do not conform to well-defined notions of expected behavior. It is very important in data mining for discovering novel or rare events, anomalies, vicious actions, exceptional phenomena, etc. We are investigating outlier detection for categorical data sets. This problem is especially challenging because of the difficulty of defining a meaningful similarity measure for categorical data. In this paper, we propose a formal definition of outliers and an optimization model of outlier detection, via a new concept of holoentropy that takes both entropy and total correlation into consideration. Based on this model, we define a function for the outlier factor of an object which is solely determined by the object itself and can be updated efficiently. We propose two practical 1-parameter outlier detection methods, named ITB-SS and ITB-SP, which require no user-defined parameters for deciding whether an object is an outlier. Users need only provide the number of outliers they want to detect. Experimental results show that ITB-SS and ITB-SP are more effective and efficient than mainstream methods and can be used to deal with both large and high-dimensional data sets where existing algorithms fail.