Do not crawl in the dust  different ur ls similar text

Do Not Crawl In The DUST: Different URLs Similar Text Uri Schonfeld Department of Electrical Engineering Technion Joint Work with Dr. Ziv Bar Yossef and Dr. Idit Keidar

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Talk Outline

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Even the WWW Gets Dusty

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],DUST Rules!

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],DUST is Bad

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],We Want To

[object Object],[object Object],[object Object],[object Object],[object Object],How do we Fight DUST Today? (1) Standard Canonization

[object Object],[object Object],[object Object],[object Object],[object Object],Standard Canonization is not Enough

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],How do we Fight DUST Today? (2) Shingles

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Shingles are Not Perfect

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],More Related Work

[object Object],[object Object],[object Object],[object Object],[object Object],Our contributions

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Types of DUST

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Our Basic Framework No Fetch Required

[object Object],[object Object],[object Object],[object Object],How to detect likely DUST rules?

Large Support Principle ,[object Object],[object Object],[object Object],Large Support Principle The support of a valid DUST rule is large

Rule Support: An Equivalent View ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Rule Support: An Equivalent View ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Large Buckets ,[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],Small Bucket Principle I am a DUCK not a DUST Small Buckets Principle Most of the support of valid Alias DUST rules is likely to belong to small buckets

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Algorithm – Detecting Likely DUST Rules No Fetch here!

Size and Comments ,[object Object],[object Object],[object Object],[object Object],[object Object]

Eliminating Redundant Rules ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],No Fetch here!

Validating Likely Rules ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],Comments About Validation

[object Object],[object Object],[object Object],[object Object],[object Object],Experimental Setup

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Recall

[object Object],DUST Distribution 47.1 DUST 25.7% Images 7.6% Soft Errors 17.9% Exact Copy 1.8% misc

[object Object],[object Object],[object Object],[object Object],Conclusions

[object Object],[object Object],[object Object],Things to fix

[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Parametric DUST

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],False Rules

Filtering out False Rulese ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

DustBuster – cleaning up the rules ,[object Object],[object Object],[object Object],[object Object],[object Object]

DustBuster – Validation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

DustBuster - Validation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Detect Alias DUST – take 2 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Eliminate Redundancies ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],No Fetch here!

Validate a Single Rule ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Validate Rules ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Do not crawl in the dust  different ur ls similar text

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (8)

Semelhante a Do not crawl in the dust  different ur ls similar text

Semelhante a Do not crawl in the dust  different ur ls similar text (20)

Mais de George Ang

Mais de George Ang (20)