Towards a Large Corpus of Richly Annotated Web Tables for Knowledge Base Population -> method for annotating web tables with hypotheses

1. LD4IE Workshop, ISWC 2017
Towards a Large Corpus of RichlyAnnotated Web
Tables for Knowledge Base Population
Basil Ell1, Sherzod Hakimov1, Philipp Braukmann2, Lorenzo Cazzoli2, Fabian Kaupmann2,
Amerigo Mancino2, Junaid Altaf Memon2, Kai Rother2, Abhishek Saini2 & Philipp Cimiano1
1 Semantic Computing Group
2 Intelligent Systems Seminar for Master’s Program
CITEC, Bielefeld University
1

2. LD4IE Workshop, ISWC 2017 2
Information Extraction : Text
In the 2016 presidential elections, members of both families ran for president, but both were defeated in their bids by Donald Trump,
who defeated Jeb Bush in the Republican primaries and Hillary Clinton in the general election, which effectively ended the 36 years
of Bush-Clinton dynasty that dominated the United States federal politics.

3. LD4IE Workshop, ISWC 2017 3
Information Extraction : Text
In the 2016 presidential elections, members of both families ran for president, but both were defeated in their bids by Donald Trump,
who defeated Jeb Bush in the Republican primaries and Hillary Clinton in the general election, which effectively ended the 36 years
of Bush-Clinton dynasty that dominated the United States federal politics.
In the 2016 presidential elections, members of both families ran for president, but both were defeated in their bids by Donald Trump,
who defeated Jeb Bush in the Republican primaries and Hillary Clinton in the general election, which effectively ended the 36 years
of Bush-Clinton dynasty that dominated the United States federal politics.
Named Entity Recognition & Linking

4. LD4IE Workshop, ISWC 2017 4
Information Extraction : Text
In the 2016 presidential elections, members of both families ran for president, but both were defeated in their bids by Donald Trump,
who defeated Jeb Bush in the Republican primaries and Hillary Clinton in the general election, which effectively ended the 36 years
of Bush-Clinton dynasty that dominated the United States federal politics.
In the 2016 presidential elections, members of both families ran for president, but both were defeated in their bids by Donald Trump,
who defeated Jeb Bush in the Republican primaries and Hillary Clinton in the general election, which effectively ended the 36 years
of Bush-Clinton dynasty that dominated the United States federal politics.
2016 presidential elections, ran for president Donald Trump

5. LD4IE Workshop, ISWC 2017 5
dbp: http://dbpedia.org/property/
Information Extraction : Text
In the 2016 presidential elections, members of both families ran for president, but both were defeated in their bids by Donald Trump,
who defeated Jeb Bush in the Republican primaries and Hillary Clinton in the general election, which effectively ended the 36 years
of Bush-Clinton dynasty that dominated the United States federal politics.
In the 2016 presidential elections, members of both families ran for president, but both were defeated in their bids by Donald Trump,
who defeated Jeb Bush in the Republican primaries and Hillary Clinton in the general election, which effectively ended the 36 years
of Bush-Clinton dynasty that dominated the United States federal politics.
2016 presidential elections, ran for president Donald Trump
dbp:nominee

6. LD4IE Workshop, ISWC 2017 6
dbp: http://dbpedia.org/property/
Information Extraction : Text
In the 2016 presidential elections, members of both families ran for president, but both were defeated in their bids by Donald Trump,
who defeated Jeb Bush in the Republican primaries and Hillary Clinton in the general election, which effectively ended the 36 years
of Bush-Clinton dynasty that dominated the United States federal politics.
In the 2016 presidential elections, members of both families ran for president, but both were defeated in their bids by Donald Trump,
who defeated Jeb Bush in the Republican primaries and Hillary Clinton in the general election, which effectively ended the 36 years
of Bush-Clinton dynasty that dominated the United States federal politics.
2016 presidential elections, ran for president Donald Trump
dbp:nominee

7. LD4IE Workshop, ISWC 2017 7
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
dbr: http://dbpedia.org/resource/
dbo: http://dbpedia.org/ontology/
Information Extraction : Table

8. LD4IE Workshop, ISWC 2017 8
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
dbr: http://dbpedia.org/resource/
dbo: http://dbpedia.org/ontology/
Information Extraction : Table
Politicians from US Political Parties from US

9. LD4IE Workshop, ISWC 2017 9
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
dbr:Barack_Obama
dbr: http://dbpedia.org/resource/
dbo: http://dbpedia.org/ontology/
dbo:birthDate
dbo:surname
dbo:region
dbr:Illinois
Information Extraction : Table

10. LD4IE Workshop, ISWC 2017 10
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
dbr:Barack_Obama
dbr: http://dbpedia.org/resource/
dbo: http://dbpedia.org/ontology/
dbo:birthDate
dbo:surname
dbo:region
dbr:Illinois
Information Extraction : Table
dbr:Barack_Obama dbo:region dbr:Illinois.

11. LD4IE Workshop, ISWC 2017 11
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
dbr:Barack_Obama
dbo:birthDate
dbo:surname
dbo:region
dbr:Illinois
Web Table Understanding -Knowledge Base Population

12. LD4IE Workshop, ISWC 2017 12
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
dbr:George_W._Bush
dbo:birthDate
dbo:surname
dbo:region
dbr:Texas
Web Table Understanding -Knowledge Base Population
dbr: Republican_Party_(United_States)
dbo:party

13. LD4IE Workshop, ISWC 2017 13
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democraticdbr:Hillary_Clinton
dbo:birthDate
dbo:surname
Web Table Understanding -Knowledge Base Population
dbr: Democratic_Party_(United_States)
dbo:party

14. LD4IE Workshop, ISWC 2017 14
Web Table Understanding -Knowledge Base Population
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic

15. LD4IE Workshop, ISWC 2017 15
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
Donald Trump Trump Jun 14, 1946 New York Republican
Web Table Understanding -Knowledge Base Population

16. LD4IE Workshop, ISWC 2017 16
Web Table Understanding -Knowledge Base Population
dbr:Donald_Trump dbo:region dbr: New_York_(state) .Inferred Data (new)
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
Donald Trump Trump Jun 14, 1946 New York Republican

17. LD4IE Workshop, ISWC 2017 17
Web Table Understanding -Knowledge Base Population
dbr:Donald_Trump dbo:region dbr: New_York_(state) .Inferred Data (new)
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
Donald Trump Trump Jun 14, 1946 New York Republican
populate

18. LD4IE Workshop, ISWC 2017
Our Contributions
! Subtasks for processing web tables
! Corpus of annotated web tables
18

19. LD4IE Workshop, ISWC 2017
Dataset
19
WDC Web Table Corpus (2015) 233 million tables in JSON format.
This corpus was created from a set of 1.78 billion HTML pages from the Common Crawl July 2015 Web [2] corpus.
[1] http://webdatacommons.org/webtables/
[2] http://commoncrawl.org/2015/08/july-2015-crawl-archive-available/

20. LD4IE Workshop, ISWC 2017
Pipeline -Subtasks
! Table Orientation Detection
! Table Classification
! Table Segmentation
! Table Exclusion
! Language Detection
! Table Normalization
! Entity Linking
! Literal Linking
20

21. LD4IE Workshop, ISWC 2017
Pipeline -Subtasks
21

22. LD4IE Workshop, ISWC 2017
Table Orientation Detection
22
Available in WDC Web Table Corpus (horizontal, vertical)
"H0" : {
"created_by_task": “table_orientation detection",
"hypothesis_type": “table_orientation",
"orientation" : "horizontal",
"source" : "WTC",
}
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
H0: Table has horizontal
orientation

23. LD4IE Workshop, ISWC 2017
Table Classification
23
Available in WDC Web Table Corpus (relational table, entity table, matrix table, layout table)
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
"H1" : {
"created_by_task": “table_classification",
"hypothesis_type": “table_classification",
"classification" : "relational",
"source" : "WTC",
}
H1 : Table is relational

24. LD4IE Workshop, ISWC 2017
Table Segmentation
24
Available in WDC Web Table Corpus (header row index)
"H2": {
"created_by_task": "table_segmentation",
"hypothesis_name": "table_segmentation",
"header_row": [0],
"source": "WTC"
}
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
H2 : First row is a header
row

25. LD4IE Workshop, ISWC 2017
Table Exclusion
25
Available in WDC Web Table Corpus (relational tables with candidate entities)
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic

26. LD4IE Workshop, ISWC 2017
Language Detection
26
LangDetect tool[1] - 55 languages
Multiple hypotheses with confidence values
[1] https://pypi.python.org/pypi/langdetect
"H3" : {
"created_by_task" : "language_detection", "lang" : "en",
"confidence" : 0.9
}
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
H3 : Table contains
English content

27. LD4IE Workshop, ISWC 2017
Table Normalization
27
Convert cell values into canonical representation e.g. weight, distance, date, values with unit
"H4": {
"created_by_task": "table_normalization",
"hypothesis_name": "date",
"based_on_hypotheses": ["H3"]
"data_type": "xsd:date",
"original_value": "Aug, 4, 1961",
"value": "1961-08-04",
"year": 1961,
"month": 8,
"day": 4,
"refers_to_row": 1,
"refers_to_column": 2,
}
Types of hypotheses such as: date, integer, float, value with unit(kg, km …)
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
Based on hypotheses from Language Detection
H4 : expressed as
xsd:date in YYYY-MM-DD

28. LD4IE Workshop, ISWC 2017
Entity Linking
28
Index for detecting mentions —> DBpedia label properties
Based on hypotheses from Table Normalization
"H5": {
"created_by_task": "table_normalization",
"hypothesis_name": "plain",
"based_on_hypotheses": ["H3"]
"value": "Obama",
"refers_to_column": 1,
"refers_to_row": 1,
}
"H6": {
"created_by_task": "entity_linking",
"hypothesis_type": "resource",
"based_on_hypotheses": ["H5"],
"entity": "dbr:Barack_Obama",
"confidence": 0.67
}
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
Link Resources, Classes and Properties
H6 : dbr:Barack_Obama mentioned here

29. LD4IE Workshop, ISWC 2017
Literal Linking
29
Index for detecting literals —> DBpedia data type properties
Based on hypotheses from Table Normalization
"H6": {
"created_by_task": "entity_linking",
"hypothesis_type": "resource",
"based_on_hypotheses": ["H5"],
"entity": "dbr:Barack_Obama",
"confidence": 0.67
}
"H7": {
"created_by_task": "literal_linking",
"based_on_hypotheses": [“H4","H6"],
"modified_literal": "1961-08-04",
"property": "dbo:birthDate"
}
"H4": {
"created_by_task": "table_normalization",
"hypothesis_name": "date",
"based_on_hypotheses": ["H3"]
"data_type": "xsd:date",
"original_value": "Aug, 4, 1961",
"value": "1961-08-04",
"year": 1961,
"month": 8,
"day": 4,
"refers_to_row": 1,
"refers_to_column": 2,
}
Politician Surname Date of Birth State Political Party
Barack Obama Obama Aug 4, 1961 Illinois -
George W. Bush Bush July 6, 1946 Texas Republican
Hillary Clinton Clinton Oct 26, 1947 - Democratic
H7 : dbo:birthDate of dbr:Barack_Obama
mentioned here

30. LD4IE Workshop, ISWC 2017
Statistics
30
Processed 1 million tables
After exclusion task remaining tables: 349,284

31. LD4IE Workshop, ISWC 2017
Statistics -Language Detection
31

32. LD4IE Workshop, ISWC 2017
Statistics -Table Normalization
32

33. LD4IE Workshop, ISWC 2017
Statistics -Table Normalization
33

34. LD4IE Workshop, ISWC 2017
Statistics -Entity Linking
34
Average number of hypotheses

35. LD4IE Workshop, ISWC 2017
Statistics -Literal Linking
35
At least one hypothesis was generated for 556 tables (0.2 %).
Top-5 most frequent properties

36. LD4IE Workshop, ISWC 2017
Conclusion
36
• Pipeline for processing web tables
• Graph of hypotheses
• Hypotheses might disagree
• Annotated web tables corpus available to the research community

37. LD4IE Workshop, ISWC 2017
Future Work
37
• Improve Literal Linking Step
• Extend Table Normalization to other datatypes
• Add higher level tasks (Column understanding, Row understanding, Table understanding)
• Build system that performs KB population
Higher-level understanding

38. LD4IE Workshop, ISWC 2017
Thanks!
38
https://isywtu.github.io/website/