Rule-based Information Extraction is Dead! Long Live Rule-based Information Extraction Systems!

•Transferir como PPT, PDF•

1 gostou•1,096 visualizações

Poster for our ACL'2013 short paper "Rule-based Information Extraction is Dead! Long Live Rule-based Information Extraction Systems! "

Rule-based Information Extraction is DEAD
Long Live Rule-based Information Extraction Systems!
Laura Chiticariu, Yunyao Li, Frederick Reiss
IBM Research - Almaden

THE DISCONNECT: ACADEMIC vs. INDUSTRY
Implementations of Entity Extraction

Entity Extraction Papers by Year

3.5%
21%

100%

RuleBased
Hybrid

45%

50%

RuleBased

22%

75%

17%

Hybrid

17%

Machine
Learning
Based

33%
0%

NLP Papers
(2003-2012)

All Vendors

Large Vendors

Machine
Learning
Based

Fraction of NLP Papers

67%

Commercial Products
(2013)

Year of Publication

THE EXPLANATIONS
Academia
Rule-based IE

PROs
•Declarative Heuristic
•Easy to comprehend
•Easy to maintain
•Easy to incorporate domain
knowledge
•Easy to debug

ML-based IE

PROs
•Trainable
•Adaptable
•Reduces manual effort

CONs
CONs
• Heuristic
•Requires tedious manual
labor

Industry

•Requires labeled data
•Requires retraining for
domain adaptation
•Requires ML expertise to
use or maintain
• Opaque

Evaluating
Benefits
Evaluating IE on its own of IE
Precision and Recall

Evaluating
Costs
of IE
Labor cost of writing
rules

Evaluating IE as part of
a larger process
Using ill-defined metrics
that are subject to change

Labor cost
Hardware cost
Business risk

Others
What’s the research in
Rule-based IE?

BRIDGING THE GAP
Where is the research in rule-based IE? Making it more principled, effective, and efficient
Define standard IE rule language and data model.
• What is the right data model to capture text, annotations over text, and their properties?
• Can we establish a standard declarative extensible rule language to solve most IE tasks encountered so far?
Systems research based on standard IE rule language.
• Data representation
• Automatic performance optimization
• Exploring modern hardware …
ML research based on standard IE rule language
• How to learn basic primitives such as regular expressions and dictionaries?
• How to automatically generate rules that are understandable and maintainable?

Mais conteúdo relacionado

Destaque

In this paper we present an approach to Word Sense Disambiguation based on Topic Modeling (LDA). Our approach consists of two different steps, where first a binary classifier is applied to decide whether the most frequent sense applies or not, and then another classifier deals with the non most frequent sense cases. An exhaustive evaluation is performed on the Spanish corpus Ancora, to analyze the performance of our two-step system and the impact of the context and the different parameters in the system. Our best experiment reaches an accuracy of 74.53, which is 6 points over the highest baseline. All the software developed for these experiments has been made freely available, to enable reproducibility and allow the re-usage of the software.

Topic modeling and WSD on the Ancora corpus

Topic modeling and WSD on the Ancora corpus

Topic modeling and WSD on the Ancora corpus

Rubén Izquierdo Beviá

PhD Thesis F. Petroni: "Mining at scale with latent factor models for matrix completion." Sapienza University of Rome, 2016. Abstract: "Predicting which relationships are likely to occur between real-world objects is a key task for several applications. For instance, recommender systems aim at predicting the existence of unknown relationships between users and items, and exploit this information to provide personalized suggestions for items to be of use to a specific user. Matrix completion techniques aim at solving this task, identifying and leveraging the latent factors that triggered the the creation of known relationships to infer missing ones. This problem, however, is made challenging by the size of today’s datasets. One way to handle such large-scale data, in a reasonable amount of time, is to distribute the matrix completion procedure over a cluster of commodity machines. However, current approaches lack of efficiency and scalability, since, for instance, they do not minimize the communication or ensure a balance workload in the cluster. A further aspect of matrix completion techniques we investigate is how to improve their prediction performance. This can be done, for instance, considering the context in which relationships have been captured. However, incorporating generic contextual information within a matrix completion algorithm is a challenging task. In the first part of this thesis, we study distributed matrix completion solutions, and address the above issues by examining input slicing techniques based on graph partitioning algorithms. In the second part of the thesis, we focus on context-aware matrix completion techniques, providing solutions that can work both (i) when the revealed entries in the matrix have multiple values and (ii) all the same value."

Mining at scale with latent factor models for matrix completion

Mining at scale with latent factor models for matrix completion

Mining at scale with latent factor models for matrix completion

Fabio Petroni, PhD

Invited Talk at Modern Data Management Systems Summit on August 29-30, 2014 at Tsinghua University in Beijing, China. http://ise.thss.tsinghua.edu.cn/MDMS/English/program.jsp Abstract: Modern enterprises are increasingly relying on complex analyses on large data sets to drive business decisions. Tasks such as root cause analysis from system logs and lead generation based on social media, customer retention and digital marketing are rapidly gaining importance. These applications generally consist of three major analytic phases: text analytics, semi-structured data processing (joins, group-by, aggregation), and statistical/predictive modeling. The size of the datasets in conjunction with the complexity of the analysis necessitates large-scale distributed processing of the analytical algorithms. At IBM we are building tools and technologies based on declarative languages to support each of these analytic phases. The declarative nature of the language abstracts away the need for programmer-optimization. Furthermore, the syntax of these languages is designed to appeal to the corresponding communities. As an example for statistical modeling, we expose a high-level language with syntax similar to R -- a very popular statistical processing language. In this talk I will give an overview of some real-world big data applications we are currently working on and use that to motivate the need for declarative analytics consisting of the three major phases discussed above. I will then describe, in some detail, declarative systems for text analytics along with a discussion on speeds, feeds and comparisons.

The Power of Declarative Analytics

The Power of Declarative Analytics

The Power of Declarative Analytics

RANLP2013: DutchSemCor, in Quest of the Ideal Sense Tagged Corpus

RANLP2013: DutchSemCor, in Quest of the Ideal Sense Tagged Corpus

RANLP2013: DutchSemCor, in Quest of the Ideal Sense Tagged Corpus

Rubén Izquierdo Beviá

CLTL python course: Object Oriented Programming (3/3)

CLTL python course: Object Oriented Programming (3/3)

CLTL python course: Object Oriented Programming (3/3)

Rubén Izquierdo Beviá

Transparent Machine Learning for Information Extraction: State-of-the-Art and...

Transparent Machine Learning for Information Extraction: State-of-the-Art and...

Transparent Machine Learning for Information Extraction: State-of-the-Art and...

Poster for our ACL paper "Polyglot: Multilingual Semantic Role Labeling with Unified Labels". Abstract: We present POLYGLOT, a semantic role labeling system capable of semantically parsing sentences in 9 different languages from 4 different language groups. A core differentiator is that this system predicts English Proposition Bank labels for all supported languages. This means that for instance a Japanese sentence will be tagged with the same labels as an English sentence with similar semantics would be. This is made possible by training the system with target language data that was automatically labeled with English PropBank labels using an annotation projection approach. We give an overview of our system, the automatically produced training data, and discuss possible applications and limitations of this work. We present a demonstrator that accepts sentences in English, German, French, Spanish, Japanese, Chinese, Arabic, Russian and Hindi and outputs a visualization of its shallow semantics.

Polyglot: Multilingual Semantic Role Labeling with Unified Labels

Polyglot: Multilingual Semantic Role Labeling with Unified Labels

Polyglot: Multilingual Semantic Role Labeling with Unified Labels

KafNafParserPy: a python library for parsing/creating KAF and NAF files

KafNafParserPy: a python library for parsing/creating KAF and NAF files

KafNafParserPy: a python library for parsing/creating KAF and NAF files

Rubén Izquierdo Beviá

DutchSemCor workshop: Domain classification and WSD systems

DutchSemCor workshop: Domain classification and WSD systems

DutchSemCor workshop: Domain classification and WSD systems

Rubén Izquierdo Beviá

F. Petroni and L. Querzoni: "HSIENA: a hybrid publish/subscribe system." In: Proceedings of the 1st International Workshop on Dependable and Secure Computing for Large-scale Complex Critical Infrastructures (DESEC-LCCI), 2012. Abstract: "The SIENA publish/subscribe system represents a prototypical design for a distributed event notification service implementing the content-based publish/subscribe communication paradigm. A clear shortcoming of SIENA is represented by its static configuration that must be managed and updated by human administrators every time one of its internal processes (brokers) needs to be added or repaired (e.g. due to a crash failure). This problem limits the applicability of SIENA in large complex critical infrastructures where self-adaptation and -configuration are crucial requirements. In this paper we propose HSIENA, a hybrid architecture that complements SIENA by adding the ability to self-reconfigure after broker additions and removals. The architecture has a novel design that mixes the classic SIENA’s distributed architecture with a highly available cloud-based storage service."

HSIENA: a hybrid publish/subscribe system

HSIENA: a hybrid publish/subscribe system

HSIENA: a hybrid publish/subscribe system

Fabio Petroni, PhD

This is the slides used in our 3-hour tutorial at VLDB'2014. Yunyao Li, Ziyang Liu, Huaiyu Zhu: Enterprise Search in the Big Data Era: Recent Developments and Open Challenges. PVLDB 7(13): 1717-1718 (2014) Abstract: Enterprise search allows users in an enterprise to retrieve desired information through a simple search interface. It is widely viewed as an important productivity tool within an enterprise. While Internet search engines have been highly successful, enterprise search remains notoriously challenging due to a variety of unique challenges, and is being made more so by the increasing heterogeneity and volume of enterprise data. On the other hand, enterprise search also presents opportunities to succeed in ways beyond current Internet search capabilities. This tutorial presents an organized overview of these challenges and opportunities, and reviews the state-of-the-art techniques for building a reliable and high quality enterprise search engine, in the context of the rise of big data.

Enterprise Search in the Big Data Era: Recent Developments and Open Challenges

Enterprise Search in the Big Data Era: Recent Developments and Open Challenges

Enterprise Search in the Big Data Era: Recent Developments and Open Challenges

CLIN-2015 Presentation Word Sense Disambiguation is still an unsolved problem in Natural Language Processing. We claim that most approaches do not model the context correctly, by relying too much on the local context (the words surrounding the word in question), or on the most frequent sense of a word. In order to provide evidence for this claim, we conducted an in-depth analysis of all-words tasks of the competitions that have been organized (Senseval 2&3, Semeval-2007, Semeval-2010, Semeval 2013). We focused on the average error rate per competition and across competitions per part of speech, lemma, relative frequency class, and polysemy class. In addition, we inspected the “difficulty” of a token(word) by calculating the average polysemy of the words in the sentence of a token. Finally, we inspected to what extent systems always chose the most frequent sense. The results from Senseval 2, which are representative of other competitions, showed that the average error rate for monosemous words was 33.3% due to part of speech errors. This number was 71% for multiword and phrasal verbs. In addition, we observe that higher polysemy yields a higher error rate. Moreover, we do not observe a drop in the error rate if there are multiple occurrences of the same lemma, which might indicate that systems rely mostly on the sentence itself. Finally, out of the 799 tokens for which the correct sense was not the most frequent sense, system still assigned the most frequent sense in 84% of the cases. For future work, we plan to develop a strategy in order to determine in which context the predominant sense should be assigned, and more importantly when it should not be assigned. One of the most important parts of this strategy would be to not only determine the meaning of a specific word, but to also know it’s referential meaning. For example, in the case of the lemma ‘winner’, we do not only want to know what ‘winner’ means, but we also want to know what this ‘winner’ won and who this ‘winner’ was.

Error analysis of Word Sense Disambiguation

Error analysis of Word Sense Disambiguation

Error analysis of Word Sense Disambiguation

Rubén Izquierdo Beviá

F. Petroni, L. Del Corro and R. Gemulla: "CORE: Context-Aware Open Relation Extraction with Factorization Machines." In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2015. Abstract: "We propose CORE, a novel matrix factorization model that leverages contextual information for open relation extraction. Our model is based on factorization machines and integrates facts from various sources, such as knowledge bases or open information extractors, as well as the context in which these facts have been ob- served. We argue that integrating contextual information—such as metadata about extraction sources, lexical context, or type information—significantly improves prediction performance. Open information extractors, for example, may produce extractions that are unspecific or ambiguous when taken out of context. Our experimental study on a large real-world dataset indicates that CORE has significantly better prediction performance than state-of- the-art approaches when contextual information is available."

CORE: Context-Aware Open Relation Extraction with Factorization Machines

CORE: Context-Aware Open Relation Extraction with Factorization Machines

CORE: Context-Aware Open Relation Extraction with Factorization Machines

Fabio Petroni, PhD

Juan Calvino y el Calvinismo

Juan Calvino y el Calvinismo

Juan Calvino y el Calvinismo

Rubén Izquierdo Beviá

Text analysis and Semantic Search with GATE

Text analysis and Semantic Search with GATE

Text analysis and Semantic Search with GATE

Information Extraction

Information Extraction

Information Extraction

Rubén Izquierdo Beviá

Destaque (16)

Topic modeling and WSD on the Ancora corpus

Topic modeling and WSD on the Ancora corpus

Topic modeling and WSD on the Ancora corpus

Mining at scale with latent factor models for matrix completion

Mining at scale with latent factor models for matrix completion

Mining at scale with latent factor models for matrix completion

The Power of Declarative Analytics

The Power of Declarative Analytics

The Power of Declarative Analytics

RANLP2013: DutchSemCor, in Quest of the Ideal Sense Tagged Corpus

RANLP2013: DutchSemCor, in Quest of the Ideal Sense Tagged Corpus

RANLP2013: DutchSemCor, in Quest of the Ideal Sense Tagged Corpus

CLTL python course: Object Oriented Programming (3/3)

CLTL python course: Object Oriented Programming (3/3)

CLTL python course: Object Oriented Programming (3/3)

Transparent Machine Learning for Information Extraction: State-of-the-Art and...

Transparent Machine Learning for Information Extraction: State-of-the-Art and...

Transparent Machine Learning for Information Extraction: State-of-the-Art and...

Polyglot: Multilingual Semantic Role Labeling with Unified Labels

Polyglot: Multilingual Semantic Role Labeling with Unified Labels

Polyglot: Multilingual Semantic Role Labeling with Unified Labels

KafNafParserPy: a python library for parsing/creating KAF and NAF files

KafNafParserPy: a python library for parsing/creating KAF and NAF files

KafNafParserPy: a python library for parsing/creating KAF and NAF files

DutchSemCor workshop: Domain classification and WSD systems

DutchSemCor workshop: Domain classification and WSD systems

DutchSemCor workshop: Domain classification and WSD systems

HSIENA: a hybrid publish/subscribe system

HSIENA: a hybrid publish/subscribe system

HSIENA: a hybrid publish/subscribe system

Enterprise Search in the Big Data Era: Recent Developments and Open Challenges

Enterprise Search in the Big Data Era: Recent Developments and Open Challenges

Enterprise Search in the Big Data Era: Recent Developments and Open Challenges

Error analysis of Word Sense Disambiguation

Error analysis of Word Sense Disambiguation

Error analysis of Word Sense Disambiguation

CORE: Context-Aware Open Relation Extraction with Factorization Machines

CORE: Context-Aware Open Relation Extraction with Factorization Machines

CORE: Context-Aware Open Relation Extraction with Factorization Machines

Juan Calvino y el Calvinismo

Juan Calvino y el Calvinismo

Juan Calvino y el Calvinismo

Text analysis and Semantic Search with GATE

Text analysis and Semantic Search with GATE

Text analysis and Semantic Search with GATE

Information Extraction

Information Extraction

Information Extraction

Mais de Yunyao Li

The Role of Patterns in the Era of Large Language Models

The Role of Patterns in the Era of Large Language Models

The Role of Patterns in the Era of Large Language Models

Keynote talk at HILDA'2023 at SIGMOD on June 18, 2023. Abstract: The ability to build large-scale knowledge bases that capture and extend the implicit knowledge of human experts is the foundation for many AI systems. We use an ontology-driven approach for the building, growing and serving of such knowledge bases. This approach relies on several well-known building blocks: document conversion, natural language processing, entity resolution, data transformation and fusion. In this talk, I will discuss wide range of real-world challenges related to the building of these blocks and present our work to address these challenges via better human-machine cooperation.

Building, Growing and Serving Large Knowledge Graphs with Human-in-the-Loop

Building, Growing and Serving Large Knowledge Graphs with Human-in-the-Loop

Building, Growing and Serving Large Knowledge Graphs with Human-in-the-Loop

EMNLP'2022 Tutorial "Meaning Representations for Natural Languages: Design, Models and Applications" Instructors: Jeffrey Flanigan, Ishan Jindal, Yunyao Li, Tim O’Gorman, Martha Palmer Abstract: We propose a cutting-edge tutorial that reviews the design of common meaning representations, SoTA models for predicting meaning representations, and the applications of meaning representations in a wide range of downstream NLP tasks and real-world applications. Reporting by a diverse team of NLP researchers from academia and industry with extensive experience in designing, building and using meaning representations, our tutorial has three components: (1) an introduction to common meaning representations, including basic concepts and design challenges; (2) a review of SoTA methods on building models for meaning representations; and (3) an overview of applications of meaning representations in downstream NLP tasks and real-world applications. We will also present qualitative comparisons of common meaning representations and a quantitative study on how their differences impact model performance. Finally, we will share best practices in choosing the right meaning representation for downstream tasks.

Meaning Representations for Natural Languages: Design, Models and Applications

Meaning Representations for Natural Languages: Design, Models and Applications

Meaning Representations for Natural Languages: Design, Models and Applications

Natural language understanding is a fundamental task in artificial intelligence. English understanding has reached a mature state and successfully deployed in multiple IBM AI products and services, such as Watson Natural Language Understanding and Watson Discovery. However, scaling existing products/services to support additional languages remain an open challenge. In this talk, we will discuss the open challenges in supporting universal natural language understanding. We will share our work in the past few years in addressing these challenges. We will also showcase how universal semantic representation of natural languages can enable cross-lingual information extraction in concrete domains (e.g. compliance) and show ongoing efforts towards seamless scaling existing NLP capabilities across languages with minimal efforts.

Taming the Wild West of NLP

Taming the Wild West of NLP

Taming the Wild West of NLP

Invited talk at Document Intelligence workshop at KDD'2021. Harvesting information from complex documents such as in financial reports and scientific publications is critical to building AI applications for business and research. Such documents are often in PDF format with critical facts and data conveyed in table and graphs. Extracting such information is essential to extract insights from these documents. In IBM Research, we have a rich agenda in this area that we call Deep Document Understanding. In this talk, I will focus on our research on Deep Table Understanding — extracting and understanding tables from PDF documents. I will introduce key challenges in table extraction and understanding and how we address such challenges, from how to acquire data at scale to enable deep neural network models to how to build, customize and evaluate such models. I will also describe how our work enables real-world use cases in domains such as finance and life science. Finally, I will briefly present TableQA, an important downstream task enabled by Deep Table Understanding.

Towards Deep Table Understanding

Towards Deep Table Understanding

Towards Deep Table Understanding

Final deck for our popular tutorial on "Explainability for Natural Language Processing" at KDD'2021. See links below for downloadable version (with higher resolution) and recording of the live tutorial. Title: Explainability for Natural Language Processing Presenter: Marina Danilevsky, Shipi Dhanorkar, Yunyao Li and Lucian Popa and Kun Qian and Anbang Xu Website: http://xainlp.github.io/ Recording: https://www.youtube.com/watch?v=PvKOSYGclPk&t=2s Downloadable version with higher resolution: https://drive.google.com/file/d/1_gt_cS9nP9rcZOn4dcmxc2CErxrHW9CU/view?usp=sharing @article{kdd2021xaitutorial, title={Explainability for Natural Language Processing}, author= {Marina Danilevsky, Shipi Dhanorkar and Yunyao Li and Lucian Popa and Kun Qian and Anbang Xu}, journal={KDD}, year={2021} } Abstract: This lecture-style tutorial, which mixes in an interactive literature browsing component, is intended for the many researchers and practitioners working with text data and on applications of natural language processing (NLP) in data science and knowledge discovery. The focus of the tutorial is on the issues of transparency and interpretability as they relate to building models for text and their applications to knowledge discovery. As black-box models have gained popularity for a broad range of tasks in recent years, both the research and industry communities have begun developing new techniques to render them more transparent and interpretable.Reporting from an interdisciplinary team of social science, human-computer interaction (HCI), and NLP/knowledge management researchers, our tutorial has two components: an introduction to explainable AI (XAI) in the NLP domain and a review of the state-of-the-art research; and findings from a qualitative interview study of individuals working on real-world NLP projects as they are applied to various knowledge extraction and discovery at a large, multinational technology and consulting corporation. The first component will introduce core concepts related to explainability inNLP. Then, we will discuss explainability for NLP tasks and reporton a systematic literature review of the state-of-the-art literaturein AI, NLP and HCI conferences. The second component reports on our qualitative interview study, which identifies practical challenges and concerns that arise in real-world development projects that require the modeling and understanding of text data.

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Explainability for Natural Language Processing

NOTE: Please check out the final version here with small but important updates and links to downloadable version and recording: https://www.slideshare.net/YunyaoLi/explainability-for-natural-language-processing-249992241 Updated version on our popular tutorial on "Explainability for Natural Language Processing" as a tutorial at KDD'2021. Title: Explainability for Natural Language Processing @article{kdd2021xaitutorial, title={Explainability for Natural Language Processing}, author= {Marina Danilevsky, Dhanorkar, Shipi and Li, Yunyao and Lucian Popa and Kun Qian and Anbang Xu}, journal={KDD}, year={2021} } Presenter: Marina Danilevsky, Dhanorkar, Shipi and Li, Yunyao and Lucian Popa and Kun Qian and Anbang Xu Website: http://xainlp.github.io/ Abstract: This lecture-style tutorial, which mixes in an interactive literature browsing component, is intended for the many researchers and practitioners working with text data and on applications of natural language processing (NLP) in data science and knowledge discovery. The focus of the tutorial is on the issues of transparency and interpretability as they relate to building models for text and their applications to knowledge discovery. As black-box models have gained popularity for a broad range of tasks in recent years, both the research and industry communities have begun developing new techniques to render them more transparent and interpretable.Reporting from an interdisciplinary team of social science, human-computer interaction (HCI), and NLP/knowledge management researchers, our tutorial has two components: an introduction to explainable AI (XAI) in the NLP domain and a review of the state-of-the-art research; and findings from a qualitative interview study of individuals working on real-world NLP projects as they are applied to various knowledge extraction and discovery at a large, multinational technology and consulting corporation. The first component will introduce core concepts related to explainability inNLP. Then, we will discuss explainability for NLP tasks and reporton a systematic literature review of the state-of-the-art literaturein AI, NLP and HCI conferences. The second component reports on our qualitative interview study, which identifies practical challenges and concerns that arise in real-world development projects that require the modeling and understanding of text data.

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Explainability for Natural Language Processing

The ability to build large-scale domain-specific knowledge bases that capture and extend the implicit knowledge of human experts is the foundation for many AI systems. We use an ontology-driven approach for the creation, representation and consumption of such domain-specific knowledge bases. This approach relies on several well-known building blocks: natural language processing, entity resolution, data transformation and fusion. I will present several human-in-the-loop work that target domain experts (rather than programmers) to extract the domain knowledge from the human expert and map it into the "right" models or algorithms. I will also share successful use cases in several domains, including Compliance, Finance, and Healthcare: by using these tools we can match the level of accuracy achieved by manual efforts, but at a significantly lower cost and much higher scale and automation.

Human in the Loop AI for Building Knowledge Bases

Human in the Loop AI for Building Knowledge Bases

Human in the Loop AI for Building Knowledge Bases

Slides for talk given at Women in Engineering on March 20, 2021. Abstract: Natural language understanding is a fundamental task in artificial intelligence. English understanding has reached a mature state and successfully deployed in multiple IBM AI products and services, such as Watson Natural Language Understanding and Watson Discovery. However, scaling existing products/services to support additional languages remain an open challenge. In this talk, we will discuss the open challenges in supporting universal natural language understanding. We will share our work in the past few years in addressing these challenges. We will also showcase how universal semantic representation of natural languages can enable cross-lingual information extraction in concrete domains (e.g. compliance) and show ongoing efforts towards seamless scaling existing NLP capabilities across languages with minimal efforts.

Towards Universal Language Understanding

Towards Universal Language Understanding

Towards Universal Language Understanding

Tutorial at AACL'2020 (http://www.aacl2020.org/program/tutorials/#t4-explainability-for-natural-language-processing). More recent version: https://www.slideshare.net/YunyaoLi/explainability-for-natural-language-processing-249912819 Title: Explainability for Natural Language Processing @article{aacl2020xaitutorial, title={Explainability for Natural Language Processing}, author= {Dhanorkar, Shipi and Li, Yunyao and Popa, Lucian and Qian, Kun and Wolf, Christine T and Xu, Anbang}, journal={AACL-IJCNLP 2020}, year={2020} Presenter: Shipi Dhanorkar, Christine Wolf, Kun Qian, Anbang Xu, Lucian Popa and Yunyao Li Video: https://www.youtube.com/watch?v=3tnrGe_JA0s&feature=youtu.be Abstract: We propose a cutting-edge tutorial that investigates the issues of transparency and interpretability as they relate to NLP. Both the research community and industry have been developing new techniques to render black-box NLP models more transparent and interpretable. Reporting from an interdisciplinary team of social science, human-computer interaction (HCI), and NLP researchers, our tutorial has two components: an introduction to explainable AI (XAI) and a review of the state-of-the-art for explainability research in NLP; and findings from a qualitative interview study of individuals working on real-world NLP projects at a large, multinational technology and consulting corporation. The first component will introduce core concepts related to explainability in NLP. Then, we will discuss explainability for NLP tasks and report on a systematic literature review of the state-of-the-art literature in AI, NLP, and HCI conferences. The second component reports on our qualitative interview study which identifies practical challenges and concerns that arise in real-world development projects which include NLP.

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Keynote talk given at Pacific Asia Conference on Language, Information and Computation (PACLIC 34) on Pacific Asia Conference on Language, Information and Computation (PACLIC 34) on October 24, 2020. Title: Towards Universal Natural Language Understanding Abstract: Understanding the semantics of the natural language is a fundamental task in artificial intelligence. English semantic understanding has reached a mature state and successfully deployed in multiple IBM AI products and services, such as Watson Natural Language Understanding and Watson Compare and Comply. However, scaling existing products/services to support additional languages remain an open challenge. In this talk, we will discuss the open challenges in supporting universal natural language understanding. We will share our work in addressing these challenges in the past few years to provide the same unified semantic representation across languages. We will also showcase how such universal semantic understanding of natural languages can enable cross-lingual information extraction in concrete domains (e.g. insurance and compliance) and show promise towards seamless scaling existing NLP capabilities across languages with minimal efforts.

Towards Universal Language Understanding (2020 version)

Towards Universal Language Understanding (2020 version)

Towards Universal Language Understanding (2020 version)

Keynote talk at TextXD 2019(https://www.textxd.org) Abstract: Understanding the semantics of the natural language is a fundamental task in artificial intelligence. English semantic understanding has reached a mature state and successfully deployed in multiple IBM AI products and services, such as Watson Natural Language Understanding and Watson Compare and Comply. However, scaling existing products/services to support additional languages remain an open challenge. In this demo, we will present Polyglot, a multilingual semantic parser capable of semantically parsing sentences in 9 different languages from 4 different language groups into the same unified semantic representation. We will also showcase how such universal semantic understanding of natural languages can enable cross-lingual information extraction in concrete domains (e.g. insurance and compliance) and show promise towards seamless scaling existing NLP capabilities across languages with minimal efforts.

Towards Universal Semantic Understanding of Natural Languages

Towards Universal Semantic Understanding of Natural Languages

Towards Universal Semantic Understanding of Natural Languages

Poster corresponding to our NAACL'2015 paper "An In-depth Analysis of the Effect of Text Normalization in Social Media" Abstract: Recent years have seen increased interest in text normalization in social media, as the in-formal writing styles found in Twitter and other social media data often cause problems for NLP applications. Unfortunately, most current approaches narrowly regard the normalization task as a “one size fits all" task of replacing non-standard words with their standard counterparts. In this work we build a taxonomy of normalization edits and present a study of normalization to examine its effect on three different downstream applications (de- pendency parsing, named entity recognition, and text-to-speech synthesis). The results sug- gest that how the normalization task should be viewed is highly dependent on the targeted application. The results also show that normalization must be thought of as more than word replacement in order to produce results comparable to those seen on clean text. Paper: https://www.aclweb.org/anthology/N15-1045

An In-depth Analysis of the Effect of Text Normalization in Social Media

An In-depth Analysis of the Effect of Text Normalization in Social Media

An In-depth Analysis of the Effect of Text Normalization in Social Media

Slides for our COLING'18 paper: http://aclweb.org/anthology/C18-1058 Fundamental to several knowledge-centric applications is the need to identify named entities from their textual mentions. However, entities lack a unique representation and their mentions can differ greatly. These variations arise in complex ways that cannot be captured using textual similarity metrics. However, entities have underlying structures, typically shared by entities of the same entity type, that can help reason over their name variations. Discovering, learning and manipulating these structures typically requires high manual effort in the form of large amounts of labeled training data and handwritten transformation programs. In this work, we propose an active-learning based framework that drastically reduces the labeled data required to learn the structures of entities. We show that programs for mapping entity mentions to their structures can be automatically generated using human-comprehensible labels. Our experiments show that our framework consistently outperforms both handwritten programs and supervised learning models. We also demonstrate the utility of our framework in relation extraction and entity resolution tasks.

Exploiting Structure in Representation of Named Entities using Active Learning

Exploiting Structure in Representation of Named Entities using Active Learning

Exploiting Structure in Representation of Named Entities using Active Learning

Slides for our COLING'16 paper http://aclweb.org/anthology/C/C16/C16-1058.pdf Abstract: Semantic role labeling (SRL) is the task of identifying and labeling predicate-argument structures in sentences with semantic frame and role labels. A known challenge in SRL is the large number of low-frequency exceptions in training data, which are highly context-specific and difficult to generalize. To overcome this challenge, we propose the use of instance-based learning that performs no explicit generalization, but rather extrapolates predictions from the most similar instances in the training data. We present a variant of k-nearest neighbors (kNN) classification with composite features to identify nearest neighbors for SRL. We show that high-quality predictions can be derived from a very small number of similar instances. In a comparative evaluation we experimentally demonstrate that our instance-based learning approach significantly outperforms current state-of-the-art systems on both in-domain and out-of-domain data, reaching F1-scores of 89,28% and 79.91% respectively

K-SRL: Instance-based Learning for Semantic Role Labeling

K-SRL: Instance-based Learning for Semantic Role Labeling

K-SRL: Instance-based Learning for Semantic Role Labeling

Coling poster

Coling demo

Slides deck for SIGMOD 2017 Tutorial. ABSTRACT: The volume of natural language text data has been rapidly increasing over the past two decades, due to factors such as the growth of the Web, the low cost associated to publishing and the progress on the digitization of printed texts. This growth combined with the proliferation of natural language systems for search and retrieving information provides tremendous opportunities for studying some of the areas where database systems and natural language processing systems overlap. This tutorial explores two more relevant areas of overlap to the database community: (1) managing natural language text data in a relational database, and (2) developing natural language interfaces to databases. The tutorial presents state-of-the-art methods, related systems, research opportunities and challenges covering both area.

Natural Language Data Management and Interfaces: Recent Development and Open ...

Natural Language Data Management and Interfaces: Recent Development and Open ...

Natural Language Data Management and Interfaces: Recent Development and Open ...

SystemT: Declarative Information Extraction

SystemT: Declarative Information Extraction

SystemT: Declarative Information Extraction

Tyler Baldwin, Yunyao Li, Bogdan Alexe, Ioana Roxana Stanoi: Automatic Term Ambiguity Detection. ACL (2) 2013: 804-809 Abstract: While the resolution of term ambiguity is important for information extraction (IE) systems, the cost of resolving each instance of an entity can be prohibitively expensive on large datasets. To combat this, this work looks at ambiguity detection at the term, rather than the instance, level. By making a judgment about the general ambiguity of a term, a system is able to handle ambiguous and unambiguous cases differently, improving through-put and quality. To address the term ambiguity detection problem, we employ a model that combines data from language models, ontologies, and topic modeling. Results over a dataset of entities from four product domains show that the proposed approach achieves significantly above baseline F-measure of 0.96.

Automatic Term Ambiguity Detection

Automatic Term Ambiguity Detection

Automatic Term Ambiguity Detection

Mais de Yunyao Li (20)

The Role of Patterns in the Era of Large Language Models

The Role of Patterns in the Era of Large Language Models

The Role of Patterns in the Era of Large Language Models

Building, Growing and Serving Large Knowledge Graphs with Human-in-the-Loop

Building, Growing and Serving Large Knowledge Graphs with Human-in-the-Loop

Building, Growing and Serving Large Knowledge Graphs with Human-in-the-Loop

Meaning Representations for Natural Languages: Design, Models and Applications

Meaning Representations for Natural Languages: Design, Models and Applications

Meaning Representations for Natural Languages: Design, Models and Applications

Taming the Wild West of NLP

Taming the Wild West of NLP

Taming the Wild West of NLP

Towards Deep Table Understanding

Towards Deep Table Understanding

Towards Deep Table Understanding

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Human in the Loop AI for Building Knowledge Bases

Human in the Loop AI for Building Knowledge Bases

Human in the Loop AI for Building Knowledge Bases

Towards Universal Language Understanding

Towards Universal Language Understanding

Towards Universal Language Understanding

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Explainability for Natural Language Processing

Towards Universal Language Understanding (2020 version)

Towards Universal Language Understanding (2020 version)

Towards Universal Language Understanding (2020 version)

Towards Universal Semantic Understanding of Natural Languages

Towards Universal Semantic Understanding of Natural Languages

Towards Universal Semantic Understanding of Natural Languages

An In-depth Analysis of the Effect of Text Normalization in Social Media

An In-depth Analysis of the Effect of Text Normalization in Social Media

An In-depth Analysis of the Effect of Text Normalization in Social Media

Exploiting Structure in Representation of Named Entities using Active Learning

Exploiting Structure in Representation of Named Entities using Active Learning

Exploiting Structure in Representation of Named Entities using Active Learning

K-SRL: Instance-based Learning for Semantic Role Labeling

K-SRL: Instance-based Learning for Semantic Role Labeling

K-SRL: Instance-based Learning for Semantic Role Labeling

Coling poster

Coling demo

Natural Language Data Management and Interfaces: Recent Development and Open ...

Natural Language Data Management and Interfaces: Recent Development and Open ...

Natural Language Data Management and Interfaces: Recent Development and Open ...

SystemT: Declarative Information Extraction

SystemT: Declarative Information Extraction

SystemT: Declarative Information Extraction

Automatic Term Ambiguity Detection

Automatic Term Ambiguity Detection

Automatic Term Ambiguity Detection

Último

The microservices honeymoon is over. When starting a new project or revamping a legacy monolith, teams started looking for alternatives to microservices. The Modular Monolith, or 'Modulith', is an architecture that reaps the benefits of (vertical) functional decoupling without the high costs associated with separate deployments. This talk will delve into the advantages and challenges of this progressive architecture, beginning with exploring the concept of a 'module', its internal structure, public API, and inter-module communication patterns. Supported by spring-modulith, the talk provides practical guidance on addressing the main challenges of a Modultith Architecture: finding and guarding module boundaries, data decoupling, and integration module-testing. You should not miss this talk if you are a software architect or tech lead seeking practical, scalable solutions. About the author With two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Dubai, known for its towering skyscrapers, luxurious lifestyle, and relentless pursuit of innovation, often finds itself in the global spotlight. However, amidst the glitz and glamour, the emirate faces its own set of challenges, including the occasional threat of flooding. In recent years, Dubai has experienced sporadic but significant floods, disrupting normalcy and posing unique challenges to its infrastructure. Among the critical nodes in this bustling metropolis is the Dubai International Airport, a vital hub connecting the world. This article delves into the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Retrieval augmented generation (RAG) is the most popular style of large language model application to emerge from 2023. The most basic style of RAG works by vectorizing your data and injecting it into a vector database like Milvus for retrieval to augment the text output generated by an LLM. This is just the beginning. One of the ways that we can extend RAG, and extend AI, is through multilingual use cases. Typical RAG is done in English using embedding models that are trained in English. In this talk, we’ll explore how RAG could work in languages other than English. We’ll explore French, Chinese, and Polish.

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Introduction to Multilingual Retrieval Augmented Generation (RAG)

[BuildWithAI] Introduction to Gemini.pdf

[BuildWithAI] Introduction to Gemini.pdf

[BuildWithAI] Introduction to Gemini.pdf

MINDCTI Revenue Release Quarter One 2024

MINDCTI Revenue Release Quarter One 2024

MINDCTI Revenue Release Quarter One 2024

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

Why Teams call analytics are critical to your entire business

Why Teams call analytics are critical to your entire business

Architecting Cloud Native Applications

Architecting Cloud Native Applications

Architecting Cloud Native Applications

Angeliki Cooney has spent over twenty years at the forefront of the life sciences industry, working out of Wynantskill, NY. She is highly regarded for her dedication to advancing the development and accessibility of innovative treatments for chronic diseases, rare disorders, and cancer. Her professional journey has centered on strategic consulting for biopharmaceutical companies, facilitating digital transformation, enhancing omnichannel engagement, and refining strategic commercial practices. Angeliki's innovative contributions include pioneering several software-as-a-service (SaaS) products for the life sciences sector, earning her three patents. As the Senior Vice President of Life Sciences at Avenga, Angeliki orchestrated the firm's strategic entry into the U.S. market. Avenga, a renowned digital engineering and consulting firm, partners with significant entities in the pharmaceutical and biotechnology fields. Her leadership was instrumental in expanding Avenga's client base and establishing its presence in the competitive U.S. market.

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Angeliki Cooney

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

presentation ICT roal in 21st century education

presentation ICT roal in 21st century education

presentation ICT roal in 21st century education

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Rustici Software

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Understanding the FAA Part 107 License ..

Understanding the FAA Part 107 License ..

Understanding the FAA Part 107 License ..

Christopher Logan Kennedy

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Último (20)

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Introduction to Multilingual Retrieval Augmented Generation (RAG)

Introduction to Multilingual Retrieval Augmented Generation (RAG)

[BuildWithAI] Introduction to Gemini.pdf

[BuildWithAI] Introduction to Gemini.pdf

[BuildWithAI] Introduction to Gemini.pdf

MINDCTI Revenue Release Quarter One 2024

MINDCTI Revenue Release Quarter One 2024

MINDCTI Revenue Release Quarter One 2024

Why Teams call analytics are critical to your entire business

Why Teams call analytics are critical to your entire business

Why Teams call analytics are critical to your entire business

Architecting Cloud Native Applications

Architecting Cloud Native Applications

Architecting Cloud Native Applications

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

How to Troubleshoot Apps for the Modern Connected Worker

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

Artificial Intelligence Chap.5 : Uncertainty

presentation ICT roal in 21st century education

presentation ICT roal in 21st century education

presentation ICT roal in 21st century education

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Apidays New York 2024 - The value of a flexible API Management solution for O...

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Corporate and higher education May webinar.pptx

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Strategies for Landing an Oracle DBA Job as a Fresher

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Understanding the FAA Part 107 License ..

Understanding the FAA Part 107 License ..

Understanding the FAA Part 107 License ..

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Rule-based Information Extraction is Dead! Long Live Rule-based Information Extraction Systems!

1. Rule-based Information Extraction is DEAD Long Live Rule-based Information Extraction Systems! Laura Chiticariu, Yunyao Li, Frederick Reiss IBM Research - Almaden THE DISCONNECT: ACADEMIC vs. INDUSTRY Implementations of Entity Extraction Entity Extraction Papers by Year 3.5% 21% 100% RuleBased Hybrid 45% 50% RuleBased 22% 75% 17% Hybrid 17% Machine Learning Based 33% 0% NLP Papers (2003-2012) All Vendors Large Vendors Machine Learning Based Fraction of NLP Papers 67% Commercial Products (2013) Year of Publication THE EXPLANATIONS Academia Rule-based IE PROs •Declarative Heuristic •Easy to comprehend •Easy to maintain •Easy to incorporate domain knowledge •Easy to debug ML-based IE PROs •Trainable •Adaptable •Reduces manual effort CONs CONs • Heuristic •Requires tedious manual labor Industry •Requires labeled data •Requires retraining for domain adaptation •Requires ML expertise to use or maintain • Opaque Evaluating Benefits Evaluating IE on its own of IE Precision and Recall Evaluating Costs of IE Labor cost of writing rules Evaluating IE as part of a larger process Using ill-defined metrics that are subject to change Labor cost Hardware cost Business risk Others What’s the research in Rule-based IE? BRIDGING THE GAP Where is the research in rule-based IE? Making it more principled, effective, and efficient Define standard IE rule language and data model. • What is the right data model to capture text, annotations over text, and their properties? • Can we establish a standard declarative extensible rule language to solve most IE tasks encountered so far? Systems research based on standard IE rule language. • Data representation • Automatic performance optimization • Exploring modern hardware … ML research based on standard IE rule language • How to learn basic primitives such as regular expressions and dictionaries? • How to automatically generate rules that are understandable and maintainable?