Authors: Isabel Segura-Bedmar, Paloma Martínez, Daniel Sánchez-Cisneros. DDI Extraction 2011: Drug-Drug Interaction Extraction 2011, Huelva, España (September 5, 2011) Lessons from the Drug-Drug Interaction Extraction Task

1. DDIExtraction 2011
Lessons from the Drug-Drug Interaction
Extraction Task
Huelva, September, 7th
Isabel Segura-Bedmar, Paloma Martinez, Daniel Sánchez-
Cisneros, LABDA, UC3M
http://labda.inf.uc3m.es/

2. Drug-Drug Interactions (DDIs)

A DDI occurs when one drug influences the
level or activity of another drug.

DDIs are a serious problem for patient safety.

Overwhelming amount of information available
on DDIs: databases, journals, technical reports,
books, etc.

Information Extraction (IE) can provide an
interesting way of reducing the time spent by
health care professionals on reviewing the
literature.

3. DDIExtraction Task

Automatic extraction of drug-drug
interactions from text

Main goals:

To produce useful resources for training
and testing.

To learn which approaches are
successful and practical.

To encourage in developing useful tools
to extract DDIs from texts.

4. DDIExtraction Task
http://labda.inf.uc3m.es/DDIExtraction2011

5. Summary of Approaches
• Main approaches at the top level
– Markov
– Support Vector Machine (SVM)
• Typically with a small window around the word of interest
– Rules: typically manually generated?
• Also may have pre- or post-processing in
addition to the main approach
• Lots of different features used
– Some features need entire other sub-systems to
obtain a value
• E.g., a part-of-speech (POS) tagger
• The other sub-system may use an approach different
from the main one

6. Summary of Results: Comments
• Like many similar tasks, task 1A has its share of
unique aspects
– The 1-2 teams that did not really train against the
training data did not do as well
• Results for open and closed submissions are
typically fairly close
• At the higher scores
• Between the submissions from the same user

7. DDIExtraction 2011
Closing remarks
Huelva, September, 7th
Isabel Segura-Bedmar, Paloma Martinez, Daniel Sánchez-
Cisneros, LABDA, UC3M
http://labda.inf.uc3m.es/

8. DDIExtraction 2011 Task

Pilot task.

Several limitations:

Drug names were automatically
annotated.

Annotation process performed by one
unique annotator.

Many annotation errors.

9. DDIExtraction Task in SEMEVAL
2013
http://www.cs.york.ac.uk/semeval/proposal-16.html

10. DDIExtraction Task in SEMEVAL
2013

New train and test datasets: unified format.

Identify drug-drug interaction pairs from full
text articles (HTML, PDF).

Drugs and drug-drug interactions manually
annotated by two different pharmacists.

Calculate Inter-annotator agreement (IAA).

11. DDIExtraction Task in SEMEVAL
2013

September 7, 2011 Call for participation

April 10, 2012 Full Training Data available for
participants

January 1, 2013 Start of evaluation period

February 1, 2013 End of Evaluation Period

March 1, 2013 Paper submission deadline

Summer 2013 Workshop co-located with ACL or
NAACL

12. Acknowledgments

DrugBank for providing the training and test
data collections.

Manuel Maña for collaboration in organizing the
DDIExtraction task.

Thanks to the PC members for their invaluable
help.

Thanks to the participating teams for their effort
in developing the participating systems and
improving the quality of the datasets.

13. Thanks!!!
See you on SEMEVAL 2013!!!