Creating Simple Web Text for People with Intellectual Disabilities and to Train Artificial Intelligence

Creating Simple Web Text for
People with ID and to Train AI
John Rochford
Director, INDEX Program
Eunice Kennedy Shriver Center
Invited Expert
World Wide Web Consortium

INDEX Staff: Representing the Diversity of our Community

Cognitive Disabilities
• Clinical Examples
•intellectual disabilities, Autism/ASD, learning disabilities,
TBI, Schizophrenia, Alzheimer’s, Depression, aging, etc.
•Functional Examples
•cannot understand typical text
•are easily distracted
•are confused by inconsistent interfaces
•do not recover easily from errors

Cognitive Web Accessibility Examples, 1 to 3
Deliver content in more than one way.
Give guided help.
Work well with assistive technology.

Cognitive Web Accessibility Examples, 4 to 6
Focus attention.
Augment text with relevant images.
Use simple language.

Web Text Simplification Goal
Make Web text so
simple it can be
understood the first
time it is read.

Simplified Text Example
Typical: “When the process of freeing a vehicle that has
been stuck results in ruts or holes, the operator will fill
the rut or hole created by such activity before removing
the vehicle from the immediate area.”
Simple: “If you make a hole while freeing a stuck vehicle,
you must fill the hole before you drive away.”
Source

Simple Web Text Examples
Simple Wikipedia
Easy To Read News

2 Ways We Are Simplifying Text
Short Term: Operationalize
plain language guidelines.
Long Term: Develop
Artificial Intelligence
capabilities.

Simplifying Web Text: Plain Language
Make plain-language guidelines
easy to follow to simplify text.
Make
Test if simplified text is more
understandable than typical text.
Test
Publish techniques for people to
simplify text easily and reliably.
Publish

Operationalize Plain Language Guidelines: Steps 1 & 2
Gathered 33 plain language guidelines.
Narrowed to 19 parsable by algorithms.

Operationalize Plain Language Guidelines: Steps 3 & 4
Refined to 6 that consistently reduced
reading grade levels for 4 experimenters.
Conducted trials to determine the order
they should be applied.

Plain Language Guidelines In Order: 1st Set
1. Use short, simple words.
2. Write short sentences.
3. Avoid using acronyms and abbreviations.

Plain Language Guidelines In Order: 2nd Set
4. Use an active voice in the present tense.
5. Use correct grammar and spelling.
6. Remove proper nouns.

Example Operationalization
Rule 1: Use short, simple words.
Replace 3+ syllable words with words that:
• have 1 or 2 syllables; and
• are commonly used.

Study Participants
25 adults with ID
Tested at 3rd to 5th grade
17 to 63 years of age
Paid for their time

Text Passages, Questions, Answers
• 8 Collected Text Passages
• Simplified 4 Text Passages
• 2 Literal Questions
• 2 Inferential Questions
• 3 multiple-choice answers

Sample Typical Passage Webpage
Lux Level is a luxurious, in-theatre dining
experience at select theatres. Movie-goers can
indulge themselves with premium reserved-
seating, in-seat dining throughout the show, as well
as other special amenities. Each seat is equipped
with a server call button so your server is always
there when you need them. This truly is the finest
movie-going experience available today.

Sample Typical Passage, Literal Question Webpage
Lux Level is a luxurious, in-theatre
dining experience at select theatres.
Movie-goers can indulge themselves
with premium reserved-seating, in-
seat dining throughout the show, as
well as other special amenities. Each
seat is equipped with a server call
button so your server is always there
when you need them. This truly is
the finest movie-going experience
available today.
Where can Lux Level be
found?
1. At all theatres.
2. At a few theatres.
3. At no theatres yet.

Sample Passage To Be Simplified
On Miniclip you can play games to compile games
statistics such as high scores, and rankings on your
own player page and can receive awards for playing
games. You can also create your own Miniclip avatar
with clothing and accessories to represent you in
our fun Miniclip games. You can play single player
games or play advanced massive multiplayer games
with other players from around the world.

Sample Simplified Passage Webpage
On Miniclip, you can play games to collect game stats.
These can be high scores and rankings on your own
player page. You can get awards for playing games. You
can also create your own Miniclip avatar with clothing
and extras. It can be you in our fun Miniclip games. You
can play single player games. Or you can play advanced,
huge games with many players. Those players live
around the world.

Sample Simple Passage, Inferential Question Webpage
On Miniclip, you can play games to
collect game stats. These can be high
scores and rankings on your own player
page. You can get awards for playing
games. You can also create your own
Miniclip avatar with clothing and extras.
It can be you in our fun Miniclip games.
You can play single player games. Or you
can play advanced, huge games with
many players. Those players live around
the world.
How can you play with people
who live far away?
1. Pick an advanced game.
2. Pick a single player game.
3. Pick an award.

Study Results: 3 Subgroups
60%
70%
80%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Very Poor Poor to Fair Good to Very Good
% Correct Answers – Typical Text Passages

Study Results: Simplified Text By Subgroup

Study Results: All Participants: Answers
13%
23%
Inferential Questions
Literal Questions
% Improvement of Simple- Over
Typical-Text Passage Answers

Study Results: Overall Comprehension Improvement
18%

Reason # 1 Why Study Result Is Important
Reasonable assumption: people with higher
levels of executive functioning will also
benefit from text simplification.
Examples: people with learning disabilities,
functional illiteracy, non-native language.

Reason # 2 Why Study Result Is Important
Parallel between people with ID and AI: both have a
very-limited understanding of the world.
Despite that, operationalized plain language guidelines
help people with ID understand simplified text.
We build them into algorithms to help AI “understand”
simplified text sufficiently to recognize and create it.

Knowledge Representation And Reasoning
A knowledge base has facts about the world.
AI can reason about those facts.
AI can deduce new facts.

AI Training Data Sources And Preprocessing
Sentence Pairs (Typical and Simple)
•manually matched from Wikipedia and Simple Wikipedia;
•automatically matched with algorithms; and/or
•matched or created by crowdsourcing.
Data Preprocessing
•Open-source tools with sample data sets.
•Examples: Google word2vec, Stanford NLTK.

Artificial Intelligence
Pattern Recognition
Picture Source

This Person Does Not Exist Dot Com

We Must Not Teach AI
Our Biases
1 in 4 U.S. adults have a disability.
• Accurate distribution in data sets
• Self-driving cars example (source: Jutta)
Example Barriers to Self Identification
• Insurance higher cost or denial
• Employment and housing discrimination
IBM AI Fairness Toolkit on GitHub Image Source

Partners
•Amazon Machine
Learning Research
Awards
• IBM Watson
•UMass Boston
•Worcester
Polytechnic
University

Thank You!
•John Rochford
•John.Rochford@umassmed.edu
•Twitter: @ClearHelper
•Slide Deck: bit.ly/CSUN2019

Creating Simple Web Text for People with Intellectual Disabilities and to Train Artificial Intelligence

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Creating Simple Web Text for People with Intellectual Disabilities and to Train Artificial Intelligence

Semelhante a Creating Simple Web Text for People with Intellectual Disabilities and to Train Artificial Intelligence (20)

Último

Último (20)

Creating Simple Web Text for People with Intellectual Disabilities and to Train Artificial Intelligence