9. What I mean by “task”
Information Multiple Evolving
seeking occurs interactions information
over time with the system needs
Human Computer
10. Two kinds of feedback
Person System
Person trains system to find documents
System Person
System indicates possibilities to guide
person
11. Person System
People don’t use relevance feedback, right?
They do when suitably motivated.
Two examples:
Ancestry.com
Predictive coding
12. Relevance feedback at Ancestry.com
Search
People find historical records about specific individuals
Facts from records are saved to individuals in family trees
Relevance feedback
Saved facts are automatically incorporated into subsequent
queries
Relevance feedback is inferred from saved records
Many motivated users
Hundreds of hours of system use
Lots of interaction
Person System
13. Relevance Feedback in
Predictive Coding
Predictive coding is a technique for training a classifier to find
relevant documents
Used in e-discovery to increase accuracy and reduce costs
Machine learning algorithm is trained through hundreds of
relevance judgments; applied to millions of documents
Big (and getting bigger) business
Person System
14. System Person
System provides hints about potential actions
Information scent
Which documents are new
Which terms are effective
Ways to expand/reformulate the query
Examples
Facets indicating numbers of matching documents
Previously-saved or viewed documents
History of queries, related queries
Previews, hints, etc.
15. Interacting with the past
Previously
saved
records for
this person
Ancestry.com
System Person
19. Interacting with the future
Query preview Query nudges
As searcher types, shows As searcher types, changes
distribution of new vs. re- halo color to encourage longer
retrieved documents in a queries
search session
7
6
5
Number of Query Terms
4
3
2
No instr.
1
0 Instruction
No halo System Person
Halo
20. Design Challenges
How do we get people to use
relevance feedback?
How do we help people discover
which queries will be effective?
How do we help people plan?