Presented by David Piraino, Chief Imaging Information Officer, Imaging Institute Cleveland Clinic, Cleveland Clinic & Daniel Palmer, Chief Imaging Information Officer, Imaging Institute Cleveland Clinic, Cleveland Clinic Most patient specifc medical information is document oriented with varying amounts of associated meta-data. Most of pateint medical information is textual and semi-structured. Electronic Medical Record Systems (EMR) are not optimized to present the textual information to users in the most understandable ways. Present EMRs show information to the user in a reverse time oriented patient specific manner only. This talk discribes the construction and use of Solr search technologies to provide relevant historical information at the point of care while intepreting radiology images. Radiology reports over a 4 year period were extracted from our Radiology Information System (RIS) and passed through a text processing engine to extract the results, impression, exam description, location, history, and date. Fifteen cases reported during clinical practice were used as test cases to determine if ""similar"" historical cases were found . The results were evaluated by the number of searches that returned any result in less than 3 seconds and the number of cases that illustrated the questioned diagnosis in the top 10 results returned as determined by a bone and joint radiologist. Also methods to better optimize the search results were reviewed. An average of 7.8 out of the 10 highest rated reports showed a similar case highly related to the present case. The best search showed 10 out of 10 cases that were good examples and the lowest match search showed 2 out of 10 cases that were good examples.The talk will highlight this specific use case and the issues and advances of using Solr search technology in medicine with focus on point of care applications.

1. Next Generation Electronic Medical Records and
Search: A Test Implementation in Radiology
David Piraino,MD Daniel Palmer, PhD
Cleveland Clinic John Carroll University

2. Introduction
• Most patient specific medical information is document oriented
with varying amounts of meta-data.
• Most of patient medical information is textual and semi-structured.
• Electronic Medical Record Systems (EMR) are not optimized to
present textual information
• EMRs currently show information in reverse time order only.
• This talk describes the construction and use of Solr search
technologies to provide relevant historical information at the point
of care while interpreting radiology images.

3. Grand challenges (2008)
in clinical decision support
• Improve the human–computer interface
• Disseminate best practices in CDS design, development, and implementation
• Summarize and prioritize patient-level information
• Prioritize and filter recommendations to the user
• Create an architecture for sharing executable CDS modules and services
• Combine recommendations for patients with co-morbidities
• Prioritize CDS content development and implementation
• Create internet-accessible clinical decision support repositories
• Use free text information to drive clinical decision
support
• Mine large clinical databases to create new CDS
Dean F. Sittig et al, Journal of Biomedical Informatics 41 (2008) 387–392

4. Too Much Information (2012)
• In the time-pressured clinical setting,
clinicians faced with large amounts of patient
data in formats that are not readily
interpretable often feel ‘information
overload’.
Ketan Mane et al, Journal of Biomedical Informatics 45(2012) 101-106

5. What is out of place?
• Blue
• Green
• Cleveland
• Red
• Yellow

6. What is out of place?
• Boston
• new york
• Cleveland
• Chicago
• Denver
• San Diego
• atlanta
• Toronto
• Mexico City
• Columbus
• Nashville
• Paris
• Seattle
• Vancouver
• Washington DC
• Miami
• dallas
• Houston

7. Large number of images,
varying levels of
applicability, incomplete
histories, data stored in
many different locations
Chaos in Primary Care(2011)
Information
Overload
Information
Scatter
Unrelated
Information
Mental
Workload
Situation
Awareness
Further Cognitive Influences
Problem solving
Problem identification
Decision making
Diagnosis
Treatment
Moderators
Interruptions
Expertise
Time
Information Chaos in Primary Care: Implications for Physician
Performance and Patient Safety
John W Beasley, MD1,2, Tosha B. Wetterneck, MD, MS3, Jon Temte, MD, PhD1, Jamie A
Lapin, MS2, Paul Smith, MD1, A. Joy Rivera-Rodriguez, MS2, and Ben-Tzion Karsh, PhD*,1,2
Journal of the American Board Family Medicine. 2011 November; 24(6): 745–751
1Department of Family Medicine, UW-Madison School of Medicine and Public Health
2Department of Industrial and Systems Engineering, UW-Madison
3Department of Medicine, UW-Madison School of Medicine and Public Health

8. Existing Information Confusion
ED visit
Telephone
Office
ED
Office
Admission
Surgery
Optho
ED visit
Telephone
Office
ED
Office
Admission
Surgery
Optho
Labs
CBC
PSA
Glucose
Potassium
Glucose
Urinalysis
Patient Image history presented as a list
Key components missing

9. Inspiration
• Boston 2012
And we hope to be inspired again this week with your help

10. Warning 28 Days Later
• One person with other full time job
• Running on moderately high end workstation
• Indexed 7 million radiology reports
• Providing types of searches that would
otherwise be “impossible”

12. MRI shoulder without contrast
Relevant Previous Reports

13. MRI shoulder without contrast
There is evidence for a full thickness tear of the supraspinatus tendon
Updated relevance

14. MRI shoulder without contrast
There is evidence for a full thickness tear of the supraspinatus tendon
There is a partial tear of the subscapularis tendon with anterior medial
dislocation of the long head of the biceps tendon
Additional Update to Relevance

15. Evaluation
• 15 cases reported during clinical practice were used as test
cases to determine if "similar" historical cases were found.
• For these 15 cases all searches completed within 3 seconds
• Considered only the top 10 matches returned by search
• Number of cases that illustrated the questioned diagnosis as
determined by a bone and joint radiologist.

16. Results for the 15 cases
• Average performance:
– 7.8 out of the 10 highest rated reports showed a
similar case highly related to the present case.
• Best performance:
– 10 out of 10 cases relevant
• Worst performance:
– only 2 out of 10 cases relevant

17. In Practice
• An example case:
– Medical image: vascular mass in the hand
– LucidWorks search considered first 10 results
• Based on text, eliminated unrelated cases
– Found and studied 2 pertinent cases
• Showed similar masses with similar uncertainty
• Used to generate data sets for other research
projects

18. Input Flow
Input Stream
HL7 stream
or
Delimited File
Solr XML
with
new
fields
Solr
Index
and
repositoryPreprocess
algorithm
Solr processing

19. Input Stream (HL7 Protocol)
XXXX|Date|XXX-01-01
|XXXX|XX:17:00.0|14||XXX-XXX-RADIOLOGY-CCF|XXX|XXX|CCF|I|XXXX|LMBR
|XXXX|A|MRA OF HEAD|MR||||||* * *Final Report* * * DATE OF EXAM: XXXXX
12:07AM LMM 0432 - MRA OF HEAD /
ACCESSION # XXXXX PROCEDURE REASON: cva
* * * * Physician Interpretation * * * * RESULT: MRA OF THE HEAD WITHOUT CONTRAST
HISTORY: Subarachnoidxxxx TECHNIQUE: Time of flight MRA of the cervical circulation was
performed. COMPARISON: none FINDINGS: Examination is xxxxxxxx. IMPRESSION: Small
xxxxxxxx. Transcriptionist: PSC Transcribe Date/Time: Jan 1 XXXX 10:14P Dictated by :
XXXXXX, MD This examination was interpreted and the report reviewed and electronically
signed by: XXXXX, MD On Date|

20. <add>
<doc>
<field name="department">Radiology</field>
<field name="category">report</field>
<field name="pid">EXXXXXX</field>
<field name="sex">Male</field>
<field name="id">XXXXX</field>
<field name="did">XXXXX</field>
<field name="modality">CT</field>
<field name="title">MRI of the HEAD</field>
<field name="date">XXX-01-09T09:34:00Z</field>
<field name="year">XXX</field>
<field name="month">01</field>
<field name="day">09</field>
<field name="hour">09</field>
<field name="history">Subarachnoidxxxx</field>
<field name="site">WRC</field>
<field name="physician">XXXXX</field>
<field name="body"> On the head XXXXXXXXXX on the base of the neck. </field>
<field name="impression"> 1. XXXX. 2. XXXXXXX. 3. XXXXXXXX </field>
<field name=“positive">XXXXXXXX</field>
<field name=“negative">XXXX</field>
<field name=“neutral">XXXX</field>
<field name=“anatomy”>skull</field>
<field name=“side”>none</field>
</doc>
</add>
Solr Input XML stream

21. Search Flow
Extracted text
Solr
Query
Relevant
Documents
Preprocess
algorithm
Query Solr
Clinical encounter Radiology report
Data Extractor Data Extractor
More information
Processed Text
Solr query
constructor

22. Similar Imaging Diagnosis
Patient: Anatomy, Modality, Diagnosis, and Time
Patient
Pathology
Lab
Patient
Clinical
notes
(Provider
Diagnosis
Time)
Speculative
Interface

23. Solr – current
implementation
Similar Imaging Diagnosis
Patient: Anatomy, Modality, Diagnosis, and Time
Patient
Pathology
Lab
Patient
Clinical
notes
(Provider
Diagnosis
Time)

24. ImageSphere

25. Challenges to Building Prototype
• Time vs. Data
• Sensitivity of queries
• Automating human scan/evaluation step
• Lack of a non-radiologist fitness function
• Migration from development-only LucidWorks
platform to embedded Solr API queries

26. Time vs. Data
• 2-3 cases max viewed
(10 considered)
• High relevance required
• Potentially 10’s of
thousands to select from

27. Sensitivity of Queries
• Many query parameters
– proximity, boost, not
• Yields range of results
– 10/10 through 0/10
• 2 orders of magnitude in
query times
• (wrist fracture)
• (wrist fracture)~2
• (wrist fracture)~10
• wrist^3 fracture
• -(no near fracture)

28. Queries: Good News/Bad News
• Basic queries provide great results
– Better than expected
– Top 10 results quickly yield cases to view
• Query refinement proves to be difficult
– Little or no correlation between query
modifications and changes in results
– No consistent direction to investigate

29. Human in the Loop
• Top 10 results displayed in text form
• Human quickly scans and selects best
• Must maintain this ability in visual GUI
• Evaluation difficult because…

30. Fitness Function == Radiologist
• Need expert to determine value of query
results
• Large impact on debugging…
• “Live” statistics gathering and provisional data
gathering techniques

31. Migration for Prototype
• Manual process using LucidWorks proved
concept
• Use Solr API to implement an automated
delivery/display system
• Dependent on an intuitive user interface

32. Thank You and…
any Answers?

33. CONFERENCE PARTY
The Tipsy Crow: 770 5th Ave
Starts after Stump The Chump
Your conference badge gets
you in the door
TOMORROW
Breakfast starts at 7:30
Keynotes start at 8:30
CONTACT (optional)
David Piraino MD
piraind@ccf.org