This document proposes a model for assessing the quality of knowledge used in clinical decision support systems. It discusses issues with current CDSSs, such as outdated or incomplete knowledge influencing decision making. The model uses Semantic Web technologies to discover high quality knowledge from different sources based on metrics like accuracy, reliability and relevance. Knowledge brokers would apply these metrics to evaluate knowledge and ensure only high quality sources are used to support clinical decisions.

1. A Model of Knowledge quality assessment in
clinical decision support system
20 October 2015
Seyedjamal Zolhavarieh
Dave Parry
HiNZ conference 2015 , Wigram Air Force Museum , Christchurch

2. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Outline
 Rationale and significance of the study
 Covered domains in this research
 Clinical Decision Support System (CDSS)
 Previous researches in CDSS
 Semantic-Web based CDSSs
 Knowledge Quality issue
 Quality metrics
 Proposed model
 Why Semantic Web?

3. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Rationale and significance of the study
 Decision making in the healthcare domain is an essential activity for clinicians. The expert knowledge plays an
important role in decision making. However, the expert may make a wrong decision.
 The health informatics researchers have developed Clinical Decision Support System (CDSS) to help for decision
making
Figure 1_The structure of CDSS
 CDSS is computer-based toolkit that provides a decision for different clinical goals such as diagnosis and treatment.
 Out of date, limited or uncompleted knowledge in CDSS can have negative influence on decision making.

4. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Rationale and significance of the study (cont.)
 The knowledge used in CDSS must be both up to date and relevant for the cases
that are being presented to it.
 However finding the latest accurate clinical knowledge to support clinical
decision making is difficult:
 The knowledge is changing rapidly.
 The knowledge might be located in many different repositories in different formats.
 The range of knowledge required in a particular case might be very wide especially in the case
of co-morbidities.
 Issues:
 Whether the CDSS can cope with rare or unusually presenting diagnoses ?
 How to make sure that the knowledge provided by CDSS are reliable ?
This research proposes a framework to discover high quality knowledge
in terms of relevancy and accuracy from different clinical knowledge
sources using Semantic Web (SW) technologies.
The main objective:
To assess and improve the quality of extracted knowledge for decision
making process.

5. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Covered domains in this research
CDSS
•Representing the knowledge
from different sources for
decision making
•Collaborating among CDSSs
•Transforming knowledge
SW
technology
•Machine-understandable
format
•Helps to find related
information
Knowledge
Quality
•Provide a guarantee for decision
making
•Check the validity of extracted
knowledge
•Reduce the number of mistakes in
decision making process

6. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Clinical Decision Support System (CDSS)
 CDSS can be described by five right things: 1) providing the right information, 2) to the right person, 3) in the right
format, 4) via the right channel, 5) at the right point

7. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Previous researches in CDSS
Format heterogeneity
Inconsistency in heterogeneity data model
Lack of semantic definition
Lack of semantic meaning and relationships between information from
different sources
Data heterogeneity and lack of data integration
Redundancy in a patient’s health record since one patient may visit different
healthcare institutes
Requiring information from another health institute
Lack of automatic analysis system
Weak semantic infrastructure
Semantic interoperability

8. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Semantic-Web based CDSSs
 COCOON
 ARTEMIS
 Knowledge-Centric CDSS
 A knowledge engineering approach for detecting Alzheimer disease (AD)
 Semantic-DB
 Semantic-CT
All of these works are proposed to support decision-making process, however,
the quality of extracted knowledge has not been addressed.

9. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Knowledge Quality issue
 Regarding ever growing amount of clinical knowledge.
 it is hard to identify high quality knowledge
 The clinical knowledge should be accurate and up-to-date in the various
resources.
 because the performance of decision making in CDSS is depended on the quality of
extracted knowledge.
 Most of the metrics of knowledge quality are related to structure of information
that is the backbone of constructed knowledge.
 It is not reasonable that the knowledge with high-quality structure is automatically
relevant and accurate for decision making.
 The different structures provide different decision.
Therefore, it is necessary to have a system to check the quality of knowledge
with its relevancy and accuracy to enhance human decision making.

10. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Knowledge Quality issue
 An example
 The aim is to find knowledge that shows related diseases. The following table presents the characteristics of disease.
Causative agent Bacteria
Finding Structure Joint structure
Associated Morphology Inflammation
Figure 2_Query in Snow Owl platform

11. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Knowledge Quality issue
 An example
 the query results (knowledge ontologies)
(a) SNOMED-CT
(d) ICD-10
(b) CTV3
(c) RADLEX
(e) AI-RHEUM
Figure 3_The query result for different terminologies (represented in ontology)

12. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Knowledge Quality issue
 A same query tried in the PubMed browser
 How can practitioners be sure they are getting the most relevant and accurate result ?
Figure 4_sample query and result in PubMed

13. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Quality metrics

14. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Quality metrics
 Proposed metrics for the model
Figure 5_Measures for Assessing Knowledge Quality in this research

15. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Quality metrics
Metric Description
Accuracy How accurate is the knowledge source?
Reliability The knowledge source will produce same answer for the same question in different resources.
Timeliness The resource produces an answer in an appropriate time
Age of resource The resource is up-to-date.
The age of knowledge is declared.
Provenance The Resource is based on valid authority.
Locality The resource is relevant to NZ
Relevancy The resource contains relevant knowledge to support the user query.
Completeness The answers to queries are complete.
Adoption The knowledge source gives answers that are useful.
Scalability The knowledge can be accessed from many systems without losing its meaning.
Citation The knowledge is backed up by accessible citations to research.
Structure (Relationship and Class
Richness)
The knowledge is in a form that computerised DSS can use, in a consistent structure. E.g. XML

16. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Proposed model
Our Focus
Figure 6_Proposed model for knowledge acquisition for CDSS

17. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Proposed model
 Knowledge broker
Figure 7_Knowledge Broker framework

18. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
Why Semantic Web?
 The Semantic Web technologies enable machines to search, collect, reuse and
combine information without human intervention.
 While the foundation of the first generation of the web is based on the
exchange of documents, the Semantic Web technologies propose a general
format for data interoperability.
 Knowledge will be organized in conceptual spaces according to its meaning.
 Semantic query answering instead of keyword-based query answering.

19. HiNZ Conference 2015
Christchurch
A model of knowledge quality assessment in clinical decision support system
29
June
PGR9-Presentation_Seyedjamal Zolhavarieh 19