WHODrug Koda, developed by Uppsala Monitoring Centre (UMC), is an automated coding service, which uses artificial intelligence (AI) to automate the coding of drug names and ATC selections, improving consistency and operational efficiency. It can also be used to accelerate dictionary upgrades, including the transition from WHODrug B2 format to B3.
Through API (Application Programming Interface) web services, the coding engine can be integrated with custom or off-the-shelf drug safety, medical coding, or data management systems.
In this webinar, Perficient and UMC discussed WHODrug Koda and how you can integrate it into your medical coding activities.
2. 2
Tom Hogan
Chief OperatingOfficer
Who We Are
AN INTRODUCTION TO
We’re a global digital consultancy transforming how the
world's leading enterprises and biggest brands connect
with customers and grow their businesses.
With Perficient, you get experience and expertise,
speed and agility, and a healthy dose of pragmatism to
drive your business forward.
3. 3
~4,500
Colleagues
WE ARE
Global in Our Delivery
We have office locations across North America and fully owned
global and domestic delivery centers around the world.
Allentown, PA
Ann Arbor, MI
Atlanta, GA
Boston, MA
Bozeman, MT
Cedar Rapids, IA
Charlotte, NC
Chicago, IL
Columbus, OH
Dallas, TX
Denver, CO
Detroit, MI
Fairfax, VA
Fargo, ND
Houston, TX
Irvine, CA
Lafayette, LA
Milwaukee, WI
Minneapolis, MN
New York, NY
San Francisco, CA
Seattle, WA
St. Louis, MO
Toronto, ON
Washington, D.C.
North America
Latin America
Mexico City, Mexico
Medellin, Colombia
Bogota, Colombia
Cali, Colombia
Chennai, India
Nagpur, India
Bangalore, India
India
Hangzhou, China
China
Oxford, UK
Novi Sad, Serbia
Europe
4. 4
AT-A-GLANCE
Perficient Profile
Alliance partnerships
with major tech
vendors
Multiple vendor &
industry tech and
growth awards
Global delivery centers
in China, India and
Latin America
Dedicated
solution
practices
35
Global Locations
$565M
2019 Revenue
1997
Founded in
~4,500
Colleagues
~90%
Repeat Business Rate
PRFT
Public, NASDAQ
5. 5
Industry Expertise
Our experience in the life sciences industry empowers us to help companies
discover, develop, and manufacture products that help people feel better and live
longer. We also help them enhance their customer experience and achieve
operational excellence.
I NDUSTRY
Life Sciences 30+ of the world's leading
public pharmaceutical,
biotechnology, and medical device
companies count on us .
We helped a pharmaceutical giant
transform the way it performs its medical
coding activities.
Project Spotlight
DigitalTechnologyStrategy Clinical Safety
6. 6
Caroline Halsey
Director, Life Sciences
Caz.Halsey@Perficient.com
Damon Fahimi
Product Manager, Uppsala Monitoring Centre
Damon.Fahimi@who-umc.org
Today’s Speakers
7. 77
Agenda
• WHODrug Global and Drug Coding
• An Introduction to WHODrug Koda
• Why use Koda
• Methods of Using Koda
• Perficient Koda Interface - Prototype
• Use Cases
• How to Find Out More
8. 8
Uppsala Monitoring Centre & WHODrug Global
Productname Drugcode ActiveIngredient(s) ATC
ASPIRIN
[ACETYLSALICYLIC
ACID]
00002701004 Acetylsalicylic acid A01AD, Other agents for local oral treatment
B01AC,Platelet aggregation inhibitors excl. heparin
M02AC,Preparations with salicylic acid derivatives
N02BA, Salicylic acid and derivatives
WHODrugGlobal
9. 9
The Challenges of Structuring Drug Data
Staffing
Drug coding
requires highly
skilled, educated
and trained staff
IncreasingVolumes
Both long-term and
short-term
demands
Quality&
Consistency
Dynamic nature of
drug coding. How
to keep consistency
in coding teams?
Tools & Processes
Dependencies on
tools & synonym
lists not delivering
optimal support for
drug coding
10. 10
What if…
...drug coding staff could spend their time only on the most relevant cases?
…a more significant part of the drug coding could be automated, thereby helping cope with both
short- and long-term peaks?
…UMC could gather know-how from the industry and, with the help of technology,
increase drug coding consistency?
…there was a plug-in to current processes and tools, enhancing drug coding?
Project goal:
Find an intelligent approach to automating drug coding using machines
and efficiently using the human brain when needed
11. 11
The Result: WHODrug Koda – Born in 2019
Verbatim
+
Route*
+
Indication*
* If available
Drug Name
Coding
ATC Selection
13. 13
Machine Learning Applied
How? For What? TrainingData?
Supervised machine
learning via logistical
regression
Primarily ATC code
selection, but also drug
name coding
Millions of data points
from VigiBase
Trainingdata
Input A → Output A
Input B → Output A
Input C → Output B
Prediction
Non-trainingdata
Input D → Output A
More than 500,000
“classes” of drug names
Around 1400 “classes” of ATC
codes
14. 14
VigiBase: An Invaluable Asset for Training
• Training data is key!
• VigiBase used to train and update WHODrug Koda
+ millions of additional data points
Coded in WHODrug Global
Drug name ATC code and text
Acetazolamide
S01EC, Carbonic anhydrase
inhibitors
Acetazolamide
S01EC, Carbonic anhydrase
inhibitors
Acetazolamide C03, Diuretics
Acetazolamide N03AX, Other antiepileptics
Unprocessed Data
Verbatim Route Indication
Acetasolamide Oral Cataract surgery
Acetazolamide
tablet 250mg
Unknown Retinitis central
Acetazolamide Oral Secondary hypertension
Acetazolamide Oral Seizure
15. 15
“Only” Machine Learning – Not Enough
Built-in CodingRules Spell Checks and Algorithms Know-how and Controlled
Feedback Loops
Following the latest
regulatory expectations
and best practices for
WHODrug coding
Making sense of the data
Systemic
(foundational)
Dynamic
(supplementary)
Examples of drug verbatims
Naphazoline (naphcon) 0.1% ophthalmic solution
sodium fluoride (prevident5000 booster plus) 1.1% dental
paste
ERELSA TABLETS 50MG
Pharmacists and medical
expertise at UMC and
feedback from users
16. 16
Average Results With External Data
Drugname coding ATC Selection
≈ 95%of all drug namescoded*
*High certainty: 80%; Lower certainty: 15%; Uncoded: 5%
≈ 100% selected**
**High certainty: 80%; Lower certainty: 20%
What about the precision for the predictions?
≈ 99%***
***Based on high certainty predictions
For an example;please see: Herrgårdet al., 2020. Assessment of Machine Learning Methods in Coding of Concomitant Medications in Clinical Trials.Abstract for the 2020
PhUSE Connect USA 2020. https://www.phusewiki.org/docs/2020%20US%20Connect%20Florida/Presentations/ML/Final%20Papers/ML13%20pdf.pdf
17. 17
How to Access WHODrug Koda
WEB APPLICATION
For evaluations / smaller studies
API
Allowing implementation within
current processes
18. 18
UMC and Perficient
• Perficient is licensed and approved by UMC for integrating WHODrug Koda
into medical coding tools
• Contact UMC for general questions about WHODrug Koda
• Contact Perficient for questions about our services
https://www.who-umc.org/whodrug/whodrug-portfolio/whodrug-koda/
19. 19
Why Use Koda
Koda can support medical coding in various scenarios:
• Assigning drug codes only
• Assigning drug codes and preferred ATC codes based on indication
• Recoding / refreshing WHODrug encoded data
• Up-versioning to a more recent version
• Up-versioning from B2 to B3 format
Koda is already trained
• Time, effort, and volume of data required to train an AI system is significant
• Koda is already trained and ready to use
20. 20
Methods of Using Koda
Two ways to use Koda:
1. Upload / download data through Koda UI on UMC website
• Requires manual process of data extraction and bringing coding results back
into source data
• Koda UI is currently used by UMC customers
2. Upload / download data through UMC Koda webservice API
• Allows direct integration of Koda functionality into coding process
• Requires technical knowledge of webservice integration into systems
21. 21
Perficient Koda Interface – Prototype
Perficient has built a prototype of a generic Oracle interface to the Koda
webservice API
• Consists of a table for drug data to be coded and some programming
objects
• Oracle-based
• Requirements:
• Oracle 11 or above
• UMC security certificates installed
• Open network connection from DB server to UMC server
• Additional custom adapter required for each specific source system
(medical coding system, PV system, etc.)
22. 22
WHODrug Koda
Interface
Source System
YOUR COMPANY
WHODrug Koda
Encoded drug or
suggested drug
Drugs to be coded
CustomAdapter
Drugs to be coded
Encoded drug or
suggested drug
23. 23
Example Workflow Step 1 – Table With Uncoded Data
Line Drug Route Indication Code ATC Comments
1 Paracetamol Entry with single ATC code
2 Aspirin Oral Anti-coagulant Entry with 4 ATC codes from 4 different classes
3 Aspirin Thrombosis Entry with 4 ATC codes from 4 different classes
4 Aspirin thrombose Entry with 4 ATC codes from 4 different classes
5 Aspirin Thrombosis prophylaxis Entry with 4 ATC codes from 4 different classes
6 Aspirin Headache in the morning Entry with 4 ATC codes from 4 different classes
7 Artificial tears Drops Non-unique entry including an umbrella term
8 A b c Another entry ABC exists
9 ibuprofen headache Entry with some ATCs not very distinctive (e.g. two from M
class)
10 Vitamins Non-unique drug name (one regular entry and one umbrella
term)
11 x37ytc12 Non-existing entry
12 sebcur Unique name which was non-unique in older versions
13 sebcur [salicylic acid] Non-unique entry as in older version
14 sebcur salicylic acid Concatenation of name with ingredients
24. 24
Example Workflow Step 2 - Interface Execution
1. Transformationof datato be coded intoKodaformat (JSON)
{"WhoDrugVersionShortName":"GLOBALB3Sep19","UsePreferredBaseRule":"true","UseGenericRule":"true","Description":"ClientJob ID =
13774594","DrugsToCode":[{"Id":13774573,"Verbatim":"paracetamol","Route":null,"Indication":"headache"},{"Id":13774574,"Verbatim":"aspirin","Route":"oral","Indication":"anti-
coagulant"},{"Id":13774575,"Verbatim":"aspirin","Route":null,"Indication":"thrombose"},{"Id":13774576,"Verbatim":"aspirin","Route":null,"Indication":"thrombosis"},{"Id":13774577,"Ver
batim":"aspirin","Route":"oral","Indication":"thrombosis prophylaxis"},{"Id":13774578,"Verbatim":"aspirin","Route":null,"Indication":"headachein the morning"},{"Id":13774579,
"Verbatim":"acetylsalycilicacyd","Route":null,"Indication":null},{"Id":13774580,"Verbatim":"aspirin50mg","Route":null,"Indication":null},{"Id":13774581,"Verbatim":"ab c","Route":null,
"Indication":null},{"Id":13774582,"Verbatim":"artificaltears","Route":"drops","Indication":null},…
2. CreateHTTPSconnection to Kodawebservice API using UMCsecurity certificates,
senddata, andget a response
{"Id":"db7a0032-3656-4736-864e-d0d75495d8b6","Description":"ClientJob ID = 13774594","WhoDrugVersionLongName":"WHODRUGGLOBAL B3 September 1,
2019","WhoDrugVersionShortName":"GLOBALB3Sep19","UsePreferredBaseRule":true,"UseGenericRule":true,"EncodingJobStatus":"Finished","QueuedDateUTC":"2020-01-
31T16:29:56.1800000Z","StartedDateUTC":"2020-01-31T16:29:56.6400000Z","FinishedDateUTC":"2020-01-31T16:30:02.3500000Z","NoOfRecords":21,"CodingItem“
:[{"Id":"13774573","Verbatim":"paracetamol","Route":null,"Indication":"headache","EncodedDrug":{"Tradename":"PARACETAMOL","Ingredients":["Paracetamol"],"DrugCode":"000200
01001","AtcCode":"N02BE","SuggestedAtcs":[]},"SuggestedDrugs":[],"CodingPath":"DirectHit"},{"Id":"13774574","Verbatim":"aspirin","Route":"oral","Indication":"anti-
coagulant","EncodedDrug":{"Tradename":"ASPIRIN[ACETYLSALICYLICACID]","Ingredients":["Acetylsalicylicacid"],"DrugCode":"00002701004","AtcCode":"B01AC",
"SuggestedAtcs":[]},"SuggestedDrugs":[],"CodingPath":"NonUniqueByIndication"},{"Id":"13774575","Verbatim":"aspirin","Route":null,"Indication":"thrombose","EncodedDrug":{"Tradena
me":"ASPIRIN[ACETYLSALICYLICACID]","Ingredients":["Acetylsalicylicacid"],"DrugCode":"00002701004","AtcCode":"N02BA","SuggestedAtcs":[]},"SuggestedDrugs":[]…
3. Parseresponse fromKodaformat(JSON) and updatedatatable
25. 25
Example Workflow Step 3 – Table Contains Coded Data
Line Drug Route Indication Code ATC Comments
1 Paracetamol 00020001001 N02BE ATC provided as drug has only one ATC code
2 Aspirin Oral Anti-coagulant 00002701004 B01AC ATC code correct
3 Aspirin Thrombosis 00002701004 B01AC ATC code correct
4 Aspirin thrombose 00002701004 N02BA Wrong ATC (indication misspelled)
5 Aspirin Thrombosis
prophylaxis
00002701004 B01AC ATC code correct
6 Aspirin Headache in the
morning
00002701004 N02BA ATC code correct
7 Artificial tears Drops No code as non-unique,but list of suggested drugs as
expected
8 A b c 00256502021 No ATC code as no indication
9 ibuprofen headache 00109201001 No ATC code, but 2 suggested ATC codes from M class
10 Vitamins No code as non-unique.Returns 2 suggested drugs as
expected, one of which is an umbrella term
11 x37ytc12 Uncoded as expected
12 sebcur 00021201106 No ATC code as no indication
13 sebcur [salicylic acid] 00021201106 Entry from old WHODrug version correctly coded
although no longer non-unique in new version
14 sebcur salicylic acid No encoded drug code, but single suggested drug code,
which is correct
26. 26
Potential Interface Use
Possible integrations of an Oracle-based interface could include:
• Argus Safety
• Connect Koda as WHODrug interactive coding webservice to Argus Product screen
• Integrate WHODrug coding via Koda into custom case save procedure
• Oracle TMS
• Auto-create unapproved VTAs for coding omissions which have been sent to Koda
We are prototyping an Oracle-based approach. The integration of Koda with other source systems can be
explored.
28. 28
Caroline Halsey
Director, Life Sciences
Caz.Halsey@Perficient.com
Damon Fahimi
Product Manager, Uppsala Monitoring Centre
Damon.Fahimi@who-umc.org
Thank You
Notas do Editor
My name is Caroline Halsey, I am a Director in the Life Sciences Business Unit at Perficient. I have 20 years experience in the pharmaceutical industry including:
Clinical data management
Project & program management
Process design & system implementation
Computer system validation
For the past few years I have been focussed on medical coding processes and systems. Today I am here to describe about how you could integrate the UMC’s WHODrug Koda coding engine into your coding process via its webservice API and I am pleased to be joined by Damon Fahimi from the UMC to discuss this.
Damon will introduce WHODrug coding and WHODrug Koda, then I will discuss how it could be used and what we are working on at Perficient to enable integration of Koda into medical coding processes
As you have heard, Koda can help you with your daily drug coding business, but it can also help to recode data by upgrading your WHODrug version or even your WHODrug format.
For example, when recoding a study from B2 to B3 a large number of terms may no longer code. Submitting those terms to WHODrug Koda could dramatically reduce the manual effort of recoding.
One of the main challenges in the use of AI in medical coding is the time, effort and volume of data required to train the AI system. With WHODrug KODA the training has already been done for you."
We’ve heard from Damon, one way to use Koda is via uploading and downloading data through the Koda UI, but it is also possible to fully integrate it into your systems by using the Koda webservice Application Programming Interface.
Now, using webservice APIs is not always straightforward, as you have to deal with technicalities like transforming data, generating network traffic, security cerificates etc.
It would simplify integration if you had an additional interface level available, which hides all the technical bits. Some of my Perficient colleagues have built a prototype of such an interface, and this is what I’d like to talk about today.
I’ll give an overview of the design, usage and possible integration and let you know how to find out more if you are interested
In order to build such an interface we needed first to decide on a platform, and we have chosen Oracle. This choice was kind of arbitrary, and of course you could build a similar interface on any other platform.
This prototype consists basically of
a table which contains the drug data to be coded and mimics the source application
plus some programming objects in PL/SQL, which interact with the webservice API right out of the database.
The only system requirement is that you have Oracle version 12 or higher, and you need to have the UMC certificates installed and an open network connection from your database to the UMC server. No further infrastructure is needed.
Here you can see the designed workflow.
The source system or application sends the drugs to be coded via the custom adapter to the interface.
The interface converts the data into the format expected by the Koda application and calls the Koda webservice API
It then gets back the response from Koda with coding information, parses the response and updates the drug data with the coding information in the source system.
Triggering the interface could of course be an automated process, e.g. a regular batchjob or automated execution if the table of drugs has been updated.
This design actually hides all the technical complicated bits from the end user or end application.
Now lets have a look at a real example.
The columns with blue heading here reflect data which is exchanged with Koda.
Initially the drug table contains drug verbatims, some of them with route and / or indication.
Now we trigger the interface.
The interface now
Transforms the data into Koda format
Builds the network connection and send data to Koda
Gets the response in Koda format
Parses the response
Puts coding information back into the source table
This is the part that nobody wants to deal with as it looks like this <trigger animation>
This is what the table looks like after the interface execution.
The interface has updated the code and ATC code column with some values.
Please note that here you can only see the encoded information, the comments column shows some information on suggested drugs if available. Lets have a look at a few examples in more detail...
Line 1 is paracetamol, which is correctly coded to its drug code and ATC code although no indication was provided. This is because Paracetamol has only one ATC code.
Lines 2 to 6 are some records with Aspirin, which has 4 ATC codes from 4 different classes, so is fairly distinctive. For correctly spelled indications the drug code and ATC is always correct, for the misspelled indication in line 4 the ATC code is incorrect.
Line 7 to 11 are some examples where we have tried non-unique drugs, non-existing names, umbrella terms, drugs with multiple ATC codes from the same class etc. All codes which were provided were as expected and correct. Suggested information is also as expected and correct.
Of special interest are lines 12 to 14, which is an example on how you could use Koda for a version or format upgrade. The tradename SEBCUR was non-unique in a previous WHODrug version and became unique in a more recent version. Koda is able to correctly encode or suggest the code from the current version if you pass in the tradename with ingredients attached as it was stored in the previous version.
Such an Oracle based interface could be potentially integrated with any Oracle based system, for example Argus Safety, where you could use it for coding during case entry, or for coding during case save.
Or you could integrate it with Oracle TMS, where you could for example auto-create non-approved VTAs for coding omissions which you send to Koda and where Koda returns a code.
As already mentioned, this prototype is currently Oracle based, but could be explored for other platforms as well, so please don’t go away with the iompression that this would only work if you use Argus and TMS.
Seed Question: Will this interface only work for Oracle-based systems?
Response: the prototype interface is Oracle based, but with the appropriate custom adapter, in principle, any source system could be used.