2. Search Strategy
UniMelb online library
Ovid MedLine
Search keywords
Health Informatics
Emergency Medicine
Triage
Machine Learning
Results
Between 20 and 90 results shortlisted for various search combinations
10 articles shortlisted
5 articles chosen
3. Decision Rationale
Reviewed articles
Chose top 5 based on following criteria:
Relevance
Potential impact
Realistic
Achieved statistical and real-world significance
4. Machine Learning-Based Electronic Triage More Accurately
Differentiates Patients With Respect to Clinical Outcomes
Compared With the Emergency Severity Index
Levin, S., Toerper, M., Hamrock, E., Hinson, J., Barnes, S., Gardner, H… (2018). Machine Learning-
Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes
Compared With the Emergency Severity Index. Annals of Emergency Medicine. 71(5). Retrieved
from https://www.annemergmed.com/article/S0196-0644(17)31442-7/fulltext
Highlights
Outcomes were Critical Care, Emergency Procedure and Hospitalisation
Secondary ”clinical” outcomes – elevated troponin or lactate
Looked at E-Triage vs traditional triage (ESI)
E-Triage - Used Machine Learning (Random Forest) to predict outcomes
Patients up-triaged from a Triage Level Three were 5x more likely to need critical care
5. Validation of deep-learning based triage and
acuity score using a large national dataset
Kwon, J-m., Lee, Y., Lee, Y., Lee, S., Park, H., Park, J. (2018). PLoS ONE. 13(10): e0205836. Retrieved
from https://doi.org/10.1371/journal.pone.0205836
Highlights
Outcomes – In-Hospital Mortality, Critical Care and Hospitalisation
Creation of a DTAS (Deep Learning Triage and Acuity Scale)
Compared Deep-Learning with KTAS, MEWS and more traditional Logistic Regression and Random Forest Machine
Learning algorithms
DTAS showed a much higher AUROC and AUPRC with statistical significance compared with other
modalities assessed for mortality, Critical Care and Hospitalisation
6. Machine Learning Based Prediction of Clinical Outcomes
for Children During Emergency Department Triage
Goto, T., Camargo, C., Faridi, M., Freishtat, R., Hasegawa, K. (2019) JAMA Network Open. 2(1).
Retrieved from https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2720586
Highlights
Outcomes – Critical Care and Hospitalisation
Derived 4 Machine Learning based algorithms
Lasso Regression, Random Forest, Gradient-boosted Decision Tree and Deep Neural Network
Demonstrated that Machine Learning had a better discriminatory ability; it was less likely to under-triage
critically ill children and over-triage children who are less ill
7. Validity of the Manchester Triage System in emergency patients
receiving life-saving intervention or acute medical treatment – a
prospective observational study in the emergency department
Graff, I., Latzel, B., Glien, Procula., Fimmers, R., Dolscheid-Pommerich, R. (2018). Journal of
Evaluation in Clinical Practice. 25(398-403). Retrieved from
https://onlinelibrary.wiley.com/doi/abs/10.1111/jep.13030
Highlights
Outcomes – Life Saving Intervention (LSI) and Acute Emergency Treatment (AET)
Prospective observational study
Looked at the ability of the Manchester Triage System (MTS) to identify patients requiring LSI to Triage
Category 1 and AED to either category 1 or 2
Confirmed the sensitivity of MTS whilst balancing the need of avoiding over-triage of false positive
patients
8. A Machine Learning Approach to Predicting Need for
Hospitalization for Pediatric Asthma Exacerbation at the
time of Emergency Department Triage
Patel, S., Chamberlain, D., Chamberlain J. (2018) A Machine Learning Approach to Predicting
Need for Hospitalization for Pediatric Asthma Exacerbation at the time of Emergency Department
Triage. Academic Emergency Medicine Journal. Retrieved from
https://www.ncbi.nlm.nih.gov/pubmed/30382605
Highlights
Outcome: Hospitalisation
Defined and tested four machine learning algorithms
Decision Tree, LASSO Regression, Random Forest, Gradient Boosting machine
AUROC of 0.85 for hospitalisation
9. Triage future predictions
My “best of” Triage and Informatics in the ED gives some insight into the future direction of this
topic
The importance of triage, its accuracy and ability to predict outcomes is significant
Machine learning offers possibilities into improving the accuracy of triage, either as a support to
traditional triage or as a major part of the triage process itself
Deep learning is another, even more advanced iteration of Machine Learning where the
algorithm is able to learn what it is required to know and what importance to place on it
ED triage is an extremely important aspect of the Emergency Department (ED) process for patients. In rapid succession, patients are sorted into grading of severity of their presentations. This sets the tone for their hospital stay, with numerous studies attesting to the importance of triage on the care received for patients (citation)
With the ever increasing demands being placed on hospital EDs as hospitals get busier and busier, the importance of ED Triage “getting it right” as quickly as possible is more important than ever.
For this reason, I decided to look at the latest work from the Health Informatics sphere into ED Triage. This includes Machine Learning, prediction tools, Clinical Decision support systems and the use of automated triaging systems.
There is some fantastically interesting ideas and articles out there in the literature, and following are but five of them.
Keyword search -> I found that with my question I needed to choose articles from a variety of searches. I found there was no single search that gave me articles that were relevant and interesting, without the search being excessively broad
Criteria:
Relevance – to the topic chosen
Potential impact – could I see it making a difference in either my or any emergency department
Realistic – is the idea realistic; can I see it being created and implemented
Statistical significance – given by the achievement of a predetermined statistical outcome i.e.. p<0.05
Real world significance – was the statistic achieved appreciable, i.e.. If implemented, would the improvement in the outcome in question be noticed outside of the statistics
Definitions:
Critical Care - ICU or in-hospital mortality
Emergency Procedure - procedural intervention within 12 hours of presentation
Hospitalisation – need for admission
Random Forest Algorithm
Uses a set of decision trees, in this case 100, (each predicting a given outcome) and aggregates them to give a single probabilistic prediction (Yiu, 2019)
Three separate Random Forest Algorithms were created for each of Critical Care, Emergency Procedure and Hospitalisation
Results
Showed a higher discriminatory ability for patients triaged to level 3 by ESI
The critical care outcome was 5x more likely to occur in patients where E-Triage up-triaged them to a 1 or 2
References:
Yiu, T. (2019) Understanding Random Forest. Retrieved from: https://towardsdatascience.com/understanding-random-forest-58381e0602d2
Deep Learning
Uses Artificial Neural Networks – a set of multiple layers of data analysis - which allows for “feature learning”, that is, the algorithm learns the features needed for given tasks, using several non-linear modules.
Deep learning requires much more data points, as it learns the features needed within the data only after it has been exposed to over a million data points. (Kapoor, 2019)
AUROC – area under receiver operator curve – common binary classifier, comparing sensitivity vs 1-specificity
AUPRC – area under precision and recall curve – used for an imbalanced data set (i.e. much more –ve than +ve) as it is a better descrimator when the total negatives are large
References:
Kapoor, A. (2019). Deep Learning vs. Machine Learning: a Simple Explanation. Retrieved from https://hackernoon.com/deep-learning-vs-machine-learning-a-simple-explanation-47405b3eef08
Lasso Regression – a type of linear regression that allows the creator to select important predictors, allowing the result to be more clinically useful. Lasso regression also uses regularisation, which shrinks the range of results, aiming for a simpler, more accurate model. (Lasso Regression, 2019)
Random Forest - Uses a set of decision trees, (each predicting a given outcome) and aggregates them to give a single probabilistic prediction
Gradient-boosted Decision Tree – an additive model of decision trees, estimated by gradient descent (an optimisation algorithm, aiming to simplify the algorithm) (Parr, Howards. n.d)
Deep Neural Network – Machine learning that consists of multiple layers of nonlinear processing units that learn the value of parameters, resulting in the best prediction of outcome (Goto et al. 2019)
References:
Lasso Regression (2019). Statistics How to. Retrieved from: https://www.statisticshowto.datasciencecentral.com/lasso-regression/
Parr, T., Howard, J. (n.d). How to explain Gradient Boosting. Retrieved from https://explained.ai/gradient-boosting/index.html
Goto, T., Camargo, C., Faridi, M., Freishtat, R., Hasegawa, K. (2019) JAMA Network Open. 2(1). Retrieved from https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2720586
LSI – “therapeutic measures with the goal of stopping a acutely life-threatening condition and stabilising the patient” (Graff et al, 2018)
AED – Given by Graff et al. (2018) as “therapeutic measures of an acute illness or for prevention of an expected imminent life-threatening condition”
Results:
Very high NPV for both LSI and AET (0.99 and 0.98) i.e.. Of patients not triaged to level 1 or 1 / 2, a very high proportion did not need the specific outcome (LSI/AET)
Sensitivities of 70% and 82% for correct capture of LSI and AET to appropriate triage categories
Machine Learning Algorithms – previously described for other studies
Results:
This study was able to predict, with an AUROC of 0.85, the need for admission of patients aged 2-18 presenting with an asthma exacerbation, based off triage notes and immediately available external findings such as wind speed, pressure, influenza outbreaks, patient home environment and socioeconomic status (based off area code of address)