The document discusses several active areas of work in artificial intelligence applications in radiology at Brown University's AI radiology lab, including COVID-19 detection from chest x-rays, tumor assessment, stroke diagnosis, and more. It provides details on techniques like contrast dropout to deal with missing data, human-in-the-loop approaches, automatic quality estimation, treatment response evaluation, and federated learning to share models without sharing patient data. Performance results and example visualizations from various models are also included.

1. Artificial Intelligence
Applications in
Radiology
Harrison Bai
5/6/21

2. Lab website
https://sites.brown.edu/airadiology/people/

3. Active areas of work
1. COVID-19
2. Treatment response assessment on imaging (brain, TACE, lung, colorectal)
4. Tumor characterization on imaging
3. Rapid diagnosis of large-vessel ischemic stroke, patient selection and outcome
prediction
5. Infrastructure development
6. Federated learning
7. Image registration (CT-guided tumor ablation)
8. Radiology reports natural language processing

7. Receiver operating characteristic (ROC) curve for model
performance on Brown-April test set. RT-PCR results were
used as the ground truth label. Seven radiologists were asked to
identify whether each CXR manifested indications of COVID-19.
Their sensitivity and specificity scores are compared to those of
the AI model.
ROC curve for model performance on the External and Xiangya-
February test sets. RT-PCR results were used as the ground truth
label. As several CXR images were associated with each patient, the
mean predictive output was used to determine if the patient is COVID-
19 positive.

8. Gradient-weighted class activation mapping images of COVID-19 patients from each test set. All images were predicted
correctly as COVID-19 positive. (a) Image and (d) associated map from Brown-April cohort. (b) Image and (e) associated map from
External cohort. (c) Image and (f) associated map from Xiangya-Feburary cohort.

10. Technique for dealing with missing sequence
and imaging artifact: Sequence dropout

11. Technique for dealing with missing sequence
and imaging artifact: Sequence dropout
The segmentation for different subregions (WT, TC, ET) using the conventional UNet3D (top) and contrast-dropout UNet3D (bottom).
The contrast-dropout method significantly improved the robustness against missing inputs in deployment and can be used to analyze
the impact of each input contrast.

12. Human-in-the-loop AI
 In the short- to mid-term, the utilization of AI needs to be combined with
human intervention and supervision
 Active learning strategy – annotation
 Treatment response evaluation on imaging
 Automatic quality estimation to flag the failed cases for humans to review
and/or edit

13. Human in the loop annotation

14. Automatic quality estimation
Correlations between the estimated uncertainty from mean foreground probably and mean probably variation and the Dice scores. The mean probability
metric showed higher correlations.

15. Automatic quality estimation
other class
necrotic/non-enhancing
tumor core
peritumoral edema
enhancing tumor
0
20
40
60
80
100
normalized uncertainty
T2 UNet3D segmentation
whole tumor uncertainty tumor core uncertainty enhancing tumor uncertainty
Uncertainty map estimated using the mean probably within a ROI.

16. Treatment response evaluation

17. DIANA
siim.org | #SIIM20 |
@SIIM_Tweets
17
PACS
DIANA Analysis and
Control Scripts
Imaging
Systems
Workstation/Device
Dose Monitor
Traditional Workflow
HTTP
Image/Report
Transfer
Protocols/Tools
2
*End of PHI
User Access
1
4
3 5
Communication Modality
ICH Prediction
Epidural Hemorrhage
High Probability (99%)
BA Prediction
7 years and 5.34
months
External Database
Evaluation of DIANA AI
Pipelines
 Integrated with RIH
PACS
 Configured to
monitor live PACS
for 2 months
o Bone age
o Intracranial
hemorrhage
 Ground truth
o mPower
o Radiology reports

18. Federated learning
 Domain shift – distribution difference between source data and target data
leading to performance degradation
 Annotations vary across imaging sites
 Share weights without sharing data
 AWS NVIDIA Clara Train SDK using TensorFlow 1.14
 Semi-supervised and unsupervised learning
 Data access limited by strict sharing policies

19. 点击此处添加标题
在此输入相关文字，在此输入相关文字，在此输入相关文
字在此输入相关文字，在此输入相关文字，在此输入相关
文字， 在此输入相关文字，在此输入相关文字，在此输入
相关文字在此输入相关文字，在此输入相关文字，在此输
入关文字，