More than Just Lines on a Map: Best Practices for U.S Bike Routes
The Developing Human Connectome Project (dHCP): the power of Big Data
1. The Developing Human Connectome
Project (dHCP): the power of Big Data
Dr Emma C. Robinson
@emrobSci ecr05
emma.robinson@kcl.ac.uk
https://emmarobinson01.com/
2. The Developing Human Connectome Project
• Mapping the emergence of brain
connectivity from 20-44 weeks
PMA
• 861 neonates (144 Pre-term),
266 foetuses (so far)
• Acquisitions (MRI):
Resting state fMRI
Multi-shell HARDI
Structural T1 and T2
• Supported by
Genetics-> 4.3 million SNP array
Cognitive test scores/eye tracking
Demographics
http://www.developingconnectome.org/
c/o Dr Bernard Kainz, Imperial College
3. Data Releases
3
• 1st Pilot data release
• https://data.developingconnectome.org/app/template/Login
.vm
• 40 neonatal subjects:
• T1, T2, fMRI and dMRI volumes (minimally processed)
• output of surface extraction pipelines
• Released as Torrent
5. Data Releases
5
• 2nd Data Release
•558 session (505 subjects) of Neonatal data:
• T1, T2, fMRI and dMRI volumes, original, cleaned, pre-processed
and mapped to volumetric template space
• Cortical surface meshes and features
• http://www.developingconnectome.org/information-registration-
and-download/
• Subject to open access terms of use (data agreement)
• Released as Torrent
• Troubleshooting documents will be available
Support via https://neurostars.org/tags/developing-hcp
6. Challenges of working with
developing data
• Developing data is affected by
Motion (severe cases account for < 2% )
Limited scan times
Relatively low resolution
Inverted T1/T2 contrast
spatio-temporal evolution
7. Neonatal Structural Pipeline
Makropoulos and Robinson et al. The Developing Human Connectome Project: a Minimal Processing Pipeline for Neonatal Cortical Surface
Reconstruction. NeuroImage (2018)
• Reconstruction with motion
correction
• Tissue segmentation with
DRAW-EM
• Surface mesh modelling
• Feature Extraction
• Visually QC’d (sub-set)
High intensity white matter correction
Makropoulos, Antonios, et al. "Automatic whole brain MRI segmentation of the developing neonatal brain." IEEE transactions on medical imaging 33.9
(2014): 1818-1831.
Cordero-Grande, Lucilio, et al. "Three-dimensional motion corrected sensitivity encoding reconstruction for multi-shot multi-slice MRI: Application
to neonatal brain imaging." MRM (2018)”
8. Neonatal fMRI Pipeline
• Inspired by the HCP
pipelines and FSL FEAT
pipeline
• But optimised to address
the challenges of neonatal
data
Head motion
9. Neonatal fMRI Pipeline
• Inspired by the HCP
pipelines and FSL FEAT
pipeline
• But optimised to address
the challenges of neonatal
data
Head motion
motion by susceptibility
10. Neonatal fMRI Pipeline: Key features
fMRI EDDY (FREDDY) Integrated dynamic distortion correction and
slice-to-volume motion correction
Pre-
Eddy
Post-
Eddy
Correction of intra-volume
motion
Andersson, Jesper LR, et al. "Towards a comprehensive framework for movement and distortion correction of
diffusion MR images: Within volume movement." Neuroimage 152 (2017): 450-466.
11. fMRI EDDY (FREDDY) Integrated dynamic distortion correction and
slice-to-volume motion correction
Correction of motion-by-susceptibility distortions
Neonatal fMRI Pipeline: Key features
Andersson, Jesper LR, et al. "Towards a comprehensive framework for movement and distortion correction of
diffusion MR images: Within volume movement." Neuroimage 152 (2017): 450-466.
13. Neonatal fMRI
Pipeline
*Harrison, Samuel J., et al. "Large-scale probabilistic functional modes from resting state fMRI." NeuroImage 109 (2015): 217-231.
16 PROFUMO*
modes qualitatively
assessed as
corresponding to
adult resting-state
networks.
14. Neonatal dMRI Pipeline
• 300 diffusion volumes (20 b0)
• b = 400 s/mm2
(64)
• b = 1000 s/mm2
(88)
• b = 2600 s/mm2
(128)
• Correction for eddy currents, susceptibility and motion
performed with FSL’s Eddy
• Virtual dissection (atlas based)
Bastiani et al., Automated
processing pipeline for
neonatal diffusion MRI in the
developing Human
Connectome Project.
NeuroImage 185 (2019): 750-
763.
Jesper L. R. et al. An integrated approach to correction for off-resonance effects and subject movement in diffusion MR
imaging. NeuroImage, 125:1063-1078, 2016.
15. • Micro-structural parameter estimates using NODDI (Zhang et al 2012)
Microstructure Tracts (virtual dissection)
Neonatal dMRI Pipeline
Bastiani et al., Automated processing
pipeline for neonatal diffusion MRI in the
developing Human Connectome Project.
NeuroImage 185 (2019): 750-763.
38 39 40 4138 39 40 41
16. dHCP spatio-temporal atlases
• New volumetric and surface templates spanning 36-44 weeks
gestation
Andreas Schuh et al.
Unbiased construction
of a temporally
consistent
morphological atlas of
neonatal brain
development
bioRxiv (2018):
251512.
17. dHCP spatio-temporal atlases
• New volumetric and surface templates spanning 36-44 weeks
gestation
Jelena Bozek et al.
Construction of a Neonatal
Cortical Surface Atlas Using
Multimodal Surface
Matching in the Developing
Human Connectome Project
NeuroImage 179 (2018): 11-
29.
18. Surface-template alignment
• Optimised Multi-modal Surface matching
• Scripts available at
https://github.com/ecr05/dHCP_template_alignment
Robinson, Emma C., et al. "Multimodal surface matching with higher-order smoothness constraints." NeuroImage (2018).
Robinson, Emma C., et al. "MSM: a new flexible framework for Multimodal Surface Matching." Neuroimage (2014)
19. Some recent developments in MSM
MSM now also allows
smooth deformation
of cortical anatomies
Robinson, Emma C., et al.
"Multimodal surface matching
with higher-order smoothness
constraints." NeuroImage (2
018).
20. New MSM: finding trends in longitudinal
cortical development
• 24 very preterm infants (born <30 weeks PMA, 15 male, 15 female)
• scanned 2-4 times before or at term-equivalent (36-40 weeks PMA)
Garcia, Kara E., Robinson E.C. et al. "Dynamic patterns of cortical expansion during folding of the preterm human brain." PNAS (2018)
21. The power of Big Data
• 505 subjects (558) sessions
• Higher statistical power
• More robust models
• Sufficient to train Deep Networks (?)
22. Deep Learning for Brain Imaging
Transfer learning: Applied to motion artefact
detection of neonatal diffusion MRI (dMRI)
•Like fMRI highly
sensitive to motion
•Standard practice to
remove noisy slices
•Train CNN classifier
using transfer learning
Data from
Developing Human
Connectome
Project (dHCP)
Kelly, C., et al. Transfer learning and convolutional neural net fusion
for motion artefact detection.
Red boxed highlight motion artifacted slices
23. Deep Learning for Medical Imaging
Transfer learning: Applied to motion artefact
detection of neonatal diffusion MRI (dMRI)
•Trained on 36 subjects
•Multiple CNNs trained
on different categories
of dMRI data
•Output of predictions
merged by random
forest
94.8%-99.8%
accuracy
Human level
~99.25%
Kelly, C., Pietsch, M., Counsell, S., & Tournier, J. D. Transfer learning and convolutional neural net fusion for motion artefact detection.
24. Predicting gestational age from Neonatal
Structural connectome
• Training a neural network
to predict*
• GA at birth
• GA at scan
• From dMRI connectomes
• Mae:
• 1.5 weeks (GA at birth)
• 0.65 (GA at scan)
*replication of method in Girault, Jessica B., et al. "White matter connectomes at birth
accurately predict cognitive abilities at age 2." NeuroImage192 (2019): 145-155.
25. Geometric (surface) deep learning
• Train CNNs on spatial
filters fit to the cortical
surface
e.g.
Seong, Si-Baek, et
al Frontiers in
Neuroinformatics 12
(2018): 42.
Zhao, Fenqiang, et al. "Spherical U-
Net on Cortical Surfaces: Methods
and Applications." IPMI 2019.
26. • 3 channels: cortical
thickness, curvature and
myelin
• Projected to 2D(via sphere)
• ResNet - 5 blocks of residual
layers (2 units per block
o Accuracy for prem vs term
classification = 100%
o GA at scan mae=0.493
x Train mae=0.198; Test
mae= 0.493
Geometric (surface) deep
learning
Regression of Age at scan
27. • Outperforms ROI
analysis
• 100 Voronoi parcels
• Average data for
each parcel
• GA regression Test
mae= 0.95
x Train mae=0.41; Test mae= 0.95
Geometric (surface) deep
learning
28. • Features visualised using Grad CAM
Increasing GA
32 37 41 44
Means
Selvaraju, Ramprasaath R., et al. "Grad-cam: Visual explanations
from deep networks via gradient-based localization." Proceedings of
the IEEE International Conference on Computer Vision. 2017.
Geometric (surface) deep
learning
29. Modelling brain Development with
Gaussian Process Regression
• 446 dHCP neonates scanned
cross sectionally
• Aligned using 2-channel
registration of T2 and cortical
mantel
• Input variables, GA, PMA, sex
• Gaussian Process regression
estimated
• brain tissue intensity on T1 and
T2
• local tissue shape (dx,dy,dz
deformation maps)
• x
GP model of brain growth
J O’Muircheartaigh, EC Robinson et al, Modelling brain development: investigating white
matter injury in term and preterm born neonates, submitted
30. Modelling brain Development with
Gaussian Process Regression
• 446 dHCP neonates scanned
cross sectionally
• Aligned using 2-channel
registration of T2 and
cortical mantel
• Gaussian Process regression
estimated
• brain tissue intensity on T1
and T2
• local tissue shape (dx,dy,dz
deformation maps)
GP model of brain growth
J O’Muircheartaigh, EC Robinson et al, Modelling brain development: investigating white
matter injury in term and preterm born neonates, submitted
31. Modelling brain Development with
Gaussian Process Regression
• 446 dHCP neonates scanned
cross sectionally
• Aligned using 2-channel
registration of T2 and
cortical mantel
• Gaussian Process regression
estimated
• brain tissue intensity on T1
and T2
• local tissue shape
GP model of intensity changes
J O’Muircheartaigh, EC Robinson et al, Modelling brain development: investigating white
matter injury in term and preterm born neonates, submitted
32. Modelling brain Development with
Gaussian Process Regression
What would a term-aged infant
look like if they were born with
varying degrees of prematurity?
J O’Muircheartaigh, EC Robinson et al, Modelling brain development: investigating white
matter injury in term and preterm born neonates, submitted
33. Modelling brain Development with
Gaussian Process Regression
Comparisons of subjects versus
the group can be used to
identify punctate lesions
J O’Muircheartaigh, EC Robinson et al, Modelling brain development: investigating white
matter injury in term and preterm born neonates, submitted
Mean ROC
AUC=0.894
34. Polygenic Risk for Neuropsychiatric
Disease and Abnormal Deep Grey
Matter Development
• Polygenic Risk analysis
• Using genetic risk scores
estimated for 5 major
psychiatric disorders
(Smoller et al 2013)
• Applied to 194 preterm
eprime subjects
• Linked to deep grey matter
volume
Cullen, Harriet, et al. "Polygenic risk for neuropsychiatric disease and vulnerability to abnormal
deep grey matter development." Scientific reports 9.1 (2019): 1976.
35. • Increasing polygenic risk
score (PRS)
Ø Associated with reduced
lentiform nucleus volume
- In the full mixed-ancestral
cohort (R2 = 0.06, p = 8 ×
10−4)
- In the subsample of European
infants (R2=0.06, p = 8 ×
10−3)
●
● ●
●
●0.00
0.05
0.10
0.15
0.001 0.01 0.05 0.1 0.5
P value cut−off
VarianceExplainedbyPRS
●
●
●
●
●
0.00
0.05
0.10
0.15
0.001 0.01 0.05 * 0.1 * 0.5
P value cut−off
VarianceExplainedbyPRS
0.15
byPRS
0.15
byPRS
Full Mixed-ancestry
cohort (n=194)
Proportion of variance explained in the
lentiform nucleus volumes by the PRS at
five different P-value thresholds.
Polygenic Risk for Neuropsychiatric
Disease and Abnormal Deep Grey
Matter Development
36. Future/ongoing work
Fetal pipelines
- Deep learning driven tissue segmentation and surface
extraction
Christiaens et al., TMI 2019 ; Christiaens et al., ISMRM 2018
Subject with severe moti
37. Future/ongoing work
Fetal pipelines
- Deep learning driven tissue segmentation and surface
extraction
- Fetal diffusion advanced motion correction through
Spherical Harmonics And a Radial Decomposition
(SHARD)
Christiaens et al., TMI 2019 ; Christiaens et al., ISMRM 2018
Subject with severe moti
Subject with severe motion
39. Acknowledgements:
Contributors
44
• Abdulah Fawaz
• Kyriaki Kaza
My team:
Cher Bass Jonathan
O’Muircheartaigh
Daan Christiaens
Yassine
Benchekroun
Dr Chris Kelly
• Harriet Cullen
• Dr Lucillio
Cordero Grande
• Max Pietsch
• Dr Maria Deprez
• Dr Emer Hughes
• Dr Serena
Counsell
• Dr Jana Hutter
• Dr Adrian Price
• Dr J-Donald
Tournier
http://www.developingconnectome.org/teams-and-
collaborators-v2/
… And many more
43. Neonatal Volume QC
2 raters rated
• 160 images
Example QC from single rater
From left to right: poor
quality to best quality
44. Neonatal Surface QC
2 raters rated
• 43 images
• Patches of size
50x50x50mm
• White surface only
Comparison of intensity-based surface refinement (green) to segmentation result (yellow)
Example QC from single rater
