Large scale NGS pipelines using the MOLGENIS platform: processing the Genome of the Netherlands (Morris Swertz)

1. Large scale NGS pipelines using the MOLGENIS platform: processing the Genome of the Netherlands Morris Swertz , UMC Groningen, Netherlands and members of BBMRI-NL, NBIC, MOLGENIS BOSC 2011, July 15, Vienna

2. BOSC 2010 we demonstrated the MOLGENIS software toolkit Use (web) Animal Observatory NextGenSeq Mutation database Model organisms Model (xml) Generator (java) Swertz et al (2010) BMC Bioinformatics 11(Suppl 12):S12, http://www.molgenis.org

3. Get stuff for free as others build it already Connect to annotation services Plugin rich analysis tools Connect to statistics UML documentation of your model Edit & trace your data Import/export to Excel find.investigation() 102 downloaded obs<-find.observedvalue( 43,920 downloaded #some calculation add.inferredvalue(res) 36 added      

4. Three steps: Model –> Generate –> Use Swertz et al (2010) BMC Bioinformatics 11(Suppl 12):S12, http://www.molgenis.org

5. Three steps: Model –> Generate –> Use 9200 INFO [FormScreenGen] generated generatedavaicreenopMenuainrotocolsForm.java 9293 INFO [FormScreenGen] generated generatedavaicreenopMenuainrotocolsrotocolMenuarametersForm.java 9325 INFO [FormScreenGen] generated generatedavaicreenopMenuainrotocolsrotocolMenurotocolComponentsForm.java 9496 INFO [FormScreenGen] generated generatedavaicreenopMenuainntologiesntologyTermsForm.java 9528 INFO [FormScreenGen] generated generatedavaicreenopMenuainntologiesntologySourcesForm.java 9606 INFO [FormScreenGen] generated generatedavaicreenopMenuainntologiesntologySourcesntologyTermsForm.java 9638 INFO [FormScreenGen] generated generatedavaicreenopMenuainntologiesodeListsForm.java 9700 INFO [FormScreenGen] generated generatedavaicreenopMenuainntologiesodeListsodesForm.java 9965 INFO [MenuScreenGen] generated generatedavaicreenopMenuMenu.java 10012 INFO [MenuScreenGen] generated generatedavaicreenopMenuainMenu.java 10059 INFO [MenuScreenGen] generated generatedavaicreenopMenuainnvestigationsnvestigationMenuMenu.java 10152 INFO [MenuScreenGen] generated generatedavaicreenopMenuainnvestigationsnvestigationMenurotocolApplicationsrotocolApplicationMenuMenu.java 10230 INFO [MenuScreenGen] generated generatedavaicreenopMenuainbservationTargetsMenu.java 10293 INFO [MenuScreenGen] generated generatedavaicreenopMenuainrotocolsrotocolMenuMenu.java 10324 INFO [MenuScreenGen] generated generatedavaicreenopMenuainntologiesMenu.java 11354 INFO [PluginScreenGen] generated Molgenis33Workspaceolgenis4phenotypeeneratedavaicreenopMenuaineportPlugin.java 11557 INFO [PluginScreenGen] generated Molgenis33Workspaceolgenis4phenotypeeneratedavaicreenopMenuainntologiesntologyManagerPlugin.java 11604 INFO [PluginScreenGen] generated Molgenis33Workspaceolgenis4phenotypeeneratedavaicreenopMenuodel_documentationPlugin.java 11604 INFO [PluginScreenGen] generated Molgenis33Workspaceolgenis4phenotypeeneratedavaicreenopMenuprojectApiPlugin.java 11620 INFO [PluginScreenGen] generated Molgenis33Workspaceolgenis4phenotypeeneratedavaicreenopMenuttpApiPlugin.java 11635 INFO [PluginScreenGen] generated Molgenis33Workspaceolgenis4phenotypeeneratedavaicreenopMenuebServicesApiPlugin.java 11651 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwrittenavalugineportnvestigationOverview.ftl 11807 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwrittenavaluginntologyBrowserntologyBrowserPlugin.ftl 11807 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwrittenavaluginopmenuocumentationScreen.ftl 11807 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwrittenavaluginopmenuprojectApiScreen.ftl 11823 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwrittenavaluginopmenuttpAPiScreen.ftl 11823 WARN [PluginScreenFTLTemplateGen] Skipped because exists: handwrittenavaluginopmenuoapApiScreen.ftl 11854 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwrittenavalugineportnvestigationOverview.java 12057 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwrittenavaluginntologyBrowserntologyBrowserPlugin.java 12072 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwrittenavaluginopmenuocumentationScreen.java 12088 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwrittenavaluginopmenuprojectApiScreen.java 12088 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwrittenavaluginopmenuttpAPiScreen.java 12088 WARN [PluginScreenJavaTemplateGen] Skipped because exists: handwrittenavaluginopmenuoapApiScreen.java 12103 INFO [MolgenisServletContextGen] generated WebContentETA-INFontext.xml 12259 INFO [SoapApiGen] generated generatedavaioapApi.java 12353 INFO [CsvExportGen] generated generatedavaoolssvExport.java 12431 INFO [CsvImportByNameGen] generated generatedavaoolssvImportByName.java 12636 INFO [CopyMemoryToDatabaseGen] generated generatedavaioolsopyMemoryToDatabase.java Real example: Generates 150 files, 30k lines of Java, MySQL, CXF, Tomcat config, and R code + docs

6. Three steps: Model –> Generate –> Use Swertz et al (2010) BMC Bioinformatics 11(Suppl 12):S12, http://www.molgenis.org

7. Currently: Towards an integrated app suite XGAP for GWAS/GWL Disease specific databases BBMRI biobank catalogue GWAS central data manager NGS cyber infrastructure MAGE-TAB microarray AnimalDB Swertz et al (2010) BMC Bioinformatics 11(Suppl 12):S12, http://www.molgenis.org

10. Motivation: GWAS revolution in human genetics

11. Motivation: GWAS revolution in human genetics

12. Motivation: GWAS revolution in human genetics

13. Motivation: GWAS revolution in human genetics

14. Motivation: GWAS revolution in human genetics

15. GREAT! Ankylosing Spondylitis Celiac Disease Crohn’s disease Multiple Sclerosis Psoriasis Rheumatoid Arthritis Systemic Lupus Erythematosus Type 1 Diabetes Ulcerative Colitis

16. BUT … these explain a small part of heritability

17. Missing heritability? Where might it be hiding?

18. However: Sequencing candidate loci implicates unknown (rare) variants

19. More insight into the specific genetic architecture of individual populations is crucial First analysis of 1000G project data Durbin et al., Nature 2010 common known

20. More insight into the specific genetic architecture of individual populations is crucial First analysis of 1000G project data shows that the majority of the newly identified and rare variants are population specific (and there are no Dutch in 1000G) Durbin et al., Nature 2010 common known new

22. Idea 2: lets impute 100.000 existing Dutch GWAS data  Imputation is the process of inferring any missing or untyped genetic variants from typed flanking genetic variants, based on the known local LD relationship GWAS data

30. Challenge 2: Alignment, Variant Calling, and QC pipelines Alignment Variant calling Alignment to human genome (Build 37) Clean up alignment (mark duplicates, realignment, recalibration) Quality control SNP calling Indel calling Variant Filtering ~ 1 Week ~ 1 Week QC: Immunochip concordance

33. Challenge 4: Did we analyze it all? Correctly? Completely? Batches: UModqR 60 HUMcriR 90 HUMhxsR 222 HUMrutR 235 HUMjxbR 153 HUMsnrR 10

37. Solution 2: data management via sample-lane worksheet sample flowcell lane lib machine date file A24a FC80R35ABXX L3 HUMhxsRJODIAAPE I433 101119 101119_I433_FC80R35ABXX_L3_HUMhxsRJODIAAPE A24a FC80F2RABXX L3 HUMhxsRJODIABPE I481 101120 101120_I481_FC80F2RABXX_L3_HUMhxsRJODIABPE A24a FC80GHKABXX L2 HUMhxsRJODIBAPE I114 101202 101202_I114_FC80GHKABXX_L2_HUMhxsRJODIBAPE A24b FC80R35ABXX L4 HUMhxsRJPDIAAPE I433 101119 101119_I433_FC80R35ABXX_L4_HUMhxsRJPDIAAPE A24b FC80F2RABXX L4 HUMhxsRJPDIABPE I481 101120 101120_I481_FC80F2RABXX_L4_HUMhxsRJPDIABPE A24b FC80GHKABXX L3 HUMhxsRJPDIBAPE I114 101202 101202_I114_FC80GHKABXX_L3_HUMhxsRJPDIBAPE A24b FC81C8UABXX L3 HUMhxsRJPDIBAPE I340 110114 110114_I340_FC81C8UABXX_L3_HUMhxsRJPDIBAPE A24c FC80R35ABXX L5 HUMhxsRJQDIAAPE I433 101119 101119_I433_FC80R35ABXX_L5_HUMhxsRJQDIAAPE A24c FC80F2RABXX L6 HUMhxsRJQDIABPE I481 101120 101120_I481_FC80F2RABXX_L6_HUMhxsRJQDIABPE A24c FC80GHKABXX L4 HUMhxsRJQDIBAPE I114 101202 101202_I114_FC80GHKABXX_L4_HUMhxsRJQDIBAPE A25a FC80R35ABXX L6 HUMhxsRJRDIAAPE I433 101119 101119_I433_FC80R35ABXX_L6_HUMhxsRJRDIAAPE A25a FC81C8UABXX L2 HUMhxsRJRDIAAPE I340 110114 110114_I340_FC81C8UABXX_L2_HUMhxsRJRDIAAPE A25a FC80F54ABXX L7 HUMhxsRJRDIABPE I171 101122 101122_I171_FC80F54ABXX_L7_HUMhxsRJRDIABPE A25a FC80GHKABXX L5 HUMhxsRJRDIBAPE I114 101202 101202_I114_FC80GHKABXX_L5_HUMhxsRJRDIBAPE A25b FC80R35ABXX L7 HUMhxsRJSDIAAPE I433 101119 101119_I433_FC80R35ABXX_L7_HUMhxsRJSDIAAPE A25b FC80EE1ABXX L5 HUMhxsRJSDIABPE I171 101122 101122_I171_FC80EE1ABXX_L5_HUMhxsRJSDIABPE A25b FC80GHKABXX L6 HUMhxsRJSDIBAPE I114 101202 101202_I114_FC80GHKABXX_L6_HUMhxsRJSDIBAPE A25b FC80GHJABXX L1 HUMhxsRJSDIBAPE I117 101208 101208_I117_FC80GHJABXX_L1_HUMhxsRJSDIBAPE A25c FC80R35ABXX L8 HUMhxsRJTDIAAPE I433 101119 101119_I433_FC80R35ABXX_L8_HUMhxsRJTDIAAPE A25c FC80F54ABXX L5 HUMhxsRJTDIABPE I171 101122 101122_I171_FC80F54ABXX_L5_HUMhxsRJTDIABPE A25c FC80GHKABXX L7 HUMhxsRJTDIBAPE I114 101202 101202_I114_FC80GHKABXX_L7_HUMhxsRJTDIBAPE A25c FC81C7KABXX L5 HUMhxsRJTDIBAPE I125 110115 110115_I125_FC81C7KABXX_L5_HUMhxsRJTDIBAPE A26a FC80PEWABXX L5 HUMhxsRJUDIAAPE I198 101120 101120_I198_FC80PEWABXX_L5_HUMhxsRJUDIAAPE A26a FC80F2RABXX L7 HUMhxsRJUDIABPE I481 101120 101120_I481_FC80F2RABXX_L7_HUMhxsRJUDIABPE A26a FC80GHKABXX L8 HUMhxsRJUDIBAPE I114 101202 101202_I114_FC80GHKABXX_L8_HUMhxsRJUDIBAPE A26b FC80N58ABXX L5 HUMhxsRJVDIAAPE I245 101120 101120_I245_FC80N58ABXX_L5_HUMhxsRJVDIAAPE A26b FC80PNWABXX L2 HUMhxsRJVDIABPE I453 101119 101119_I453_FC80PNWABXX_L2_HUMhxsRJVDIABPE A26b FC80G37ABXX L1 HUMhxsRJVDIBAPE I127 101126 101126_I127_FC80G37ABXX_L1_HUMhxsRJVDIBAPE A26c FC80LDLABXX L1 HUMhxsRJWDIAAPE I453 101119 101119_I453_FC80LDLABXX_L1_HUMhxsRJWDIAAPE A26c FC80PNWABXX L3 HUMhxsRJWDIABPE I453 101119 101119_I453_FC80PNWABXX_L3_HUMhxsRJWDIABPE A26c FC80G37ABXX L2 HUMhxsRJWDIBAPE I127 101126 101126_I127_FC80G37ABXX_L2_HUMhxsRJWDIBAPE

41. Solution 5: a tool to submit and monitor compute jobs

47. Alignment results Alignment Variant calling Alignment to human genome (Build 37) Clean up alignment (mark duplicates, realignment, recalibration) Quality control Individual SNP calling Indel calling Variant Filtering ~ 1 Week ~ 1 Week >94% reads aligned >13x avg coverage

48. SNP calling result (GoNL Pilot Chr20 – 1KG Phase I) 16,045 177,389 648,284 1KG Estimated Chr20 Ti/Tv: 2.36 GoNL Pilot Only SNPs 16,045 %dbSNP 2.05 Ti/Tv 2.20 1KG Phase 1 Only SNPs 648,284 %dbSNP 10.23 Ti/Tv 2.36 Intersection SNPs 177,389 %dbSNP 65.91 Ti/Tv 2.41