Learning new climate science by thinking creatively with machine learning
•
0 gostou•67 visualizações
Presentation for: GFDL/AOS Summer Internship Lecture Series The popularity of machine learning is rapidly growing in nearly all areas of Earth science. However, there is also some hesitancy in adopting the use of these methods due to concerns about their reliability, reproducibility, and interpretability – thus, they are often described as “black boxes.” In this talk, I will introduce a few simple examples from climate science that leverage new visualization methods to peer into the machine learning “black box,” which help us to better understand their predictions while also learning new science. These same machine learning visualization tools can be easily adapted for a wide variety of applications and other scientific fields of study.
Recomendados
Recomendados
Mais conteúdo relacionado
Semelhante a Learning new climate science by thinking creatively with machine learning
Semelhante a Learning new climate science by thinking creatively with machine learning (20)
Mais de Zachary Labe
Mais de Zachary Labe (18)
Último
Último (20)
Learning new climate science by thinking creatively with machine learning
- 1. Learning new climate science by thinking creatively with machine learning @ZLabe Zachary M. Labe Postdoc in Princeton AOS and NOAA GFDL 28 June 2022 Summer Internship Lecture Series
- 2. Zack Labe Pioneer Coal Mine Blue Whale of Catoosa Centralia Underground Fire Roadside America Greenland Sea (81°N) Enjoy: roadside oddities, diners, and horror movies Hobbies: gardening, #scicomm ( @ZLabe), hiking, traveling to lighthouses Graduate work: Arctic – midlatitude climate variability (sea-ice changes) Recent work: using ANNs to detect patterns of climate variability/change Now at GFDL I am thinking about: detection and attribution of extreme events Relevance LENS X-Single Forcing Runs Hometown – Linglestown, Pennsylvania BSc – Atmospheric Science at Cornell University PhD – Earth System Science at UC Irvine Postdoc – climate variability using AI at Colorado State U. Postdoc – climate attribution with T. Delworth & N. Johnson he/him
- 3. What does machine learning mean to you? Question
- 4. Machine Learning is not new! But…
- 5. Machine Learning is not new!
- 6. Artificial Intelligence Machine Learning Deep Learning Computer Science
- 7. Computer Science Artificial Intelligence Machine Learning Deep Learning Supervised Learning Unsupervised Learning Labeled data Classification Regression Unlabeled data Clustering Dimension reduction
- 8. • Do it better • e.g., parameterizations in climate models are not perfect, use ML to make them more accurate • Do it faster • e.g., code in climate models is very slow (but we know the right answer) - use ML methods to speed things up • Do something new • e.g., go looking for non-linear relationships you didn’t know were there Very relevant for research: may be slower and worse, but can still learn something WHY SHOULD WE CONSIDER MACHINE LEARNING?
- 9. GROWING DATA
- 11. Adapted from: Kotamarthi, R., Hayhoe, K., Mearns, L., Wuebbles, D., Jacobs, J., & Jurado, J. (2021). Global Climate Models. In Downscaling Techniques for High-Resolution Climate Projections: From Global Change to Local Impacts (pp. 19-39). Cambridge: Cambridge University Press. doi:10.1017/9781108601269.003 CLIMATE MODELS Horizontal Grid Vertical Levels Past/Present/Future Fully-Coupled System 20-40 Petabytes of data
- 12. ADAPTED FROM EYRING ET AL. 2016 CMIP6
- 13. GROWING TOOLS
- 16. Python tools for machine learning
- 17. Today’s weather or climate scientist is far more likely to be debugging code written in Python… than to be poring over satellite images or releasing radiosondes. “ D. Irving| Bulletin of the American Meteorological Society| 2016
- 18. Machine learning for weather IDENTIFYING SEVERE THUNDERSTORMS Molina et al. 2021 Toms et al. 2021 CLASSIFYING PHASE OF MADDEN-JULLIAN OSCILLATION SATELLITE DETECTION Lee et al. 2021 DETECTING TORNADOES McGovern et al. 2019
- 19. Machine learning for climate FINDING FORECASTS OF OPPORTUNITY Mayer and Barnes, 2021 PREDICTING CLIMATE MODES OF VARIABILITY Gordon et al. 2021 TIMING OF CLIMATE CHANGE Barnes et al. 2019
- 20. Machine learning for oceanography CLASSIFYING ARCTIC OCEAN ACIDIFICATION Krasting et al. 2022 TRACK AND REVEAL DEEP WATER MASSES Sonnewald and Lguensat, 2021 ESTIMATING OCEAN SURFACE CURRENTS Sinha and Abernathey, 2021
- 24. NSF AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography (AI2ES) https://www.ai2es.org/
- 25. E.g., Research to Operations (R2O) Tornado Warning Special Marine Warning Severe Thunderstorm Warning Flash Flood Warning
- 26. E.g., Establish robust, responsible AI for severe weather detection Tornado Warning Special Marine Warning Severe Thunderstorm Warning Flash Flood Warning
- 27. Artificial Intelligence Machine Learning Deep Learning
- 28. X Y Our data
- 29. X Y Our data
- 30. What could be a problem with this type of model fitting? Question
- 31. Training Data (1880-1970) Validation Data (1971-2014) Testing Data (2015-2100) Algorithm learns patterns in the data to build a prediction model Tune the hyperparameters to select the ‘best’ model for your task Evaluate the skill of your prediction model on data it has never seen before Input maps of sea surface temperatures to identify El Niño or La Niña. Does the machine learning model have the same skill in a future climate?
- 32. Training Data (1880-1970) Validation Data (1971-2014) Testing Data (2015-2100) Algorithm learns patterns in the data to build a prediction model Tune the hyperparameters to select the ‘best’ model for your task Evaluate the skill of your prediction model on data it has never seen before Input maps of sea surface temperatures to identify El Niño or La Niña. Does the machine learning model have the same skill in a future climate?
- 33. Training Data (1880-1970) Validation Data (1971-2014) Testing Data (2015-2100) Algorithm learns patterns in the data to build a prediction model Tune the hyperparameters to select the ‘best’ model for your task Evaluate the skill of your prediction model on data it has never seen before Input maps of sea surface temperatures to identify El Niño or La Niña. Does the machine learning model have the same skill in a future climate?
- 34. Training Data (1880-1970) Validation Data (1971-2014) Testing Data (2015-2100) Algorithm learns patterns in the data to build a prediction model Tune the hyperparameters to select the ‘best’ model for your task Evaluate the skill of your prediction model on data it has never seen before Input maps of sea surface temperatures to identify El Niño or La Niña. Does the machine learning model have the same skill in a future climate?
- 35. X1 X2 INPUTS Artificial Neural Networks [ANN]
- 36. Linear regression! Artificial Neural Networks [ANN] X1 X2 W1 W2 ∑ = X1W1+ X2W2 + b INPUTS NODE
- 37. Artificial Neural Networks [ANN] X1 X2 W1 W2 ∑ INPUTS NODE Linear regression with non-linear mapping by an “activation function” Training of the network is merely determining the weights “w” and bias/offset “b" = factivation(X1W1+ X2W2 + b)
- 38. Artificial Neural Networks [ANN] X1 X2 W1 W2 ∑ INPUTS NODE = factivation(X1W1+ X2W2 + b) ReLU Sigmoid Linear
- 39. X1 X2 ∑ inputs HIDDEN LAYERS X3 ∑ ∑ ∑ OUTPUT = predictions Artificial Neural Networks [ANN] : : :: INPUTS
- 40. Complexity and nonlinearities of the ANN allow it to learn many different pathways of predictable behavior Once trained, you have an array of weights and biases which can be used for prediction on new data INPUT [DATA] PREDICTION Artificial Neural Networks [ANN]
- 41. TEMPERATURE
- 42. TEMPERATURE We know some metadata… + What year is it? + Where did it come from?
- 43. We know some metadata… + What year is it? + Where did it come from? TEMPERATURE
- 44. We know some metadata… + What year is it? + Where did it come from? TEMPERATURE Neural network learns nonlinear combinations of forced climate patterns to identify the year
- 45. ----ANN---- 2 Hidden Layers 10 Nodes each Ridge Regularization Early Stopping We know some metadata… + What year is it? + Where did it come from? [e.g., Barnes et al. 2019, 2020] [e.g., Labe and Barnes, 2021] TIMING OF EMERGENCE (COMBINED VARIABLES) RESPONSES TO EXTERNAL CLIMATE FORCINGS PATTERNS OF CLIMATE INDICATORS [e.g., Rader et al. 2022, in review] Surface Temperature Map Precipitation Map + TEMPERATURE
- 46. ----ANN---- 2 Hidden Layers 10 Nodes each Ridge Regularization Early Stopping We know some metadata… + What year is it? + Where did it come from? [e.g., Barnes et al. 2019, 2020] [e.g., Labe and Barnes, 2021] TIMING OF EMERGENCE (COMBINED VARIABLES) RESPONSES TO EXTERNAL CLIMATE FORCINGS PATTERNS OF CLIMATE INDICATORS [e.g., Rader et al. 2022, in review] Surface Temperature Map Precipitation Map + TEMPERATURE
- 47. THE REAL WORLD (Observations) What is the annual mean temperature of Earth?
- 48. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) Anomaly is relative to 1951-1980
- 49. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) Let’s run a climate model
- 50. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) Let’s run a climate model again
- 51. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) Let’s run a climate model again & again
- 52. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) CLIMATE MODEL ENSEMBLES
- 53. What is the annual mean temperature of Earth? THE REAL WORLD (Observations) CLIMATE MODEL ENSEMBLES Range of ensembles = internal variability (noise) Mean of ensembles = forced response (climate change)
- 54. What is the annual mean temperature of Earth? Range of ensembles = internal variability (noise) Mean of ensembles = forced response (climate change) But let’s remove climate change…
- 55. What is the annual mean temperature of Earth? Range of ensembles = internal variability (noise) Mean of ensembles = forced response (climate change) After removing the forced response… anomalies/noise!
- 56. What is the annual mean temperature of Earth? • Increasing greenhouse gases (CO2, CH4, N2O) • Changes in industrial aerosols (SO4, BC, OC) • Changes in biomass burning (aerosols) • Changes in land-use & land-cover (albedo)
- 57. What is the annual mean temperature of Earth? • Increasing greenhouse gases (CO2, CH4, N2O) • Changes in industrial aerosols (SO4, BC, OC) • Changes in biomass burning (aerosols) • Changes in land-use & land-cover (albedo) Plus everything else… (Natural/internal variability)
- 58. What is the annual mean temperature of Earth?
- 59. Greenhouse gases fixed to 1920 levels All forcings (CESM-LE) Industrial aerosols fixed to 1920 levels [Deser et al. 2020, JCLI] Fully-coupled CESM1.1 20 Ensemble Members Run from 1920-2080 Observations
- 60. So what? Greenhouse gases = warming Aerosols = ?? (though mostly cooling) What are the relative responses between greenhouse gas and aerosol forcing?
- 61. Surface Temperature Map ARTIFICIAL NEURAL NETWORK (ANN)
- 62. INPUT LAYER Surface Temperature Map ARTIFICIAL NEURAL NETWORK (ANN)
- 63. INPUT LAYER Surface Temperature Map ARTIFICIAL NEURAL NETWORK (ANN) Collection of nodes (neurons) that adjust their weights and biases across layers in order to learn signals for making predictions Learns nonlinear processes through selected parameters in the model
- 64. INPUT LAYER HIDDEN LAYERS OUTPUT LAYER Surface Temperature Map “2000-2009” DECADE CLASS “2070-2079” “1920-1929” ARTIFICIAL NEURAL NETWORK (ANN)
- 65. INPUT LAYER HIDDEN LAYERS OUTPUT LAYER Surface Temperature Map “2000-2009” DECADE CLASS “2070-2079” “1920-1929” BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI ARTIFICIAL NEURAL NETWORK (ANN)
- 66. INPUT LAYER HIDDEN LAYERS OUTPUT LAYER Layer-wise Relevance Propagation Surface Temperature Map “2000-2009” DECADE CLASS “2070-2079” “1920-1929” BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI ARTIFICIAL NEURAL NETWORK (ANN) [Barnes et al. 2020, JAMES] [Labe and Barnes 2021, JAMES]
- 67. LAYER-WISE RELEVANCE PROPAGATION (LRP) Volcano Great White Shark Timber Wolf Image Classification LRP https://heatmapping.org/ LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network WHY WHY WHY Backpropagation – LRP
- 68. LAYER-WISE RELEVANCE PROPAGATION (LRP) Volcano Great White Shark Timber Wolf Image Classification LRP https://heatmapping.org/ LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network WHY WHY WHY Backpropagation – LRP
- 69. LAYER-WISE RELEVANCE PROPAGATION (LRP) Volcano Great White Shark Timber Wolf Image Classification LRP https://heatmapping.org/ LRP heatmaps show regions of “relevance” that contribute to the neural network’s decision-making process for a sample belonging to a particular output category Neural Network Backpropagation – LRP WHY WHY WHY
- 70. LAYER-WISE RELEVANCE PROPAGATION (LRP) Image Classification LRP https://heatmapping.org/ NOT PERFECT Crock Pot Neural Network Backpropagation – LRP WHY
- 71. Returning to something we already know…
- 72. Neural Network [0] La Niña [1] El Niño [Toms et al. 2020, JAMES] Input a map of sea surface temperatures
- 73. Visualizing something we already know… Input maps of sea surface temperatures to identify El Niño or La Niña Use ‘LRP’ to see how the neural network is making its decision [Toms et al. 2020, JAMES] Layer-wise Relevance Propagation Composite Observations LRP [Relevance] SST Anomaly [°C] 0.00 0.75 0.0 1.5 -1.5
- 74. [Adapted from Adebayo et al., 2020] EXPLAINABLE AI IS NOT PERFECT THERE ARE MANY METHODS
- 75. [Adapted from Adebayo et al., 2020] THERE ARE MANY METHODS EXPLAINABLE AI IS NOT PERFECT
- 76. OUTPUT LAYER Layer-wise Relevance Propagation “2000-2009” DECADE CLASS “2070-2079” “1920-1929” BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI WHY? = LRP HEAT MAPS [Labe and Barnes 2021, JAMES]
- 77. Layer-wise Relevance Propagation BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI WHY? = LRP HEAT MAPS Machine Learning Black Box [Labe and Barnes 2021, JAMES]
- 78. Layer-wise Relevance Propagation BACK-PROPAGATE THROUGH NETWORK = EXPLAINABLE AI WHY? = LRP HEAT MAPS Find regions of “relevance” that contribute to the neural network’s decision-making process [Labe and Barnes 2021, JAMES]
- 79. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS Greenhouse gases fixed to 1920 levels [AEROSOLS PREVAIL] Industrial aerosols fixed to 1920 levels [GREENHOUSE GASES PREVAIL] All forcings [STANDARD CESM-LE] DATA
- 80. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS Greenhouse gases fixed to 1920 levels [AEROSOLS PREVAIL] Industrial aerosols fixed to 1920 levels [GREENHOUSE GASES PREVAIL] All forcings [STANDARD CESM-LE] DATA
- 81. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS Greenhouse gases fixed to 1920 levels [AEROSOLS PREVAIL] Industrial aerosols fixed to 1920 levels [GREENHOUSE GASES PREVAIL] All forcings [STANDARD CESM-LE] DATA
- 82. 1960-1999: ANNUAL MEAN TEMPERATURE TRENDS Greenhouse gases fixed to 1920 levels [AEROSOLS PREVAIL] Industrial aerosols fixed to 1920 levels [GREENHOUSE GASES PREVAIL] All forcings [STANDARD CESM-LE] DATA
- 83. CLIMATE MODEL DATA PREDICT THE YEAR FROM MAPS OF TEMPERATURE AEROSOLS PREVAIL GREENHOUSE GASES PREVAIL STANDARD CLIMATE MODEL [Labe and Barnes 2021, JAMES]
- 84. OBSERVATIONS PREDICT THE YEAR FROM MAPS OF TEMPERATURE AEROSOLS PREVAIL GREENHOUSE GASES PREVAIL STANDARD CLIMATE MODEL [Labe and Barnes 2021, JAMES]
- 85. OBSERVATIONS SLOPES PREDICT THE YEAR FROM MAPS OF TEMPERATURE AEROSOLS PREVAIL GREENHOUSE GASES PREVAIL STANDARD CLIMATE MODEL [Labe and Barnes 2021, JAMES]
- 86. HOW DID THE ANN MAKE ITS PREDICTIONS?
- 87. HOW DID THE ANN MAKE ITS PREDICTIONS? WHY IS THERE GREATER SKILL FOR GHG+?
- 88. RESULTS FROM LRP [Labe and Barnes 2021, JAMES] Low High
- 89. RESULTS FROM LRP [Labe and Barnes 2021, JAMES] Low High
- 90. RESULTS FROM LRP [Labe and Barnes 2021, JAMES] Low High
- 91. RESULTS FROM LRP [Labe and Barnes 2021, JAMES] Low High
- 92. RESULTS FROM LRP [Labe and Barnes 2021, JAMES]
- 93. ----ANN---- 2 Hidden Layers 10 Nodes each Ridge Regularization Early Stopping TEMPERATURE We know some metadata… + What year is it? (Labe & Barnes, 2021) + Where did it come from?
- 94. TEMPERATURE We know some metadata… + What year is it? (Labe & Barnes, 2021) + Where did it come from? Train on data from the Multi-Model Large Ensemble Archive
- 96. TEMPERATURE We know some metadata… + What year is it? (Labe & Barnes, 2021) + Where did it come from? NEURAL NETWORK CLASSIFICATION TASK HIDDEN LAYERS INPUT LAYER INPUT LAYER OUTPUT LAYER HIDDEN LAYERS
- 97. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 98. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 99. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 100. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 101. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 102. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 103. COMPARING CLIMATE MODELS LRP (Explainable AI) Raw data (Difference from multi-model mean) Colder Warmer High Low
- 105. MACHINE LEARNING IS JUST ANOTHER TOOL TO ADD TO OUR WORKFLOW. 1)
- 106. MACHINE LEARNING IS NO LONGER A BLACK BOX. 2)
- 107. WE CAN LEARN NEW SCIENCE FROM EXPLAINABLE AI. 3)
- 109. KEY POINTS 1. Machine learning is just another tool to add to our scientific workflow 2. We can use explainable AI (XAI) methods to peer into the black box of machine learning 3. We can learn new science by using XAI methods in conjunction with existing statistical tools Zachary Labe zachary.labe@noaa.gov @ZLabe Labe, Z.M. and E.A. Barnes (2021), Detecting climate signals using explainable AI with single-forcing large ensembles. Journal of Advances in Modeling Earth Systems, DOI:10.1029/2021MS002464 Labe, Z.M. and E.A. Barnes (2022), Comparison of climate model large ensembles with observations in the Arctic using simple neural networks, in review, DOI:10.1002/essoar.10510977.2
- 110. • Are there any challenges you have experienced in pre/post-processing Earth science data? • If traditional statistics already perform well on a prediction task, should we even bother trying machine learning techniques for it? • What are some potential issues for using machine learning methods in your summer research projects? • What are some of the best ways to get started in creating a machine learning model and selecting hyperparameters? Discussion