HDF-EOS at NOAA/NESDIS
•Transferir como PPT, PDF•
1 gostou•554 visualizações
Source: http://hdfeos.org/workshops/ws04/presentations/meng/hdfeos4.ppt
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (19)
Semelhante a HDF-EOS at NOAA/NESDIS
Semelhante a HDF-EOS at NOAA/NESDIS (20)
Mais de The HDF-EOS Tools and Information Center
Mais de The HDF-EOS Tools and Information Center (20)
Último
Último (20)
HDF-EOS at NOAA/NESDIS
- 1. HDF-EOS at NOAA/NESDIS Huan Meng, Doug Moore, Limin Zhao, Ralph Ferraro NOAA / NESDIS / ORA http://orbit-net.nesdis.noaa.gov/arad2/MSPPS hmeng@nesdis.noaa.gov NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 2. Presentation Outline Microwave Surface and Precipitation Products (MSPPS) Project Background MSPPS System Introduction Product Display and Monitoring Lessons Learned Using HDF-EOS Future Plans Other NESDIS Projects Using HDF/HDF-EOS Summary NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 3. MSPPS Project Background Project Motivation NWS/NCEP interest and demand Team Setup Satellite and Instruments NOAA-15 or NOAA-K (May 13, 1998) AMSU-A (23.8 ~ 89.0 GHz) AMSU-B (89.0 ~ 190.31 GHz) Project Goal Produce near-real-time operational surface and precipitation products from AMSU-A and AMSU-B antenna temperatures NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 4. MSPPS System Introduction Day-1 System (operational, near-real-time) Product suite Level 1 ~ 3 data processing procedure HDF-EOS file structure Prototype Day-2 System (for research, next day) Combined AMSU-A and -B swath/grid files Day-2 System (testing, near-real-time) Product suite Level 1 ~ 3 data processing procedure HDF-EOS file structure NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 5. Uniqueness of MSPPS for NESDIS First Using HDF-EOS Format Workstation Processing Day-1 System Operational in 12 ~ 18 Months NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 6. Day-1 Product Suite Product Surface AMSU-A Antenna Temperature Snow Cover Rain Rate Sea Ice Concentration Total Precipitable Water Cloud Liquid Water NOAA / NESDIS / ORA Global Land Global Ocean Ocean Ocean http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS AMSU-B
- 7. MSPPS Day-1 System AMSU-A, -B Level-1B Data Ingestion 1b (Binary) NCEP CIRA (BUFR) (HDF-EOS) NCEP GDAS NCEP Rainfall (GriB -> HDF-EOS) (GriB) Level-2 Product Generation SWATH (HDF-EOS) Level-3 Product Generation NOAA / NESDIS / ORA SWATH (HDF-EOS) GRID (HDF-EOS) Product Radiosonde (Binary) Validation & Monitoring SSM/I (Binary -> HDF-EOS) http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS TRMM (HDF) ARM (netCDF)
- 8. Day-1 AMSU-A File Structures (HDF-EOS format) AMSU-A Swath Attributes Geolocation, etc. Antenna Temperatures Products AMSU-A Grid - Geographic Geolocation, etc. Antenna Temperatures Products AMSU-A Grid - Polar Stereographic Geolocation, etc. Products NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 9. Day-1 AMSU-B File Structures (HDF-EOS format) AMSU-B Swath Attributes Geolocation, etc. Antenna Temperatures AMSU-B Grid - Geographic Geolocation, etc. Antenna Temperatures NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 10. Prototype Day-2 System ‘Combined’ Swath File Input from both AMSU-A and AMSU-B swaths Day-1 products using improved, physically-based, multispectral algorithms in day-2 More products Improved resolution and sensitivity ‘Combined’ Grid File - Geographic Projection Web site utilization NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 11. Day-2 Product Suite Product Surface Antenna Temperature Snow Cover Rain Rate Ice Water Path Sea Ice Concentration Total Precipitable Water Cloud Liquid Water Emissivity Surface/Skin Temperature Snow Depth Ocean Wind Speed Surface Wetness NOAA / NESDIS / ORA Global Land Global Global Ocean Ocean Ocean Land Land Land Ocean Land AMSU-A AMSU-B http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 12. MSPPS Day-2 System AMSU-A Level-1B Data Ingestion AMSU-B Level-1B Data Ingestion 1b (Binary) 1b (Binary) NOAA / NESDIS / ORA AMSU-A Level-2 Product Generation AMSU-B Preliminary Level-2 Data SWATH (HDF-EOS) Level-3 Product Generation SWATH (HDF-EOS) SWATH (HDF-EOS) AMSU-B Level-2 Product Generation http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS GRID (HDF-EOS) Product Validation & Monitoring
- 13. Day-2 AMSU-B File Structures (HDF-EOS format) AMSU-B Swath Attributes Geolocation, etc. Antenna Temperatures Products AMSU-B Grid - Geographic Geolocation, etc. Antenna Temperatures Products NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 14. Product Display and Monitoring Display Tools IDL, GUI for validation with subsetting SeaDAS, view_hdf, LinkWinds/WebWinds, GrADS Web Site http://orbit-net.nesdis.noaa.gov/arad2/MSPPS Display and Monitoring Day-1 image products Day-1 product monitoring Climate images Day-2 image products and comparison with day-1 NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 15. Lessons Learned Using HDF-EOS Advantages of HDF-EOS Past Challenges Format conversion to and from HDF-EOS Frequent version updates Compression method AUTOMERGE HDFE_NOMERGE Skipping Huffman gzip Continuing Problems Array size, array number? Writing to an existing swath NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 16. Future Plans NOAA-16 or NOAA-L Launch date: Sept. 20, 2000 Instruments: AMSU-A and AMSU-B Operational Day-2 System NOAA-16 product system System delivery: Dec. 2000 Validation of Day-2 Products Launch of DMSP SSMIS Launch date: Dec. 2000 NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 17. Other NESDIS Projects Using HDF/HDF-EOS Atmosphere Infrared Sounder (AIRS) Project EOS_AQUA, July 2001 Input (simulated) level 1b data in HDF-EOS format from JPL Output desired fields in HDF format to NCEP within 3 hrs Unified Validation Project Group 1: Input NCEP observation data in BUFR to HDF format Operational, next day Group 2: NOAA / NESDIS / ORA Input binary observation data and convert to HDF-EOS format Validation of AIRS, SSM/I, SSMIS, and IPO products http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
- 18. Summary HDF-EOS as Standard Output Format for MSPPS Increased Utilization of HDF-EOS at NESDIS Advantages of HDF-EOS Application Remaining Challenges NOAA / NESDIS / ORA http:// orbit-net.nesdis.noaa.gov/arad2/MSPPS
Notas do Editor
- NESDIS: National Environmental Satellite, Data, and Information Service ORA: Office of Research and Applications MSPPS: Microwave Surface and Precipitation Products System MSPPS is our team project, and the introduction to the MSPPS project will form the main body of this presentation
- Briefly speak about project background. Give general overview of our product generation system. Demonstrate project web site: show where we display our products, and how we monitor the system. Talk about the lessons we’ve learned in the past two years, and some on-going problems we have using HDF-EOS. Outline our plans for the near future. Point out other projects at NESDIS that use HDF and/or HDF/EOS data format. Presentation summary.
- The motivation for the project was that the weather forecasters and modelers from NWS/NCEP were interested in the generation of surface and precipitation products from passive microwave intruments. They require these products in real-time to help with their own work. Because of this demand, NESDIS set up the MSPPS project in early Fiscal Year 1996. There has been some personnel changes over the years, but the members of the team have usually consisted of 3 NESDIS scientists and 3~4 contractors from the QSS Group, Inc. located in Lanham, MD. The data we process are collected by two passive microwave radiometers called AMSU-A and AMSU-B (Advanced Microwave Sounding Unit). These instruments are carried on board a polar orbiting satellite called NOAA-15, which was known as NOAA-K before launch. This satellite was launched on May 13, 1998, and is the first in a series of NOAA satellites which carry these advanced microwave instruments. The purpose of the AMSU sensors is to collect data for the generation of temperature, moisture, surface, and hydrological products in cloudy regions, where visible and infrared instruments have decreased capability. AMSU-A is a 15-channel, cross-track scanning, passive microwave radiometer. Its frequency ranges from 23.8 to 89.0 GHz. It has 30 Earth views on each scan line. AMSU-B has 5 channels, is comprised of 90 Earth views, and has a frequency ranging from 89.0 to 190.31 GHz. The resolution of AMSU-B is 3 times higher than that of AMSU-A. Our project goal is to
- The MSPPS system is composed of various subsystems. I am going to give more details about the product generation subsystems, which includes our operational Day-1 system, our prototype day-2 system, which is currently running in-house, and our day-2 system which is undergoing testing.
- The MSPPS project is unique for NESDIS in these aspects. It’s the first project at NESDIS using HDF-EOS format. The whole MSPPS system operates on an SGI workstation with IRIX 6.2 operating system, instead of on a mainframe computer. Our day-1 system was set up quickly after NOAA-15 was launched and started to run operational in-house in 12 months after the launched, and officially operational in 18 months after the launch.
- Antenna temperature is derived from raw counts for the 15 channels of AMSU-A, and the 5 channels of AMSU-B. The surface products are generated using AMSU-A data. AMSU-B experienced unexpected problems during its first one and a half years of operation, and its data were unusable. This is the reason there are no AMSU-B products for day-1. The problem was caused by signal interference between data collection and data transmission. This problem has since been fixed, and we’ve been using AMSU-B data to generate some of the day-2 products.
- NCEP: National Centers for Environmental Prediction CIRA: Cooperative Institute for Research in the Atmosphere at CSU SSM/I: microwave instrument on board of Defense Meteorological Satellite Programs’ (DMSP) polar satellite. MSPPS day-1 system has been running operational since Jan 2000. The binary level-1B data is pushed to us in near-real-time. It contains the raw counts, geolocation information, etc. At level-2, antenna temperatures are derived from the raw counts for both AMSU-A and AMSU-B channels, and day-1 products are generated. The output data are stored in HDF-EOS swath files, and are then provided to NCEP and CIRA. Level-2 is processed in near-real-time. The level-3 mapped grid file is generated the next day, when all of the swaths from the previous day become available. The grid file is also stored in HDF-EOS format. Two kinds of grid files are generated: geographic and polar stereographic projections. The final components of day-1 system development are product validation, which has been completed, and the on-going monitoring of the products. Product validation used data from both level-2 swath files and level-3 grid files. Multispectral data sources were used for the validation of the day-1 products. System monitoring takes place mainly on our web page. Various monitoring tools have been developed during the project. We use these tools to display our products on our website, as well as their comparison with other product sources.
- AMSU-A Swath Attributes: dimension, navigation, scaling factors Geolocation: time, TAI93 (HDF-EOS time toolkit), lat., lon. etc.: ancillary data such as lza, sza The contents of each file varies slightly, and is dependant upon the purpose of the file.
- AMSU-B files are very similar to AMSU-A files, but no product is generated from AMSU-B data in the day-1 system.
- Prototype Day-2 System: This system is currently running in-house, and is used mainly for research purposes. It runs the next day, just like the Day-1 mapped grid file. This system reads in data from both the day-1 level-2 AMSU-A swath, and the day-1 level-2 AMSU-B swath. I will talk more about the day-2 products later. The advantages of the day-2 system over the day-1 system are: 1) new, physically based, multispectral algorithms are developed for the products previously included in the day-1 system (TPW, CLW, Sice, RR); 2) more products are generated using data from both sensors (IWP, Emissivity, SurfTmp); 3) 3x higher resolution for the AMSU-B products; 4) use of higher frequency AMSU-B channels allows for improved algorithms (RR, Snow). The prototype day-2 system is composed of both level-2 swath files and level-3 grid files. The grid file is used to generate day-2 product images and some comparison images. These images are shown on our web site.
- Green indicates a product from day-1 that remains unchanged in day-2. Black represents either a new product for day-2, or a product that was included in the day-1 suite but is now being generated in the day-2 system using an improved algorithm. Blue designates a product which is currently not included in the day-2 suite, but will be in the future.
- The day-2 system has been set up to run internally for testing purposes. In the day-2 system, AMSU-A data processing follows the same essential steps as in the day-1 system. However, AMSU-B has added complexity because generation of AMSU-B products requires data from both instruments. So for the AMSU-B data processing, a preliminary swath is first generated which includes all the information from AMSU-B except products. Then, this preliminary swath and the corresponding AMSU-A swath are opened, data is read in from the two files, AMSU-B products are computed and a final AMSU-B swath is generated which includes all of the information from the preliminary AMSU-B swath as well as the AMSU-B products. We then generate the level-3 mapped grid files for both AMSU-A and AMSU-B. As for the day-1 products, the day-2 products will undergo extensive validation. Some of the day-2 products are displayed on our web site.
- The day-2 AMSU-A file structures remain the same even though the product algorithms have changed. The only change to the AMSU-B files in terms of file structure is the addition of products to AMSU-B files.
- There are quite a few graphic software packages currently available which support HDF-EOS format. Among them, IDL is the main graphic tool we use for generating the images on our web site and for product validation. An in-house GUI based on IDL was developed for validation. It includes the ability to do simple subsetting. SeaDAS, etc. are also used in our project. I personally use SeaDAS a lot. It was developed at NASA for the SeaWIFS project. It’s based on IDL and supports HDF format. Besides the product generation subsystems, MSPPS also has some integrated subsystems to perform product monitoring and product validation against a variety of data from other sources. The results are displayed on our web site. Show web site Display and Monitoring: 1. Day-1 image products: AMSU-A AT1, AMSU-B AT1, TPW, RR 2. Day-1 product monitoring: TPW (SSM/I), RR (NCEP), Sea Ice (SSM/I) polar stereographic images; Scatter plot: TPW (0 ~ 30) Time series: TPW (0~30N) 3. Climate: TPW (all) 4. Day-2 and comparison with day-1: TPW image, TPW comparison, TPW scattering plot
- As a standard data format, HDF-EOS has some clear advantages. We’ve been enjoying its ability to carry out internal compression, its portability, flexibility, and self-defining features. In short, we like it! We are also happy to see more and more tools supporting the HDF-EOS format becoming available. We have experienced, and continue to experience challenges using HDF-EOS format. The decision was made to use HDF-EOS as the standard data format for MSPPS at the early stage of the project. We were the first to use the format at NESDIS. As an early HDF-EOS user and data provider at NESDIS, we have had some unique obstacles to overcome. A few examples are 1) when we exchange data with other data users and providers, we have to perform various format conversion to and from HDF-EOS; 2) there were frequent HDF-EOS version updates, sometimes an update solved a problem, sometimes it added more problems; 3) we also had problems with the compression method, AUTOMERGE has to be changed to HDFE_NOMERGE after a version update. Gzip compresses data better than skipping Huffman, especially for grid file. But gzip was not available in early versions. Reprocessing? Continuing Problems 1) To generate grid files, sometimes we have to change file dimension from 360 x 720 to 361 x 721 or to add a dummy array in the array defining header file for the program to run. 2) In day-2 system, we failed to append products to the existing preliminary AMSU-B swath. It has to be created all over again. From the HDF-EOS documents, we have the impression that it’s possible to write to an existing HDF-EOS file, but unfortunately we are unable to do so.
- The next satellite in the NOAA series is NOAA-16 or NOAA-L. It’s scheduled to be launched on Sept. 20, 2000. Day-2 system is planned to be the operational system for NOAA-16. SSMIS is a sensor similar to SSM/I. It’s on board DMSP’s polar satellite which is scheduled to be launched in Dec. 2000. SSMIS will serve as another data source for our product validation.
- AIRS Project for EOS_AQUA that’s scheduled to be launched in July 2001. NASA project. Level 1b swath data, calibrated, navigated data (radiance, BT) Output to NCEP, numerical model team Unified Validation Project: In an effort to unify input data for validation at NESDIS. Two groups of people are working on the project. Group 1: - NCEP observation data: radiosonde, ground observation, buoy, ship, aircraft. Output 4 times/day. - Plan to adopt HDF format as standard format also for satellite data at NESDIS. Group 2:
- HDF-EOS is the standard data format for MSPPS project. It’s used for all of our output files (swath, grid). We can see that HDF and HDF-EOS formats are increasing in popularity at NESDIS. We’ve gained lots of experience in our two years using HDF-EOS. We recognize and enjoy the various advantages of the format. However, as we are also facing some challenges associated with the data format, we are hoping to get needed help from the HDF-EOS experts at this workshop. We would greatly appreciate any assistance that can be provided.