Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Hdf eos status-workshp_xi_nov_2007
1. Current Status of HDF-EOS,
Maintenance, Developments,
and Tools
Abe Taaheri, Raytheon IIS
HDF & HDF-EOS Workshop XI
November 2007
Page 1
2. TOOLKIT / HDF-EOS Support
•
•
Maintenance support is funded on EMD Evolution
through May 2008
One year extension until May 2009
•
Current staff
- One full-time developer for maintaining:
SDP/MTD Toolkit
HDF-EOS2 & 5
HDF-EOS plug-in for HDFView
HDF-EOS2 to GeoTIFF Converter (HEG)
Other Toolkit/HDF-EOS related software
Toolkit/HDF-EOS/HEG helpdesk
- One half-time developer for HEG maintenance
Page 2
3. ECS Support of HDF-EOS
• Both HDF4 and HDF5 based flavors of HDF-EOS are part
of ECS baseline.
• OS’s: Solaris (9, 10), Irix6.5 (64, n32), Windows XP,
Linux (including 64-bit Opteron and Itanium),
Mac OS X (MTD Toolkit, HDF-EOS2 & 5)
(Dropped support for Solaris 8, HP, HP11, Dec, IBM for next release)
• adding Mac Intel support for 2007 release and full Cygwin
support for 2008
• Compilers: Fortran 77/90 & g77/pgf90 , C, C++, gcc, g++
Page 3
4. Downloadable Software and
Applications
• SDP/MTD Toolkit (Science Data Processing Tools)
• HDF-EOS2 (HDF4 based)
• HDF-EOS5 (HDF5 based)
• HE5View* (HDF5 - based browser)
• EOSView* (HDF4 - based browser)
• HDFView, Java-based browser (HDF4, HDF5, HDFEOS 2 and 5 access)
• heconvert (converts HDF4 - based
Grid/Point/Swath structures to HDF-EOS 5
equivalents)
• HEG (GeoTIFF conversion, Spatial subsetting, Field
Subsetting, reprojection, stitching, sub-sampling,
etc.)
* Dropped support for OS updates on July 2006
Page 4
5. Software Releases
Toolkit/HDF-EOS:
• Last release – March 2006
TOOLKIT 5.2.14, HDF-EOS 2.14, HDF-EOS5.1.10
with HDF4.2r1, HDF5-1.6.5-post5, Szip2.0
• Next release – November 2007
TOOLKIT 5.2.15, HDF-EOS 2.15, HDF-EOS5.1.11
with new releases of HDF4, HDF5, and SZIP
HDF-EOS Plug-in for HDFView:
• Last release – May 2006
– Version 2.3
• Next release – Dec. 2007
– HDFView2.4 was released October 31 by The HDF
Group
Page 5
6. Software Releases (Cont.)
HEG (HDF-EOS2 to GeoTIFF Converter):
• Last release – July 2007
– Version 2.8
• Next release – May 2008
– Version 2.9
Page 6
7. SDP Toolkit
•
Science Data Production (SDP) Toolkit is a collection of
tools used by
- Science software developers who produce
code to process instrument data.
- HDF-EOS users who need to
• Access metadata in HDF-EOS files
• Perform Time/Date conversion
• Access Digital Elevation Model Data
• Access Ancillary Data, such as Digital Chart
of the World database (DCW), Olson World
Ecosystem files, DEM data files, etc.
Page 7
8. SDP Toolkit (cont.)
•
Tools provided in (SDP, MTD) Toolkit:
- AA
(Ancillary Data Access)
- CBP
(CELESTIAL Body Position)
- CSC
(Coordinate System Conversion)
- CUC
(Constant and Unit Conversions)
- DEM
(Digital Elevation Model Access)
- EPH
(Ephemeris Data Access)
- GCT
(Geo Coordinate Transformation)
- IO
(Input Output (File I/O))
- MEM
(Memory Management)
- MET
(Metadata Access)
- PC
(Process Control)
- SMF
(Status Message File (Error/Status))
- TD
(Time and Date Conversion)
Page 8
9. What is HDF-EOS ?
• HDF-EOS is an extension to HDF which
standardizes storage of and access to common
Earth Sciences data structures
–By providing a single interface to data structures
common to earth science
–By providing a container for EOS inventory,
archive and product specific metadata
–By establishing a consistent relationship
between geolocation and science data.
Page 9
10. HDF-EOS 2
• HDF4 - based, storage format for EOS standard
products.
• Used operationally by MODIS, MISR, ASTER,
Landsat, AIRS and other EOS instruments
• Support for Grid/Point/Swath structures
• HDF4 and HDF-EOS 2 will be supported by
THG/NASA for the fore-seeable future.
Page 10
11. HDF-EOS 5
• Based on HDF5, a complete rewrite of HDF-EOS2
with a different user interface.
–First released in 2000.
–Format used by EOS Aura instruments
• Designed to ‘resemble’ HDF-EOS 2 to the
maximum extent possible.
–Supports same data structures
–Added prefix ‘HE5_’ to HDF-EOS 2 functions.
Page 11
12. HDFView
•
•
•
•
•
•
HDFView: A Java-based browser from THG for HDF4
and HDF5 – based data.
HDF-EOS plug-in extends HDFView to browse HDFEOS data in both versions.
The extension is a plug-in module and does not
change THG HDFView functionality. All copy, save,
editing, image display, etc. for HDF4 and 5 objects
remains.
Identical look and feel when displaying HDF and
HDF-EOS objects.
Users can access a complete breakdown of HDFEOS objects. e.g. grid information, projection info.,
dimensions, datafields, and attributes are displayed.
Current Support: SUN, SGI, LINUX, Mac, WINDOWS
platforms
Page 12
16. HDF-EOS to GeoTIFF Converter (HEG)
HEG: Utility that converts EOSDIS data from
HDF-EOS format to common Geographical
Information System (GIS)-compatible formats.
• Two versions available:
– A downloadable desktop version
– Access through NASA archive online storage (Data Pools)
– Implemented also in prototypes for Stand-alone Data Pool,
OGC (Open Geospatial Consortium) for Web Services
Chaining, and DOWS (Deploy OGC Web Services) on the
DAACs
Page 16
17. HEG Functionality
• Conversion:
– HDF-EOS (Swath or Grid ) to a single-band or multi-
•
•
•
•
band GeoTIFF’s
– HDF-EOS Swath to HDF-EOS Grid and Swath
– HDF-EOS Swath or Grid to generic Binary (with
metadata file)
Subsetting (spatial, field, band)
Stitching (mosaicing) + subsetting + Reprojecting
Reprojection ( to UTM, PS, TM, STP, LCC, LAMAZ, GEO, SIN,
Albers)
Subsampling (Subsample stacks)
• Running from Command Line (batch jobs)
or with Java-based GUI
Page 17
18. HEG Functionality
• Metadata preservation/creation.
• Currently supports MODIS, MISR, ASTER, AIRS and AMSR-E
products on TERRA and AQUA (>110 products),
• Allows file selection from users local storage
• Operable on Sun, SGI, Win, Linux, MAC
• Integrated into ECS Data Pool
– Reduces the transfer time of HDF-EOS data sets (if subset is requested)
– Provides the end-user with the exact file required by their application
• User Interface
– Portable, written in Java.
– Not dependent on COTS (eg. IDL).
Page 18
21. AST_L1B, 2003 California Fires
Geo. projection, Multi-Band GeoTIFF,
(3 bands: Visible Near Infrared (VNIR))
….. and Subsetted
Two Swaths Stitched …..
Page 21
22. Major Development/Fixes
Next Release: November 2007
SZIP Insatallation: Default decoder, Choice for encoder
Choice for configuring HDF4 with/without SZIP
Choice for disabling HDF4’s internal netCDF
Choice for installing Toolkit/HDF-EOS in 32 or 64-bit
modes in 64-bit Linux platforms
Support for new releases of HDF4/HDF5/SZIP
(HDF4.2r2, HDF5-1.6.6, SZIP-2.1)
Installation fix for a problem caused by limits.h in some
Linux platforms
Auto Configuration capability (like HDF) added for both
HDF-EOS2 and HDF-EOS5
Support for Mac Intel platform. Drop of support for
Sun5.8, Dec, HP, HP 11, IBM
Fixes for Lat/Lon to pixel conversions in DEM tools for
Linux to get results similar to those in Unix platforms Page 22
23. Major Development/Fixes (cont.)
HDF-EOS Plug-in for HDFView:
Current Support:
SUN, SGI, LINUX, WINDOWS,
and MAC platforms
Last Release: Version 2.3
– Display Point Objects
– Display pixel's lat/lon for swath and grid images
– Scientific notation in Table data
Next Release: Version 2.4
– Auto installation of plug-in into HDFView running selfextracting archives.
– Display image for selected pixels in the viewed tables
– Display Lat/Lon for a selected pixel in the data field tables
– Mac OSX fixes for one-button mice
– Better display of Plug-in User’s Guide
– New icons for activating Lat/Lon display for Swath & Grid
Page 23
24. Major Development/Fixes (cont.)
HEG: Last Release (Version 2.8)
– Added Support for:
MOD02SSH, MYD02SSH, MOD14A1.v005, MYD14A1.005
MOD29P1D.V005, MOD08_M3.v005, MOD11_L2.v005
AMSR-E Snow and Ice, AMSR-E Ocean
– MISR Bug fixes & enhancements:
Added “No-reprojection” for ouput in SOM projection
Added “Nearest Neighbour” for sampling type
Fixed problem with missing metadata for HDF-EOS outputs
– Added Albers Projection
– New "Subsetter" functionality for Parameters (aka Fields).
This results in:
• Output to HDF-EOS SWATH from input HDF-EOS SWATH
• No alteration of input data because of projection operation.24
Page
25. HDF-EOS/Toolkit
Future Plans:
•
•
•
•
Auto Configuration for TOOLKIT
Full Support for Cygwin platform
Support for HDF5-1.8
Performance improvement, if possible, for reading HDF
format ephemeris/attitude files in Toolkit
• Performance improvement in HDF-EOS5 by redesigning some
functions
• More functionalities in HDF-EOS Plug-in for HDFView:
Format conversion: HDF-EOS2 -> HDF-EOS5
Improved Help Window
Ability to modify input HDF-EOS2 granules.
Ability to cut/paste objects, modify/delete attributes.
Ability to create new grids/swaths
Page 25
28. Major Development/Fixes
Last Release
HDF-EOS5:
Added a routine to return list of aliases for a field
Used HDF5’s H5T_C_S1 datatype for string type
Added Fortran wrapper for SWwritedatameta() routine
Replaced symbol “H5E_NONE_MAJOR” with other symbols
for better description of errors
Ported to Solaris 9 and Solaris 10
Fixed POINT object for:
linkage between levels
-
updating levels
writing Forward and Backward Linkage data
char field inquiry
Page 28
29. Major Development/Fixes (cont.)
Last Release
Fixed subsetting problem in HDF-EOS5 Grid
(correction to conversion of DMS degree to Decimal
degree).
Set LD_LIBRARY_PATH for SZIP/HDF5 shared libraries
in installation and environment setting scripts
TOOLKIT/HDF-EOS2:
Ported to Solaris 9 and Solaris 10
Gdapi.c changes for adding more significant digits after
the decimal point to projection parameters if needed.
chkeph utility fixes for reading binary ephemeris files
(files with different endiannes than the machine)
SGI 7.4.2 compiler support
orbsim fixes for Linux to create files similar to those in
UNIX platforms (Random number generation fixes)
Page 29