In this talk, we will give an update on the HDF5 OPeNDAP project. We will update the new features inside OPeNDAP HDF5 data handler. We will also introduce a new HDF5-Friendly OPeNDAP client library and demonstrate how it can help users to view and analyze remote HDF-EOS5 data served by OPeNDAP HDF5 handler. A demo will be presented with a customized OPeNDAP visualization client (GrADS) that uses the library.

1. HDF5 OPeNDAP Project
Update and Demo
MuQun Yang and Hyo-Kyung Lee
(The HDF Group)
James Gallagher
(OPeNDAP, Inc.)
10/17/2008
HDF and HDF-EOS Workshop XII
1

2. OPeNDAP Review
• A software framework that allows
simple access to remote data
• Data Access Protocol
• Client-server model
• Widely used by Earth Science
Community
10/17/2008
HDF and HDF-EOS Workshop XII
2

3. OPeNDAP Concept
Typical Remote Access
View Data
(User)
FTP/HTTP
Remote Data
(HDF4)
Visualization
Tools
tools
(e.g. grads)
(gradsdap)
Handler
(hdf4_handler)
Client Library
(libnc-dap)
Server
(apache)
DAP Protocol (via http)
10/17/2008
HDF and HDF-EOS Workshop XII
3

4. OPeNDAP HDF5 Handler
View Data
(User)
Remote HDF5
HDF5 Handler
Server
(apache)
DAP Protocol (via http)
10/17/2008
HDF and HDF-EOS Workshop XII
4

5. Challenges
Mapping HDF5 to DAP
• Compound Datatype
• Groups
• Object/Regional References
10/17/2008
HDF and HDF-EOS Workshop XII
5

6. OPeNDAP HDF5 Handler with HDF-EOS5
View Data
(User)
Remote
HDF-EOS5
HDF5
Visualization
tools
(grads)
HDF5 Handler
Client Library
(libnc-dap)
Server
(apache)
DAP Protocol (via http)
10/17/2008
HDF and HDF-EOS Workshop XII
6

7. Challenges: HDF- EOS5
• Grid with No Geolocation data
 Clients expect Grid with Geolocation data
• Some Attributes stored as Extremely Long
String(s) (e.g. StructMetada.0)
 Clients expect structured attributes
10/17/2008
HDF and HDF-EOS Workshop XII
7

8. Tweaks for HDF-EOS5
Added two default HDF5 handler
configuration options:
• --enable-eos-grid
• --enable-eos-meta
10/17/2008
HDF and HDF-EOS Workshop XII
8

9. Challenges: OPeNDAP Clients
• Need special attributes on dataset.
• Need shared geolocation variables.
10/17/2008
HDF and HDF-EOS Workshop XII
9

10. Tweaks for OPeNDAP Clients
Added two optional handler
configuration options:
• --enable-short-name
• --enable-CF
10/17/2008
HDF and HDF-EOS Workshop XII
10

11. OPeNDAP HDF5 Handler with HDF-EOS5
View Data
(User)
Remote
HDF-EOS5
HDF5
Visualization
tools
(grads)
HDF5 Handler
w/ CF options
Client Library
(libnc-dap)
Server
(apache)
DAP Protocol (via http)
10/17/2008
HDF and HDF-EOS Workshop XII
11

12. Day After Server Tweaks
• Finally, Happy Clients!
10/17/2008
HDF and HDF-EOS Workshop XII
12

13. Problems of Tweaking HDF5 Handler
Remember that we added two optional
HDF5 handler configuration options:
1. --enable-short-name
2. --enable-CF
Causes ambiguity among
variable names
(e.g. /GroupA/ozone vs.
/GroupB/ozone)
Drops some key attributes
(e.g. StructMetdata, HDF_ROOT_GROUP)
10/17/2008
HDF and HDF-EOS Workshop XII
13

14. HDF5-Friendly OPeNDAP Client Library
View???
Groups
HDF5 Groups
View Data
(User)
Remote
HDF5
Visualization
tools
(grads)
HDF5 Handler
w/ CF options
HDF5-Friendly
Client Library
(liboc-dap)
(libnc-dap)
Server
(apache)
DAP Protocol (via http)
10/17/2008
HDF and HDF-EOS Workshop XII
14

15. Traditional Groups
Example:OPeNDAP clientin HDF5
library: It’s an attribute that I
don’t understand. I’ll ignore it.
HDF5-Friendly
OPeNDAP client
library: I was waiting
for this key attribute to
re-construct HDF5
10/17/2008
HDF and HDF-EOS Workshop XII
15

16. Example: Reference in HDF5
• Important for NPOESS
• Object / Regional Reference
• Map to DAP URL
10/17/2008
HDF and HDF-EOS Workshop XII
16

17. Example: dap2h5
View Group/Ref.
Group/Ref.
View Data
(User)
Remote
HDF5
• HDF5-F. Client
A test application for the client library
Library construct HDF5 from DAP output
HDF5 Handler
• It canApp.
(dap2h5)
HDF5-Friendly
Client Library
(liboc-dap)
Server
(apache)
DAP Protocol (via http)
10/17/2008
HDF and HDF-EOS Workshop XII
17

18. Our Client Library(prototype)
• One more reason:
Help Clients to view Swath

19. Visualizing HDF-EOS5 Swath Problem
Visualizing HDF-EOS5 Grids
View Swath
???
Swath
View Data
(User)
Remote
HDF-EOS5
HDF5
Visualization
Tools
tools
(gradsoc)
(grads)
HDF5 Handler
w/ CF option
HDF5-Friendly
Client Library
(liboc-dap)
(libnc-dap)
Server
(apache)
DAP Protocol (via http)
10/17/2008
HDF and HDF-EOS Workshop XII
19

20. Demo: MLS swath via GrADS
• GrADS coupled with our client
library
• The client library provides grid
mapping from swath data
• GrADS displays swath through
HDF-EOS5 specific client library
API calls

21. Summary of Client Library (Prototype)
• Finished coding
• Tested with the GrADS visualization
client
• Documented
• Working on a demo DAP to HDF5
tool by using the client library
prototype
• Will test with NCL if time allows
10/17/2008
HDF and HDF-EOS Workshop XII
21

22. Caution
• Our Client Library is ONLY A PROTOTYPE!
 It does NOT support all DAP data types
 It does NOT support all AURA files
 It does NOT support all Visualization clients
10/17/2008
HDF and HDF-EOS Workshop XII
22

23. Summary
• HDF5 Access via OPeNDAP
 Is Easy
 Is used by GES DISC to serve Aura
files
• HDF5-Friendly OPeNDAP Client Library
 Serves HDF5 better(EOS swath)
 Benefits visualization clients
10/17/2008
HDF and HDF-EOS Workshop XII
23

24. Future Work
• HDF5 to DAP2 Mapping Document
• Release HDF5-friendly OPeNDAP
Client Library (Prototype)
• URL: http://hdfgroup.org/projects/opendap
10/17/2008
HDF and HDF-EOS Workshop XII
24

25. Credits
Mike Folk (The HDF Group)
Robert McGrath (NCSA)
Peter Leonard, Daniel Kahn, Marghi Hopkins (ADNET)
Christopher Lynnes, James Johnson, Denis Nadeau, Fan Fang
(NASA)
Jennifer Adams (GrADS)
Dave Brown (UCAR)
10/17/2008
HDF and HDF-EOS Workshop XII
25

26. Acknowledgement
This work was supported basing upon the
Cooperative Agreement with the National
Aeronautics and Space Administration (NASA)
under NASA grant NNX06AC83A,
NNX08A077A and NNX06AG75A. Any opinions,
findings, and conclusions or recommendations
expressed in this material are those of the
author(s) and do not necessarily reflect the
views of NASA.
10/17/2008
HDF and HDF-EOS Workshop XII
26