This tutorial is designed for anyone who needs to work with data stored in HDF and HDF5 files.
The first part of the tutorial will focus on the HDF5 utilities to display the contents of HDF5 files, to extract and to import data from and to HDF5 files, to compare two HDF5 files, and more. Participants will be guided through the hand-on examples and will learn about different tools options. New changes and advanced features will be covered in a separate session (Updates on HDF tools) on Wednesday.
The second part of tutorial includes a hands-on session to learn the HDF (4 & 5) Java browsing tool, HDFView. The tool and special plug-ins will be used to work with the existing HDF, HDF-EOS, and netCDF-4 files, and to create a new HDF5 file. The tutorial will cover basic features of HDFView.
1. The HDF Group
HDF Tools Tutorial
Peter Cao, The HDF Group
Jonathan Kim, The HDF Group
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
1
www.hdfgroup.org
2. Tools Tutorial
⢠HDFView
⢠HDF5 Command-line tools
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
2
www.hdfgroup.org
3. HDFView
⢠View file content
⢠Edit file content
⢠Show NPOESS/JPSS related features
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
3
www.hdfgroup.org
4. HDFView
⢠Download HDF-Java 2.7 beta from
http://www.hdfgroup.org/ftp/HDF5/hdf-java-2.7
⢠Download example files from
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/WorkshopXIV/HDF5-Files/
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
4
www.hdfgroup.org
5. HDFView
⢠View file content
â˘
â˘
â˘
â˘
â˘
Open OMI-Aura_L3.he5
Open groups (try expand all)
Stretch the metadata view for showing metadata
Double click on âColumnAmountO3â to show data in table
Use âOpen Asâ to show âColumnAmountO3â in image
⢠View a subset
⢠Open weather.h5
⢠Select from preview image or selection fields
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
5
www.hdfgroup.org
6. HDFView
⢠Copy/paste a dataset and a group
⢠Create a new file: tmp.h5
⢠Copy/paste âColumnAmountO3â and show the image from
the new file
⢠Copy the group âGRIDSâ to show the whole group is copied
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
6
www.hdfgroup.org
7. HDFView
⢠Create a dataset and add an attribute
â˘
â˘
â˘
â˘
â˘
Create /d1 (32-bit float, 720x1440)
Copy âColumnAmountO3â to /d1
Add attribute âIMAGE_MINMAXRANGE = 123.0,447.0â
Export /d1 to text file
Create /d2 and import d1.txt
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
7
www.hdfgroup.org
8. HDFView
⢠NPOESS/JPSS related features
⢠Object reference
⢠Region references
⢠Quality flags
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
8
www.hdfgroup.org
9. HDFView
⢠Object reference
1)
2)
3)
4)
5)
6)
Open file: SVI-NPP.h5
Open âVIIRS-I1-SDR_Aggrâ
Click on object ID and shows object name
Right mouse to click to open data as image
Open data in table
Select two cells and open them at once
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
9
www.hdfgroup.org
10. HDFView
⢠Region references
1)
2)
3)
4)
Open âVIIRS-I1-SDR_Granâ
Right mouse to click to open data as image
Open data in table
Select two cells and open at once
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
10
www.hdfgroup.org
11. HDFView
⢠Quality flags
1) Quality flags
â˘
â˘
â˘
â˘
Land/Water Background
SST Skin Quality
SST Bulk Quality
Aerosol Correction
-- 1-bit
-- 2-bits
-- 2-bits
-- 3-bits
2) Open QF_VIIRSI1SDR_Array
3) Use "Open As" on QF_VIIRSI1SDR_Array
4) Select first bit
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
11
www.hdfgroup.org
12. HDFView
⢠Advanced features
1) Import/export images
2) Use HDF-EOS plugin
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
12
www.hdfgroup.org
16. h5dump
⢠Structure
⢠h5dump -H tools_diff1.h5
⢠h5dump -H ami.h5
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
16
www.hdfgroup.org
17. h5dump
⢠Dataset
⢠Content
⢠h5dump -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5
⢠Information
⢠h5dump âp âH -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5
⢠Subset
⢠H5dump -d /EnSight_model/geometry/parts/part_2/quad4 -s 10,0 -c 20,2 ami.h5
⢠<-s and âc must be used after âd>
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
17
www.hdfgroup.org
18. h5dump
⢠Binary
â˘
h5dump -d /EnSight_model/geometry/parts/part_2/quad4 -b LE âo out.bin ami.h5
⢠XML
⢠h5dump âx ami.h5
⢠Region
â˘
â˘
h5dump -d /Data_Products/VIIRS-I1-SDR/VIIRS-I1-SDR_Gran_0 -s 3 -c 1 SVI-NPP.h5
h5dump âR -d /Data_Products/VIIRS-I1-SDR/VIIRS-I1-SDR_Gran_0 -s 3 -c 1 SVI-NPP.h5
⢠Packed fields <only 1.8.5+>
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
18
www.hdfgroup.org
19. h5repack
⢠Remove inaccessible objects / junk spaces
⢠Change storage layout
⢠Apply compression filter
Copies a file to a new file
with different storage layouts
and compression filters
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
19
www.hdfgroup.org
20. h5repack
⢠Remove inaccessible objects
⢠h5dump tools_junk.h5
⢠h5repack tools_junk.h5 tmp.h5
⢠Change layout
⢠h5dump -H tools_bad_layout.h5
⢠h5repack tools_bad_layout.h5 tmp.h5
⢠h5repack -l CHUNK=16x16 tools_bad_layout.h5 tmp.h5
⢠Change compression
⢠h5repack -f GZIP=6 tmp.h5 tmp2.h5
⢠More: use latest version, link storage, shared message, and etc
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
20
www.hdfgroup.org
21. h5diff
Show differences between
two files or two objects
⢠Dataset
⢠Whole file
September 28-30, 2010
HDF and HDF-EOS Workshop XIV
21
www.hdfgroup.org