What is the HDF file format?
.hdf files store scientific datasets in the HDF4 (Hierarchical Data Format version 4) binary container, developed in 1988 by the National Center for Supercomputing Applications (NCSA) at the University of Illinois. HDF4 was designed to let scientists and engineers store heterogeneous numerical data - multidimensional arrays, raster images, tables, and metadata - in a single portable file.
The format is identified by the four-byte magic sequence 0E 03 13 01. Internally, HDF4 uses a flat tag-and-reference architecture: every data object is described by a 12-byte descriptor (tag, reference number, offset, length) in a Data Descriptor Block. Compression is optional per dataset and may use RLE, SKPHUFF (modified Huffman), or DEFLATE (gzip). A hard 2 GB limit per data object results from 32-bit signed offset fields.
NASA distributes remote-sensing products from the Terra and Aqua satellites in HDF4-based HDF-EOS format using the .hdf extension. HDF4 has been superseded by HDF5 - a complete redesign released in 1998 - which is now recommended for new scientific work. Both formats may carry the .hdf extension; magic bytes reliably distinguish them (HDF4: 0E 03 13 01; HDF5: 89 48 44 46).
Security & safety
RISK: LOWHDF4 is a passive binary data container with no executable code. Safe to open. Very large datasets can exhaust RAM - use pyhdf or MATLAB to read subsets. Malformed HDF4 files could trigger bugs in older versions of libhdf4; keep tools updated. Some HDF4 library versions (before 4.2.15) had CVE-rated security vulnerabilities - update to 4.2.16+.
Format details
in a nutshell- HDF5 file (occasionally) - Some tools save HDF5 data with the .hdf extension; use magic bytes to distinguish HDF4 (0x0E031301) from HDF5 (0x89HDF...).
Programs that open HDF files
Technical details
deep spec| Magic bytes | 0E 03 13 01 at offset 0 |
| Format type | Binary scientific data container (flat tag/reference architecture, HDF4) |
| Byte order | Big-endian native; each data object stores its own byte order descriptor |
| Compression | Optional per dataset: RLE, SKPHUFF (modified Huffman), or DEFLATE (gzip) |
| Data types | Multidimensional arrays (Scientific Datasets), raster images (1-32 bpp), tables (Vdata), groups (Vgroups), annotations |
| Max object size | ~2 GB per data object (32-bit signed offset field limitation) |
| Max dimensions | Up to 32 dimensions per Scientific Dataset |
| Max object references | 65,535 per tag type (16-bit reference numbers) |
| Architecture | Flat tag/reference - objects described by 12-byte descriptors (tag, ref, offset, length) in Data Descriptor Blocks |
| Primary scientific use | NASA EOS remote sensing (Terra, Aqua satellites); climate and atmospheric research |
| Superseded by | HDF5 (.h5 / .hdf5), a complete redesign released 1998; not byte-compatible with HDF4 |
| HDF4 maintenance status | Security fixes only since ~2015; no new features added |
| Conversion tool | h4h5tools (The HDF Group) converts HDF4 files to HDF5 |
| Released | 1988 (HDF 1.0 by NCSA, University of Illinois) |
| Latest version | HDF4 4.2.16 (2022); HDF4 maintenance-only since HDF5 became primary |
| Open standard | Yes · royalty-free |
| Specification | support.hdfgroup.org |
HDF conversions
Community Q&A
asked by usersNo questions yet - be the first to ask about HDF files.