CFIM Data Repository for HAND and Hydraulic Property Table
Introduction
The continental flood inundation mapping (CFIM) framework is a high-performance computing (HPC)-based computational framework for the Height Above Nearest Drainage (HAND)-based inundation mapping methodology. Using the 10m Digital Elevation Model (DEM) data produced by U.S. Geological Survey (USGS) 3DEP (the 3-D Elevation Program) and the NHDPlus hydrography dataset produced by USGS and the U.S. Environmental Protection Agency (EPA), a hydrological terrain raster called HAND is computed for HUC6 units in the conterminous U.S. (CONUS). The value of each raster cell in HAND is an approximation of the relative elevation between the cell and its nearest water stream. Derived from HAND, a hydraulic property table is established to calculate river geometry properties for each of the 2.7 million river reaches covered by NHDPlus (5.5 million kilometers in total length). This table is a lookup table for water depth given an input stream flow value. Such lookup is available between water depth 0m and 25m at 1-foot interval. The flood inundation map can then be computed by using HAND and this lookup table based on the near real-time water forecast from the National Water Model (NWM) at the National Oceanic and Atmospheric Administration (NOAA).
Data Releases
Stable versions
Texas HAND 3m Version 1.0, 20231101
Full data download URL: https://web.corral.tacc.utexas.edu/nfiedata/pin2flood/texas/
Citation: Liu, Yan, David Maidment, Andy Carter, Paola Passalacqua, and Tim Whiteaker. "Height Above Nearest Drainage (HAND) at Three-Meter Resolution for the State of Texas." United States: N. p., 2023. Web. doi:10.13139/ORNLNCCS/2204010.
Download links
The standard datasets contain files needed to operate Pin2Flood. The full datasets include everything for research and review purposes.
In January 2023, Oak Ridge National Laboratory (ORNL) computed the 3-meter hydrologic terrain and associated synthetic rating curves for the State of Texas, covering 287,535 river streams (approximately 1.5km/stream) in 209 river subbasins, funded by a Strategic Partnership Project (SPP) with the University of Texas at Austin. The computed data products (https://www.osti.gov/dataexplorer/biblio/dataset/2204010) are core datasets for supporting Pin2Flood, a field app for Texas first responders to drop a pin on a mobile device and display nearby flood inundation extent. The Texas Division of Emergency Management (TDEM) officially released the Pin2Flood app in June 2023 (Video link).
The computed Texas hydrologic terrain is the first 3-meter resolution Height Above Nearest Drainage (HAND)-based hydrologic terrain computed at Hydrologic Region scale in the U.S. Its computational intensity is comparable to computing the 10-meter resolution terrain for the continental U.S. ORNL accelerated the underlying scientific workflow and reduced the computing time from a week on 20 computing nodes to 3 days using a single high-memory 128-core machine. The availability of this high-resolution hydrologic terrain significantly improved the accuracy of flood extent mapping in Pin2Flood, verified by seven flood response exercises conducted by hundreds of Texas first responders. The Pin2Flood app is the first production application of the Continental Flood Inundation Mapping framework for flood emergency management. The success of the Pin2Flood has been demonstrated by the official release and that the Pin2Flood operational platform at TDEM was awarded the 2022 Award for Excellence in Public Safety GIS in the state category by the National Alliance for Public Safety GIS (NAPSG) Foundation.
Yan Liu in the Computational Urban Sciences Group, the PI of the SPP project, collaborated with the University of Texas at Austin and Esri in the Pin2Flood project. Computing resources were provided by the Oak Ridge Research Cloud with tremendous help from Chris Layton and Daniel Dewey. The computed Pin2Flood data has been published as an open source dataset for research and review purposes at both TACC and ORNL's CFIM website.
This 3-meter HAND is derived from the Fathom 3-meter DEM and NHDPlus V21 using an accelerated version of NOAA's Flood Inundation Mapping version 3 (FIM3).
Synthetic rating curve table attributes
Attributes in hydroTable_rp_bf_lmtdischarge_cda.csv:
- FATSGTID: river segment ID (HydroID)
- feature_id: matched COMID in NHDPlus V21
- NextDownID: downstream ID
- order_: stream order
- stage: stage heigh (m). range: 0-25m
- discharge_cms: stream flow rate (cms)
- HydraulicRadius (m): hydraulic radius
- WetArea (m2): wet area
- SLOPE: slope of the river segment
- ManningN: Manning's N of the river segment
- HUC: HUC(8) ID
- LakeID: lake ID, if part of the lake water body
- RP_Q_years: flood return period. range: 1-42. level: COMID
- CPE_Q: cumulative probability estimate. level: COMID
- EPE_Q: exceedance probability estimate. level: COMID
- BF_Q_cms_67: bankfull discharge (cms). 67% percentile. level: COMID
- RP_H_years: flood return period. range: 1-42. level: FATSGTID
- CPE_H: cumulative probability estimate. level : FATSGTID
- EPE_H: exceedance probability estimate. level: FATSGTID
- BF_H_m_67: bankfull discharge (cms). 67% percentile. level : FATSGTID
- cda_sqkm: upstream cumulative drainage area (CDA), derived from the HydroID (FATSGTID)-based river network. If a river flows into a HUC, the first HydroID's CDA is initialized as: (catchment_area + TotDASqkm - COMID_catchment_area), where TotDASqkm is carried from NHDPlusV21 MR
- cda_LimDisch_cms: matched limiting discharge (cms)
- cda_sqmi: CDA (square miles)
- cda_LimDisch_cfs: matched limiting discharge (cfs)
- cda_PctLimDisch: ratio percentage: discharge/ max_streamflow * 100. If exceeding 100, ignore or treated as 100
CONUS HAND 10m Version 0.2.0, 20200301
https://cfim.ornl.gov/data/HAND/20200301/{huc6code}.zip
Visualization and download: Map
HAND HUC6 coverage shape file (with download URL, too): Shape file
Permanent (tape) download on OLCF via Globus: follow the DOI link.
HAND and the Hydraulic Property Table version 0.2.0 was computed on the CADES Condo cluster at Oak Ridge National Laboratory on March 01, 2020. This version is created to correspond to data updates in the USGS 1/3 arcsec DEM, the USGS National Hydrography Dataset, including its Water Boundary Dataset, and the NHDPlus medium resolution dataset.
This dataset comprises 331 HUC6 units for CONUS (excluding the five great lakes units), each is a downloadable zip file.
Development versions
CONUS HAND 10m Version 0.2.1, 20200601
https://cfim.ornl.gov/data/HAND/20200601/{huc6code}.zip
HAND HUC6 coverage shape file (with download URL, too): Shape file
Permanent (tape) download on OLCF via Globus: follow the DOI link.
HAND and the Hydraulic Property Table version 0.2.1 was computed on the CADES Condo cluster and a high-memory AMD machine at Oak Ridge National Laboratory. This version provides more complete HAND coverage, esp. around each HUC6 unit's boundary area. A new etching workflow is integrated to conduct a DEM burn-in process in order to resolve the elevation issue at intersections of river and road.
This dataset comprises 331 HUC6 units for CONUS (excluding the five great lakes units), each is a downloadable zip file.
Previous Releases
- Version 0.1 was computed at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urban-Champaign and distributed by the Texas Advanced Computing Center (TACC): https://web.corral.tacc.utexas.edu/nfiedata/.
Input Data Sources
All the input datasets are made freely available to general public by U.S. government agencies.
- DEM: USGS 3DEP National Elevation Dataset 1/3 arcsec (~10m). Retrieval date: 02/17/2020.
- Hydrography: NHDPlus MR (medium resolution) and HR beta (high resolution); USGS National Hydrography Dataset (NHD) HR (high resolution); USGS Water Boundary Dataset (WBD).
- NHDPlus MR (medium resolution, 1:100000), EPA. Data release date: 10/01/2018.
- NHDPlus HR (beta) (high resolution, 1:24000), USGS. Retrieval date: 02/18/2020.
- NHD HR, USGS. Data release date: 04/30/2020.
- WBD, USGS. Data release date: 12/16/2015. Note: the 2015 version is different from the most recent version linked here. The most recent version of NHDPlus MR uses a WBD version that is earlier than 10/01/2018.
- Water forecast: the NOAA National Water Model is an operational water forecast service at the NOAA National Water Center. NWM converts weather forecast (precipitation) and other water input to water forecast (e.g., stream flow) on all 2.7 million river reaches defined in NHDPlus MR. It provides short-range (hourly for the next 18 hours), medium-range (3-hourly for the next 10 days), and long-range (6-hourly for the next 30 days) forecast data online through the NCEP NOMADS HTTP service. Only the most recent 1.5 days of data is kept on the online server, though. Google Cloud hosts an archive of NWM forecast data since 2018.
Change Log
20200601 - release v0.2.1
Version 0.2.1 is released with the following improvements:
- Flowlines now cover the entire buffered HUC boundary, filling most of the void HAND areas along WBD border area.
- Input DEM is now etched by NHD HR flowlines using a burn-in workflow. This leads to improved HAND at intersection of river and road.
- Online visualization maps are generated for both v0.2.0 and v0.2.1 and published as the HAND TMS service, which can be accessed as an XYZ raster layer in browser.
20200408
It was found that an MPI runtime error occurred when computing 6 of the 331 HUC6 units. The six units were rerun to fix that issue. Data of these units are updated accordingly: 071000 150100 160600 170102 170200 170501.
20200301 - release v0.2.0
Compared to version 0.1, the following changes have been made in version 0.2:
- The most recent versions of DEM and NHDPlus are used to update HAND and the hydraulic property table.
- The computing workflow is adapted to ORNL CADES Condo computing environment. A major change is to use the distributed shared memory for all intermediate IO.
- The computation of the hydraulic property table has been integrated into HAND workflow.
Metadata
Assume ${HUCID} denotes the HUC6 code for each unit. This is the list of files in each HUC6 HAND directory:
${HUCID}-wbd.shp HUC unit boundary shape file, extracted from USGS WBD
${HUCID}-wbdbuf.shp HUC unit boundary shape file, extracted from USGS WBD (buffered)
${HUCID}-flows.shp Flowline shape file, extracted from NHDPlus V21
${HUCID}-inlets0.shp Inlets point shape file in the HUC unit. Native projection
${HUCID}-inlets.shp Inlets point shape file in the HUC unit. EPSG:4269
${HUCID}-weights.tif Weight grid of the rasterized inlet points
${HUCID}.tif Clipped HUC unit DEM from USGS 3DEP 10m elevation dataset (buffered)
${HUCID}bi.tif Clipped HUC unit DEM etched by NHD HR (buffered)
${HUCID}fel.tif Pit-removed DEM; output of TauDEM pitremove
${HUCID}p.tif D8 flow direction raster; output of TauDEM d8flowdir
${HUCID}sd8.tif D8 slope raster; output of TauDEM d8flowdir
${HUCID}ang.tif Dinfinity flow direction raster; output of TauDEM dinfflowdir
${HUCID}slp.tif Dinfinity slope raster; output of TauDEM dinfflowdir
${HUCID}ssa.tif Contributing area raster; output of TauDEM aread8
${HUCID}src.tif Stream grid; output of TauDEM threshold (threshold=1)
${HUCID}dd.tif Buffered HAND raster; output of TauDEM dinfdistdown
${HUCID}hand.tif HAND raster, buffer removed, final result
${HUCID}_catch.sqlite Catchment polygons for all the river reaches in a HUC6 unit
${HUCID}_comid.txt Catchment ID list for a HUC6 unit (COMID, slope, flowline length, and areasqkm)
${HUCID}catchmask.tif Rasterized catchments with cell value to be the COMID of the corresponding river reach (buffered)
${HUCID}catchhuc.tif Rasterized catchments in HAND extent
hydrogeo-fulltable-${HUCID}.csv Hydraulic property table with the following fields:
CatchId,Stage,Number of Cells,SurfaceArea (m2),BedArea (m2),Volume (m3),SLOPE,LENGTHKM,AREASQKM,Roughness,TopWidth (m),WettedPerimeter (m),WetArea (m2),HydraulicRadius (m),Discharge (m3s-1)
Details of computing the hydraulic properties can be found in Zheng et al. 2018. Briefly:
Input:
h: water stage assumed for each river reach, the water level between surface and bottom of the channel.
L: river reach length.
I = {i}, where i is any inundated raster cell given water level h. I is computed on HAND raster by identifying cells whose HAND value is less than or equal to h. Associated with each i are some attributes:
area(i): area of cell i. this is a uniform value for a raster.
slope(i): slope of cell i. this is from the slope raster input.
d(i) = h - HAND(i): water depth of cell i.
Hydraulic properties as output:
SurfaceArea: S = Σ i ∈ I area(i). Surface area as the sum of inundated cell areas.
BedArea: B = Σ i ∈ I [ area(i) × √ 1 + slope2(i) ]
Volume: V = Σ i ∈ I [ area(i) × d(i) ] . Volume as the sum of each cell's inundated volume.
TopWidth: W = S ÷ L . The reach-average channel width.
WettedPerimeter: P = B ÷ L. The reach-average wetted cross-sectional perimeter.
WettedArea: A = V ÷ L. The reach-average wetted cross-sectional area.
HydraulicRadius: R = A ÷ P. The reach-average hydraulic radius. This is approximated as if A is the cross-sectional area of a half-sized cylinder and P is the half-perimeter of the cross section circle.
How to Cite This Dataset
Please cite the following publications when you use HAND and the hydraulic property table:
- Liu, Yan Y., Tarboton, David G., and Maidment, David R. Height Above Nearest Drainage (HAND) and Hydraulic Property Table for CONUS - Version 0.2.1. (20200601). Oak Ridge Leadership Computing Facility. (2020). DOI: 10.13139/ORNLNCCS/1630903
- Liu, Yan Y., Tarboton, David G., and Maidment, David R. Height Above Nearest Drainage (HAND) and Hydraulic Property Table for CONUS - Version 0.2.0. (20200301). Oak Ridge Leadership Computing Facility. (2020). DOI: 10.13139/ORNLNCCS/1608331
- Liu, Yan Y., David R. Maidment, David G. Tarboton, Xing Zheng, and Shaowen Wang. "A CyberGIS integration and computation framework for high-resolution continental-scale flood inundation mapping." JAWRA Journal of the American Water Resources Association 54, no. 4 (2018): 770-784. DOI: 10.1111/1752-1688.12660
- Zheng, Xing, David G. Tarboton, David R. Maidment, Yan Y. Liu, and Paola Passalacqua. "River channel geometry and rating curve estimation using height above the nearest drainage." JAWRA Journal of the American Water Resources Association 54, no. 4 (2018): 785-806. DOI: 10.1111/1752-1688.12661
Visualization
HAND is published as OSGeo Tile Map Service (TMS, https://wiki.osgeo.org/wiki/Tile_Map_Service_Specification) and can be rendered at a client as an XYZ layer. Note that TMS and XYZ standards differ in only one thing: Y tile coordinate. The conversion is simple: y' = 2^z - y - 1 (or in C, y = 1 << z - y - 1), where z is the zoom level (2^z is the total number of tiles at this zoom level).
If you want the CONUS view, configure the map layer as below:
{
"type": "XYZ",
"extent": [-14392000, 2436200, -7279500, 6594375],
"url": "https://cfim.ornl.gov/data/HANDTMS/VERSION/{z}/{x}/{-y}.png",
"projection": "EPSG:3857",
"minZoom": 5,
"maxZoom": 12
}
Remember to replace VERSION with the actual version number, e.g., v0.2.1.
Acknowledgements
The Continental Flood Inundation Mapping project is supported in part by the Laboratory Directed Research and Development Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the US Department of Energy under contract DE-AC05-00OR22725.
This research used resources of the Compute and Data Environment for Science (CADES) at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725. We thank Chris Layton, Greg Shutt, Suzanne Parete-Koon, and Daniel Dewey for their support on provisioning CADES HPC (Condo and a AMD high-memory node) and Cloud resources.
The data registration and publishing used the Constellation Data Portal, a feature in the Scalable Data Infrastructure for Science (SDIS) at the Oak Ridge Leadership Computing Facility (OLCF) in Oak Ridge National Laboratory. We thank Ross Miller, Hyogi Sim, Sudharshan Vazhkudai, and Mitchell Griffith for their help in providing data management resources.
LICENSE FOR USE -- MAPS AND DATA DISCLAIMER
This resource is shared under the Creative Commons Attribution CC BY,
http://creativecommons.org/licenses/by/4.0/
MAPS AND DATA DISCLAIMER
The Oak Ridge National Laboratory (ORNL) shall not be held liable for improper or incorrect use of the data described or information contained on this map or associated series of maps. The data and related map graphics are not legal, land survey or engineering documents and are not intended to be used as such.
ORNL gives no warranty, express or implied, as to the accuracy, reliability, utility or completeness of this information. The user of these maps and data assumes all responsibility and risk for the use of the maps and data. ORNL disclaims all warranties, representations or endorsements either express or implied, with regard to the information contained in this map product, including, but not limited to, all implied warranties of merchantability, fitness for a particular purpose or non-infringement.
This preliminary map product is for research and review purposes only. It is not intended to be used for emergency management operational or life safety decisions at the local or regional governmental level or by the general public. Users requiring information regarding hazardous conditions or meteorological conditions for specific geographic areas should consult directly with their city or county emergency management office.
###
Contact
Community feedback is important for us to continuously improve the quality of the HAND dataset and the underlying CFIM methodologies. Please let us know if you have any suggestions/comments or need to report an issue.