{"success":true,"data":{"author":"Songkun Yan, Yang Hong, Meihong Ma et al. (University of Oklahoma)","bucketAccessType":"public","bucketName":"skyan1002/flash-flood-benchmark-data","bucketPrefix":"","category":"Environmental Science","cloudProvider":"huggingface","createdAt":"2026-11-16T15:39:55.899Z","description":"TORRENT is a benchmark of reproducible flash-flood testbeds for the contiguous United States, built for hydrological-response analysis and model intercomparison (companion paper in Earth System Science Data). Level 1 is the national NCEI Storm Events flash-flood catalogue for 1996-2025: 42,466 episodes, 110,251 event records and 342,238 location records. Level 2 is an observation-availability catalogue for 2021-2025 (5,424 episodes, 4,599 with USGS discharge availability, each with an evidence label). Level 3 has 806 evaluation-ready episode-gauge-watershed testbeds (160 episodes, 208 gauges) with watershed boundaries from NLDI, 3DEP terrain and USGS NWIS discharge, from which a curated primary benchmark of 21 testbeds (14 episodes, 21 gauges, return period at least 2 years) is flagged; regulated and post-fire basins are labelled rather than removed. NSSL MRMS and FLASH-CREST forcing is fetched on demand by a script. The repository mirrors the v1.0 Zenodo archive byte for byte (catalog CSVs and Parquet, bundles/ zips, about 1.1 GB) and adds manifest.jsonl with resolve URLs for programmatic access. Licensed CC BY 4.0 (Zenodo); cite the TORRENT paper and DOI 10.5281/zenodo.22118051.","displayTitle":"TORRENT: CONUS Flash-Flood Benchmark of Episode-Gauge-Watershed Testbeds","doi":"10.5281/zenodo.22118051","huggingFaceUrl":"https://huggingface.co/datasets/skyan1002/flash-flood-benchmark-data","id":"6ea34706-2b48-57db-8386-374bbf2d7a7b","license":"CC-BY-4.0","licenseEvidenceUrl":"https://zenodo.org/records/22118051","objectCountEstimate":4800,"probeStatus":"ok","referenceUrl":"https://doi.org/10.5281/zenodo.22118051","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"skyan1002/flash-flood-benchmark-data\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"skyan1002/flash-flood-benchmark-data\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=skyan1002, license:cc-by-4.0, size_categories:100K<n<1M, format:parquet, downloads=460"],"format":"mixed","generatedAt":"2026-11-16T15:40:03.821Z","layout":{"example":"catalog/l1_episodes.parquet","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"504152311504159c9d771592b9194c15c497051500120000cece3b581300000046465f313939365f30315f414c5f657030303156170000323a1700044354522e0004444356170000455217000447416e17003e73000d2e00333a8a000d170034561700003556170000365617000037561700003856170000395217000431305617003e590109b8003156e600003156e600003156e600003156e600003156e600003156e60000313ee600044849529e010d173eb800044b59522e000d173e2e00004d56fa01044d445245000d17424500092e3e2b010d173e2b010d173e2b010d173e2b010d173e2b010d173e2b010d173e110209174211020d173e11020d1756e6004211020d2e56e600421102044d53565901091742730009173e5901004e56dd030d173e45000d175a45003eb800044e4a52a1000d17425c00092e425c0009175a5c003efa010d2e3efa010d173efa010d173efa010d173efa01091742fa010d173efa010d1756e60042fa010d2e42e6005681030d17425c00092e42420109175a5c00424201092e42420109174242010917424201091742420105174642010917424201091756e600464201092e56e600423c030d2e56e600424d050d2e56e600003156e60000323e33060945003256e60000323ee600044f48523c030d175a2e003efa010d2e3e14010d173efa010d173e14010d173efa010d173e14010d","encoding":"hex","key":"catalog/l1_episodes.parquet","truncatedAt":512}],"sampleObjects":["AGENTS.md","CITATION.cff","DOWNLOAD.md","agent/tools.json","bundles/testbeds_primary21.zip"],"sidecars":[{"bytes":"Attribution 4.0 International\n\n=======================================================================\n\nCreative Commons Corporation (\"Creative Commons\") is not a law firm and\ndoes not provide legal services or legal advice. Distribution of\nCreative Commons public licenses does not create a lawyer-client or\nother relationship. Creative Commons makes its licenses and related\ninformation available on an \"as-is\" basis. Creative Commons gives no\nwarranties regarding its licenses, any material licensed under their\nterms and conditions, or any related information. Creative Commons\ndisclaims all liability for damages resulting from their use to the\nfullest extent possible.\n\nUsing Creative Commons Public Licenses\n\nCreative Commons public licenses provide a standard set of terms and\nconditions that creators and other rights holders may use to share\noriginal works of authorship and other material subject to copyright\nand certain other rights specified in the public license below. The\nfollowing considerations are for informational purposes only, are not\nexhaustive, and do not form part of our licenses.\n\n     Considerations for licensors: Our public licenses are\n     intended for use by those authorized to give the public\n     permission to use material in ways otherwise restricted by\n     copyright and certain other rights. Our licenses are\n     irrevocable. Licensors should read and understand the terms\n     and conditions of the license they choose before applying it.\n     Licensors should also secure all rights necessary before\n     applying our licenses so that the public can reuse the\n     material as expected. Licensors should clearly mark any\n     material not subject to the license. This includes other CC-\n     licensed material, or material used under an exception or\n     limitation to copyright. More considerations for licensors:\n    wiki.creativecommons.org/Considerations_for_licensors\n\n     Considerations for the public: By using one of our public\n     licenses, a licensor grants the public permission to use the\n     licensed material under specified terms and conditions. If\n     the licensor's permission is not necessary for any reason--for\n     example, because of any applicable exception or limitation to\n     copyright--then that use is not regulated by the license. Our\n     licenses grant only permissions under copyright and certain\n     other rights that a licensor has authority to grant. Use of\n     the licensed material may still be restricted for other\n     reasons, including because others have copyright or other\n     rights in the material. A licensor may make special requests,\n     such as asking that all changes be marked or described.\n     Although not required by our licenses, you are encouraged to\n     respect those requests where reasonable. More considerations\n     for the public:\n    wiki.creativecommons.org/Considerations_for_licensees\n\n=======================================================================\n\nCreative Commons Attribution 4.0 International Public License\n\nBy exercising the Licensed Rights (defined below), You accept and agree\nto be bound by the terms and conditions of this Creative Commons\nAttribution 4.0 International Public License (\"Public License\"). To the\nextent this Public License may be interpreted as a contract, You are\ngranted the Licensed Rights in consideration of Your acceptance of\nthese terms and conditions, and the Licensor grants You such rights in\nconsideration of benefits the Licensor receives from making the\nLicensed Material available under these terms and conditions.\n\n\nSection 1 -- Definitions.\n\n  a. Adapted Material means material subject to Copyright and Similar\n     Rights that is derived from or based upon the Licensed Material\n     and in which the Licensed Material is translated, altered,\n     arranged, transformed, or otherwise modified in a manner requiring\n     permission under the Copyright and Similar Rights held by the\n     Licensor. For purposes of this Public License, where the Licensed\n     Material is a musical work, performance, or sound recording,\n ","encoding":"utf8","key":"LICENSE","name":"LICENSE","size":18657,"truncated":true},{"bytes":"---\nlicense: cc-by-4.0\npretty_name: TORRENT — CONUS Flash-Flood Benchmark (L1–L3), agent-friendly\ntags: [hydrology, flash-flood, benchmark, usgs, noaa, mrms, streamflow, agent]\nsize_categories: [10K<n<100K]\nconfigs:\n- config_name: l1_episodes\n  data_files: [{split: train, path: catalog/l1_episodes.parquet}]\n- config_name: l1_locations\n  data_files: [{split: train, path: catalog/l1_locations.parquet}]\n- config_name: l2_episodes\n  data_files: [{split: train, path: catalog/l2_episodes.parquet}]\n- config_name: l3_testbeds\n  data_files: [{split: train, path: catalog/l3_testbeds.parquet}]\n- config_name: l3_hwms\n  data_files: [{split: train, path: catalog/l3_hwms.parquet}]\n- config_name: master_metadata\n  data_files: [{split: train, path: catalog/master_metadata.parquet}]\n- config_name: manifest\n  data_files: [{split: train, path: manifest.parquet}]\n---\n\n# TORRENT — CONUS Flash-Flood Benchmark (L1–L3), agent-friendly\n\nTraceable, Observation-constrained, Rapid-Response, Episode–gauge–watershed\nNetwork of Testbeds: reproducible flash-flood testbeds for hydrological-response\nanalysis and model intercomparison. This mirror carries the paper-matched\n**v1.0** release (companion paper: *TORRENT*, Earth System Science Data).\n\n- **Archive of record (v1.0):** <https://doi.org/10.5281/zenodo.22118051>\n  (concept DOI, always the latest version: <https://doi.org/10.5281/zenodo.22118050>)\n- **Interactive portal:** <https://huggingface.co/spaces/skyan1002/torrent-flash-flood-agent>\n- **Figures / portal assets:** <https://huggingface.co/datasets/skyan1002/flash-flood-benchmark-figures>\n\nThis repository mirrors that archive: the CSVs under `catalog/raw_csv/` are\nbyte-identical to the archive's tables, and each testbed package contains exactly\nthe archive's files for that testbed. On top of the mirror it adds the\nmanifest/agent layer described below. Cite the paper and the DOI, not this URL.\n\n**Three levels.** **L1** national NCEI flash-flood episode catalog (1996–2025,\n**42,466** episodes; 110,251 event records, 342,238 location records).\n**L2** observation-availability catalog (2021–2025, **5,424** episodes, 4,599 of\nthem with discharge availability; evidence labels strong 462 / partial 4,360 /\nweak 431 / none 171 — all retained, none dropped).\n**L3** evaluation-ready episode–gauge–watershed\ntestbeds (**806**; 160 episodes, 208 gauges; Tier A 337 / Tier B 469).\nAutomated funnel on L3: **175 strict → 115 deduplicated → 54 NCEI-confirmed →\n31 (return period ≥ 2 yr)**, then two documented post-selection curation reviews\n(−4 flood-control regulated, −6 post-fire Rio Ruidoso) give a conservative\n**primary benchmark of 21 testbeds (14 episodes, 21 gauges)**. The ten\ncurated-out testbeds stay in L3 with `regulation_excluded` / `postfire_excluded`\nset — nothing is deleted, only labelled.\n\n## Two headline features\n\n**1. Agent-friendly.** One flat index (`manifest.jsonl`) + precomputed\n`resolve_url`s + a machine-readable tool schema (`agent/tools.json`) and\n`AGENTS.md`. No SDK, no auth. Every funnel stage is a boolean flag you can filter\non (`primary_benchmark`, `is_strict_flash_flood`, `dedup_selected`/`selected`,\n`ncei_confirmed`, `rp_ge_2yr`, …). See **AGENTS.md**.\n\n**2. Annual auto-expansion.** `pipeline/expand_l3.py` + a GitHub Actions cron\n(`pipeline/.github/workflows/annual_l3_update.yml`) discover newly qualifying\ntestbeds from each new NCEI year and append them to L3 as light rows\n(`needs_curation_review=true`); `pipeline/rebuild_catalog.py` refreshes the\nParquet **and** the manifest and pushes both here. L3 grows automatically each\nyear; the human-reviewed primary benchmark is left unchanged. Forcing is fetched\non demand (`tools/download_forcing.py`) — the code interface, not gigabytes of\nrasters.\n\n## TL;DR for agents\n```bash\nBASE=https://huggingface.co/datasets/skyan1002/flash-flood-benchmark-data/resolve/main\ncurl -sL \"$BASE/manifest.jsonl\" -o manifest.jsonl\n\n# the default 21-testbed primary benchmark\n# (record_type==\"record\" is REQUIRED: artifact rows carry the same flags)\njq -c 'select(.record_type==","encoding":"utf8","key":"README.md","name":"README.md","size":8971,"truncated":true},{"bytes":"# TORRENT v1.0 construction pipeline\n\nThe complete, scripted construction chain of the TORRENT dataset:\n**source inputs → L1 → L2 → L3 qualifying pool → strict flash-flood → dedup → return\nperiod → released tables (806 → … → primary benchmark 21 testbeds / 14 episodes / 21 gauges)**.\nFigures, web portal, and Hugging Face catalog/deploy are out of scope.\n\nThe chain has two layers:\n\n1. **Selection reconstruction (`steps/s01…s19`)** — a clean, step-by-step port of the\n   original selection chain, verified membership-exact against the authors' internal\n   reference at every stage (see `PIPELINE.md` for the per-step record). The historical\n   verified counts in `PIPELINE.md` (L3 qualifying 688 → … → internal final 26/20) refer\n   to that *pre-recovery* internal reference and are superseded by the release stage below.\n2. **Release stage (`s20…s22`, this directory)** — builds the released v1.0 tables on the\n   **corrected single-funnel basis**: the NLDI watershed-retrieval hardening (documented in\n   `PIPELINE.md` → \"NLDI watershed-availability recovery\") restored 118 transiently-failed\n   testbeds, giving the released L3 pool of **806 testbeds (160 episodes, 208 gauges)**.\n   - `s20_build_release_tables.py` re-applies the identical screening chain on the corrected\n     pool and **asserts the full released funnel on every run**\n     (806/160/208 → strict 175 → dedup 115 → NCEI-confirmed 54 → RP ≥ 2 yr 31\n     → −4 regulated, −6 post-fire → **primary 21/14/21**; paper Sect. 3.4, Appendix Table A1;\n     machine-readable copy: `qc/tableA1_funnel.csv`).\n   - `s21_build_recovered_light_packages.py` builds the light packages for the recovered\n     testbeds.\n   - `s22_compute_qc_tables.py` computes the QC tables in `qc/` (paper Tables A2–A3).\n\n## Layout\n```\nconfig.py        all paths, thresholds, CRS, API constants (single source of truth)\nlib/             shared helpers: io, geo, nwis, nldi\nsteps/sNN_*.py   one module per stage of the selection reconstruction (s01…s19)\nrun_all.py       ordered orchestrator for s01…s19 (seeds caches, runs the chain, then verify.py)\nverify.py        membership diff vs the internal reference at every stage + validators\ns20…s22          release stage (released tables, light packages, QC tables)\nPIPELINE.md      per-step input/output record + verified results + drift note + known limitations\n```\n\n## Reproducibility scope — what this archive does and does not contain\n\nThis directory publishes the **complete construction logic** — every threshold, screen,\nkeep-rule, and assertion — and the code was re-executed end-to-end in the authors' build\nenvironment, reproducing the released membership exactly (`s20` asserts the funnel above\non every run).\n\nIt is **not runnable as-is from this archive alone**: the stage-0 inputs and the frozen\nnetwork caches (upstream NCEI/L2 products, the STN HWM snapshot, cached NWIS/NLDI/MRMS\nresponses; see `PIPELINE.md` → Prerequisites) live in the authors' build environment and\nare excluded here for size, and `config.py` and the `s20`–`s22` headers reference those\nlocal paths. Everything network-derived can be re-fetched from the same public agency\nAPIs (USGS NWIS/NLDI, STN, Iowa State MT MRMS archive) — `code/tools/` provides\nstandalone utilities for exactly that (see `code/tools/README.md`) — with the usual\ncaveat that live services evolve (`PIPELINE.md` → Drift note documents the known,\nmembership-neutral drift).\n\n## Quick start (in a populated build environment)\n```bash\npython run_all.py             # offline/cache-frozen (default): s01…s19 + verify.py\npython s20_build_release_tables.py    # released tables + funnel assertions\npython s21_build_recovered_light_packages.py\npython s22_compute_qc_tables.py\n```\n- **OFFLINE (default):** network fetches are suppressed; the chain reads through the seeded\n  caches → byte-faithful reproduction of the published selection.\n- **Live run:** `PIPELINE_OFFLINE=0 python run_all.py` — fetches live; expect the documented\n  availability drift (PIPELINE.md → Drif","encoding":"utf8","key":"code/pipeline/README.md","name":"README.md","size":4433,"truncated":true},{"bytes":"# Quality-control tables\n\n- `tableA1_funnel.csv` — machine-readable copy of the construction and screening\n  funnel (paper Appendix Table A1): one row per stage with criterion, before/after\n  counts, and notes, from the 42,466-episode L1 inventory to the 21-testbed primary\n  benchmark. The same counts are asserted programmatically on every run of\n  `code/pipeline/s20_build_release_tables.py`.\n- `tableA3_qc_primary21.csv` — per-testbed water-balance and upper-tail\n  diagnostics for the 21 primary testbeds (paper Appendix Table A3), generated\n  by `code/pipeline/s22_compute_qc_tables.py` under one stated convention:\n  runoff `R` integrates the positive departures of observed discharge above the\n  pre-event baseline (median of the 24 h before episode begin) over\n  begin −24 h … end +72 h; `P` is the basin-mean MRMS event precipitation\n  depth (seven-day extended window for 04293000, whose causative rainfall\n  preceded the standard window). `RC = R/P`. Two testbeds exceed RC = 1 and are\n  flagged, not removed: 08165500 (approved but estimated indirect peak at an\n  overtopped gauge) and 01596500 (suspected MRMS underestimation of orographic\n  rainfall). `UTR_record = Q_record/Q10` and `UTR_event = Q_event/Q10`\n  (Smith et al., 2018, \"Strange Floods\").\n- `tableA2_sensitivity.csv` — one-at-a-time threshold sensitivity of the\n  screening chain, re-run on the full 806 pool with curation re-applied\n  (paper Appendix Table A2).\n- `upper_tail_ratio_primary21.csv` — upper-tail ratio source table.\n\n**Vertical datums.** Across the whole L3 pool, `data/L3/l3_testbed_hwms.csv` holds\n31,726 (testbed, HWM) rows covering 3,141 distinct in-basin high-water marks: 2,567\nof those marks (81.7%) are referenced to NAVD88 (`hwm_vertical_datum == 2`), 1 is on\nanother datum, and 573 (18.2%) carry no datum value at all — so essentially every\nmark that reports a datum reports NAVD88, but roughly one in five reports none.\nDatum coverage is markedly better behind the 21 primary testbeds: of the 730 distinct\nmarks inside those basins, 690 (94.5%) are NAVD88 and the remaining 40 (5.5%) carry\nno datum. Per-mark datum fields are preserved in `data/L3/l3_testbed_hwms.csv` and\nin the package `hwms_inside.csv` files, so any user-side filtering on datum can be\nreproduced exactly.\n","encoding":"utf8","key":"qc/README.md","name":"README.md","size":2294,"truncated":false}],"sourceObjectKey":"catalog/l1_episodes.parquet","starterCell":"from datasets import load_dataset\nds = load_dataset(\"skyan1002/flash-flood-benchmark-data\")\nprint(ds)","starterSamplePath":"AGENTS.md","totalObjects":4800},"summary":"TORRENT is a benchmark of reproducible flash-flood testbeds for the contiguous United States, built for hydrological-response analysis and model intercomparison (companion paper…","title":"TORRENT: CONUS Flash-Flood Benchmark of Episode-Gauge-Watershed Testbeds","updatedAt":"2026-11-17T17:16:21.771Z","visibility":"public","websiteUrl":"https://huggingface.co/datasets/skyan1002/flash-flood-benchmark-data"}}