{"success":true,"data":{"author":"Marco Vela et al.","bucketAccessType":"public","bucketName":"mvelar/lightning-nowcasting","bucketPrefix":"","category":"Climate & Weather","cloudProvider":"huggingface","createdAt":"2026-11-16T15:39:55.934Z","description":"Sequences of co-located GOES-16 lightning and infrared observations over tropical and subtropical South America for training models that forecast lightning 10 to 120 minutes ahead (Vela et al., WAIMLAp 2026). Each sequence is a 128 x 128 cell patch (about 256 km across) covering 4 hours at 10-minute steps: 12 input frames and 12 target frames. Each frame has eight channels: flash-extent density (FED) from the Geostationary Lightning Mapper (GLM L2 LCFA) and seven ABI L1b infrared bands. There are 77,976 sequences: a development set of 71,010 sequences from 2019-04-03 to 2020-03-31 in twelve monthly folders, and a test set of 6,966 sequences from 36 days between September 2023 and February 2024 that follow the Latin America lightning nowcasting benchmark days of Cintineo (2024). Each folder holds patches_YYYY-MM.nc (int16 sequences with shape sample x 24 x 8 x 128 x 128, scale factor 1/32767), fed_true_position_YYYY-MM.nc and a provenance CSV; about 296 GB of NetCDF-4 in total. Read with xarray or netCDF4. Licensed CC BY 4.0 by the curator; source data are NOAA GOES-16. Build code is on Zenodo (doi:10.5281/zenodo.22979408).","displayTitle":"GOES-16 GLM Lightning Nowcasting Patches over South America","huggingFaceUrl":"https://huggingface.co/datasets/mvelar/lightning-nowcasting","id":"8253ef55-cba4-53a0-b021-59f782528251","license":"CC-BY-4.0","licenseEvidenceUrl":"https://huggingface.co/datasets/mvelar/lightning-nowcasting","objectCountEstimate":41,"probeStatus":"ok","referenceUrl":"https://doi.org/10.5281/zenodo.22979408","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"mvelar/lightning-nowcasting\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"mvelar/lightning-nowcasting\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=mvelar, license:cc-by-4.0, size_categories:1K<n<10K, format:csv, downloads=448"],"format":"mixed","generatedAt":"2026-11-16T15:40:02.922Z","layout":{"example":"2019-04/provenance_2019-04.csv","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"index,source_index,split,year,doy,window,start_utc,kind,y0,x0,lat,lon,pos_frac,activity\r\n0,0,train,2019,93,0,00:00,active,402,1303,-15.7278,-58.1629,0.66925048828125,13507.798828125\r\n1,1,train,2019,93,0,00:00,active,666,1182,-10.6849,-60.9463,0.451171875,6053.2255859375\r\n2,2,train,2019,93,0,00:00,active,145,1387,-20.8506,-55.7529,0.3555908203125,1514.4195556640625\r\n3,3,train,2019,93,0,00:00,active,1224,443,-0.4387,-74.7449,0.4476318359375,9458.4521484375\r\n4,4,train,2019,93,0,00:00,active,1340,556,1.6601,-72","encoding":"utf8","key":"2019-04/provenance_2019-04.csv","truncatedAt":512}],"sampleObjects":["2019-04/fed_true_position_2019-04.nc","2019-04/patches_2019-04.nc","2019-04/provenance_2019-04.csv","2019-05/fed_true_position_2019-05.nc","2019-05/patches_2019-05.nc"],"sidecars":[{"bytes":"---\nlicense: cc-by-4.0\ntask_categories:\n- time-series-forecasting\ntags:\n- lightning\n- nowcasting\n- GOES-16\n- GLM\n- ABI\n- meteorology\n- South America\npretty_name: GOES-16 lightning nowcasting patches over South America\nsize_categories:\n- 10K<n<100K\n---\n\n# GOES-16 lightning nowcasting patches over South America\n\n## Dataset summary\n\nThis dataset contains sequences of co-located GOES-16 lightning and infrared\nobservations over tropical and subtropical South America. It was built to train and\nevaluate machine-learning models that forecast lightning 10 to 120 minutes ahead.\n\nEach sequence covers 128 × 128 grid cells (about 256 km on a side) and 4 hours at\n10-minute steps: 12 input\nframes followed by 12 target frames. Each frame has eight channels: the flash-extent\ndensity (FED) from the Geostationary Lightning Mapper (GLM) and seven infrared bands of\nthe Advanced Baseline Imager (ABI).\n\nThere are 77,976 sequences in total, of which 77,778 are used in the accompanying paper:\nthe 198 sequences of the embargo day (2020-02-05) are released but not used. The\ndevelopment set has 71,010 sequences from\n2019-04-03 to 2020-03-31, released as twelve monthly files and used for training\nand validation. The test set has 6,966 sequences from 36 days between September 2023 and\nFebruary 2024, following the days of the Latin America lightning nowcasting benchmark\n(Cintineo, 2024).\n\nThe dataset accompanies the paper *A Satellite Lightning Nowcasting Benchmark for South America: Temporal Cross-Attention in ConvLSTMs* (Vela et al., WAIMLAp 2026).\n\n- Curated by: Marco Vela\n- License: CC-BY-4.0\n- Code: doi:10.5281/zenodo.22979408 (Zenodo)\n- Source data: GOES-16 GLM L2 LCFA and ABI L1b radiances, NOAA\n\n## Files\n\nThe data are hosted in this repository, one folder per month of the development set and one\nfor the test set. Each folder holds three files:\n\n| Folder | Files |\n|---|---|\n| `2019-04/` to `2020-03/` | `patches_YYYY-MM.nc`, `fed_true_position_YYYY-MM.nc`, `provenance_YYYY-MM.csv` |\n| `test/` | `patches_test.nc`, `fed_true_position_test.nc`, `provenance_test.csv` |\n\nThe files hold about 296 GB in total. The code that builds them is in a separate Zenodo\nrecord (doi:10.5281/zenodo.22979408), which contains the scripts, the configuration and\nthis card.\n\n## Dataset structure\n\n### Files\n\nEach file is NetCDF-4 (HDF5) with three variables:\n\n| Variable | Type | Dimensions | Description |\n|---|---|---|---|\n| `sequences` | int16 | `(sample, time, channel, y, x)` = `(n, 24, 8, 128, 128)` | Packed values; `scale_factor` = 1/32767 gives values in [0, 1] |\n| `year` | int16 | `(sample,)` | Year of the sequence (UTC) |\n| `doy` | int16 | `(sample,)` | Day of year of the sequence (UTC) |\n\nSequences are in chronological order. Each one is stored as its own compressed chunk\n(zlib level 4 with byte shuffle), so reading a single sequence does not decompress its\nneighbors. Packing to int16 adds an error of at most 1.5 × 10⁻⁵ to the scaled value.\n\nThe global attributes of each monthly file give the month, the number of sequences, the\nsplits it contains, the split rule, and the index range of the month in the original\nsingle file.\n\n### Provenance tables\n\nEach data file comes with a CSV that has one row per sequence, in the same order as the\nfile:\n\n| Column | Meaning |\n|---|---|\n| `index` | Position of the sequence in the file |\n| `source_index` | Position in the single file used for training (test: same as `index`) |\n| `split` | train, embargo, validation, or test |\n| `year`, `doy` | Date (UTC) |\n| `window`, `start_utc` | Window of the day (0 to 10) and the time of its first frame |\n| `kind` | `active` (activity-weighted draw) or `background` (uniform draw) |\n| `y0`, `x0` | Grid row and column of the patch's first cell; row 0 is the southern edge |\n| `lat`, `lon` | Latitude and longitude of the patch center |\n| `pos_frac` | Share of the patch's cells with FED > 0 in some frame of the window |\n| `activity` | Total scaled FED of the patch over the window (the sampling weight is its square root) |\n\nThe tables were produced by replaying the sampler on the gridded data","encoding":"utf8","key":"README.md","name":"README.md","size":20346,"truncated":true}],"sourceObjectKey":"2019-04/provenance_2019-04.csv","starterCell":"import pandas as pd\ndf = pd.read_csv(\"hf://datasets/mvelar/lightning-nowcasting/2019-04/provenance_2019-04.csv\")\ndf.head()","starterSamplePath":"2019-04/fed_true_position_2019-04.nc","totalObjects":41},"summary":"Sequences of co-located GOES-16 lightning and infrared observations over tropical and subtropical South America for training models that forecast lightning 10 to 120 minutes ahead…","title":"GOES-16 GLM Lightning Nowcasting Patches over South America","updatedAt":"2026-11-17T17:13:48.261Z","visibility":"public","websiteUrl":"https://huggingface.co/datasets/mvelar/lightning-nowcasting"}}