{"success":true,"data":{"author":"City of Boston (Boston Police Department)","bucketAccessType":"public","bucketListable":true,"bucketName":"scigantic-boston-open-data","bucketPrefix":"bpd-fio/","category":"Urban & Civic","cloudProvider":"aws","createdAt":"2026-08-15T00:00:00.000Z","description":"The Boston Police Department's record of Field Interrogation, Observation, Frisk and Search encounters, released by BPD for transparency and oversight of the FIO programme: 356,315 rows across 27 DataStore-loaded resources spanning 2011 to 2025, plus three field-description spreadsheets. BPD publishes it, in its own words, to add transparency to the FIO programme while protecting the privacy of the individuals involved, and it is the primary public record for research and oversight of stop, frisk and search practice in the city. The structure reflects three successive records management systems, and getting that structure right is the whole problem. The 2011 to 2015 legacy system is one flat 152,230-row file with 44 columns: a sequence and FIO id, the encounter date and time, the location and BPD district, whether the subject had priors, physical description, complexion, the FIO type (the FIOFS_TYPE code encodes which of interrogation, observation, frisk and search occurred), a terrorism flag, whether a search took place and its legal basis, the stop reason, the outcome, vehicle details, the officer and supervisor identifiers and names, and the subject's recorded sex, race and ethnicity. From 2015 onward the data is published as paired tables per year: a fieldcontact table with one row per encounter and a fieldcontact_name table with one row per person involved in an encounter, which is why the name-table row counts run roughly twice the contact-table counts. Joining them on the contact identifier is required to relate an encounter to the people in it, and counting rows in the name table counts people, not stops. Two data-quality facts are load-bearing and documented in the schema card. FIO_DATE in the legacy file ranges from 1964 to 2044, with 1,455 of 152,230 rows falling outside a plausible 2011 to 2016 window, so any time series has to filter to a sane date range first or the axis stretches over eighty years. And FIO_TIME is stored as a timestamp whose date component duplicates FIO_DATE rather than as a time of day, so hour-of-day analysis has to extract the time part rather than treating the column as a datetime. The schema card also documents the demographic column vocabularies (RACE_DESC, ETHNICITY, SEX) and the search-basis and outcome codes. Published by the Boston Police Department under PDDL 1.0. This archive is served from a Scigantic mirror rather than live from the portal: the yearly files are published once and not revised, so s3://scigantic-boston-open-data/bpd-fio/ carries a byte-verbatim copy under raw/ alongside a Snappy Parquet copy under parquet/, FUSE-mounted read-only at ~/archive.","displayTitle":"Boston Police Field Interrogation and Observation (FIO) records, 2011 to 2025","dockerImageUrl":"531775725781.dkr.ecr.us-east-1.amazonaws.com/scigantic-notebook-minimal:latest","id":"95278561-7c56-5dd5-8eff-ce7de4326691","license":"Open Data Commons Public Domain Dedication and License (PDDL) 1.0","objectCountEstimate":2,"probeStatus":"ok","referenceUrl":"https://data.boston.gov/dataset/boston-police-department-fio","region":"us-east-1","schemaCard":{"accessHints":[{"language":"python-anonymous","snippet":"import boto3\nfrom botocore import UNSIGNED\nfrom botocore.client import Config\ns3 = boto3.client('s3', region_name='us-east-1', config=Config(signature_version=UNSIGNED))\ns3.list_objects_v2(Bucket='scigantic-boston-open-data', Prefix='bpd-fio/', Delimiter='/')"},{"snippet":"open('/mnt/archive/parquet/2024_annual_fio.parquet', 'rb')","language":"python-local"}],"caveats":["Files are binary — use a domain-specific parser (e.g. metpy / pynexrad / pydicom) rather than text I/O."],"format":"binary","generatedAt":"2026-08-31T02:04:56.391Z","layout":{"example":"bpd-fio/parquet/2024_annual_fio.parquet","pattern":"{prefix}/{seg1}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"50415231150415ecf20715d0e7024c159a481500120000b6f903200a000000464332343001010032320e000033320e0000372e0e00043130320e00362a0000392e1c000430382e0e0000313262000431312e1c0000323246000031324600003232700000323238000431352e4600043136320e0032b6000033320e000031329a000032329a000032322a000435342e54000032320e000033363800327e000032329a000034360e00323800003332d2000032329a00003432fc0000363638003254000033327e00003336a8003654003270000034327a010033327a01003432460000343218010034362a00327e000034327000003532b600003532d200003532c4000035328c00003532a8000035327e000035366200328c00003632380000363270000036326200003632e00000363254000036329a0000363262000038320e0000373270000037327000003732fc000037327e000037327e0000373242010037328c000037327000003832620000383662003226010038329a000039327e000038320e000039328c000038360e00320a0100383262000039327e0000382ec40004313336a80032d2000039321c00003936a80032700000392e5400043039367e003654002e26010431302e62000431302e6200003132ec040431312eee00003132b4040031320805043130362a002ea8000431302ea8000031323205003132","encoding":"hex","key":"bpd-fio/parquet/2024_annual_fio.parquet","truncatedAt":512},{"key":"bpd-fio/parquet/2024_annual_fio_name.parquet","bytes":"50415231150415f2f50f15f2d3084c15ee87011500120000f9fa07380b0000003630313930373733393136110f1833363137363631110f1837353135343035090f203230313339363239371d0f08383334320f0008393230151e143631353831320d0f1c3136303835383839110f14373635343232113c1c3138363837323338111e14383639323435153c143931363132320d960032010f08323136113c1832373737303236153c18393833383338340d4b14323233383032154b010f0833363311d2183237303836373911b41832383932313736153c1438393234363711870c32393237015e0d5a1834393939323939114b183530373930313811a508363438014d110f183939353337303111a51839393031303532113c010f043936154b18393930353131311187183939303637383111f018393930383930302db31c33313130333237391d0f0c343730370da51c33313131363339332e0f00083438300df01433313730303535c2010f0836353411b4010f0c393536370dc31c33323533333034382d0e1c33323831303736300d96010f0c333332390db4010f08343235314a010f083033363186010f0c313132314d0d1c3332383630393534119618323836313032342d2c1c333439303234373611c318353637353939330d691c32323632363432330d4b1832333736323130320f00043732311d1c323339303630","encoding":"hex","truncatedAt":512}],"sampleObjects":["bpd-fio/parquet/2024_annual_fio.parquet","bpd-fio/parquet/2024_annual_fio_name.parquet","bpd-fio/parquet/boston-police-department-fio.parquet","bpd-fio/parquet/fieldcontactforpublic2015.parquet","bpd-fio/parquet/fieldcontactforpublic2016.parquet"],"sidecars":[{"key":"bpd-fio/README.md","size":3823,"name":"README.md","bytes":"# BPD Field Interrogation and Observation (FIO)\n\nVerbatim mirror of City of Boston open data, taken from data.boston.gov.\nUpstream cadence: annual; last portal update 2026-02-10.\nLicense: Open Data Commons Public Domain Dedication and License (PDDL). PDDL is a public-domain dedication, so this\nredistribution is explicitly permitted; the city remains the source of record.\n\n- `raw/` is byte-identical to what data.boston.gov serves.\n- `parquet/` holds the same rows as Snappy Parquet, every column a string\n  (zero-padded ids like ward `06` and ZIP `02128` would be destroyed by type\n  inference). Cast explicitly when you need numbers.\n\nSee MANIFEST.json at the bucket root for source URLs, CKAN resource ids,\nsha256 digests and row counts.\n\n| file | format | rows | raw | parquet |\n| --- | --- | --- | --- | --- |\n| fio_2025.csv | CSV | 5,903 | 5.2M | fio_2025.parquet 2.3M |\n| fio_names_2025-1.csv | CSV | 10,704 | 1.3M | fio_names_2025-1.parquet 0.3M |\n| 2024_annual_fio.csv | CSV | 4,860 | 4.4M | 2024_annual_fio.parquet 1.9M |\n| 2024_annual_fio_name.csv | CSV | 8,832 | 0.9M | 2024_annual_fio_name.parquet 0.2M |\n| fio__reports_2023_2-1.csv | CSV | 5,868 | 5.1M | fio__reports_2023_2-1.parquet 2.2M |\n| fio_names_2023_2-1.csv | CSV | 10,199 | 1.2M | fio_names_2023_2-1.parquet 0.3M |\n| fio-records-2022-mark43-fieldcontact-table.csv | CSV | 5,278 | 2.6M | fio-records-2022-mark43-fieldcontact-table.parquet 1.0M |\n| fio-records-2022-mark43-fieldcontact_name-table.csv | CSV | 9,408 | 1.1M | fio-records-2022-mark43-fieldcontact_name-table.parquet 0.2M |\n| fio_2021-2.csv | CSV | 4,582 | 3.6M | fio_2021-2.parquet 1.6M |\n| fio_names_2021.csv | CSV | 7,787 | 0.9M | fio_names_2021.parquet 0.2M |\n| mark43_fieldcontacts_for_public_2020_202104151551.csv | CSV | 5,717 | 4.2M | mark43_fieldcontacts_for_public_2020_202104151551.parquet 1.9M |\n| mark43_fieldcontacts_name_for_public_2020_202104151551.csv | CSV | 10,224 | 1.0M | mark43_fieldcontacts_name_for_public_2020_202104151551.parquet 0.1M |\n| mark43_fieldcontacts_for_public_20192.csv | CSV | 1,709 | 1.1M | mark43_fieldcontacts_for_public_20192.parquet 0.5M |\n| mark43_fieldcontacts_name_for_public_2019.csv | CSV | 2,963 | 0.3M | mark43_fieldcontacts_name_for_public_2019.parquet 0.0M |\n| fiokeymark43-1-1.xlsx | XLSX | 34 | 0.0M | fiokeymark43-1-1__fieldcontact.parquet 0.0M |\n| rms_fieldcontacts_for_public_2019.csv | CSV | 6,628 | 3.9M | rms_fieldcontacts_for_public_2019.parquet 1.7M |\n| rms_fieldcontacts_name_for_public_2019.csv | CSV | 11,481 | 1.2M | rms_fieldcontacts_name_for_public_2019.parquet 0.3M |\n| rms_fieldcontacts_for_public_2018_202003111433.csv | CSV | 8,708 | 4.9M | rms_fieldcontacts_for_public_2018_202003111433.parquet 2.1M |\n| rms_fieldcontacts_name_for_public_2018_202003111443.csv | CSV | 14,093 | 1.6M | rms_fieldcontacts_name_for_public_2018_202003111443.parquet 0.4M |\n| rms_fieldcontacts_for_public_2017_202003111424.csv | CSV | 8,809 | 4.2M | rms_fieldcontacts_for_public_2017_202003111424.parquet 1.7M |\n| rms_fieldcontacts_name_for_public_2017_202003111442.csv | CSV | 15,017 | 1.7M | rms_fieldcontacts_name_for_public_2017_202003111442.parquet 0.5M |\n| fieldcontactforpublic2016.csv | CSV | 9,048 | 4.1M | fieldcontactforpublic2016.parquet 1.3M |\n| fieldcontactnameforpublic2016.csv | CSV | 14,994 | 2.2M | fieldcontactnameforpublic2016.parquet 0.5M |\n| fieldcontactforpublic2015.csv | CSV | 9,203 | 3.0M | fieldcontactforpublic2015.parquet 0.8M |\n| fieldcontactnameforpublic2015.csv | CSV | 11,989 | 1.7M | fieldcontactnameforpublic2015.parquet 0.4M |\n| fiokeynewrms.xlsx | XLSX | 34 | 0.0M | fiokeynewrms__fieldcontact.parquet 0.0M |\n| boston-police-department-fio.csv | CSV | 152,230 | 62.2M | boston-police-department-fio.parquet 8.9M |\n| fiofielddescriptions.xlsx | XLSX | 43 | 0.0M | fiofielddescriptions.parquet 0.0M |\n","truncated":false,"encoding":"utf8"}],"sourceObjectKey":"bpd-fio/parquet/2024_annual_fio.parquet","starterCell":"with open('/mnt/archive/parquet/2024_annual_fio.parquet', 'rb') as fh:\n    head = fh.read(1024)\nprint(len(head), head[:64])","starterSamplePath":"parquet/2024_annual_fio.parquet","totalObjects":"2 top-level prefixes","verifiedRead":{"attempts":[{"ok":true,"family":"csv","path":"raw/rms_fieldcontacts_name_for_public_2017_202003111442.csv","seconds":1.17}],"facts":["under 'bpd-fio/': 2 directories (e.g. parquet, raw); 2 objects named like README.md, 4 KB to 23 KB","under 'bpd-fio/raw/': 28 objects named like fieldcontactforpublic####.csv, 12 KB to 59.3 MB"],"starterCell":{"seconds":0.24,"ok":true},"path":"raw/rms_fieldcontacts_name_for_public_2017_202003111442.csv","verifiedAt":"2026-09-14T18:22:04.966Z","snippet":"import os\nimport pandas as pd\ndf = pd.read_csv(os.path.join(os.environ['SCIGANTIC_MOUNT_PATH'], 'raw/rms_fieldcontacts_name_for_public_2017_202003111442.csv'), nrows=1000)\ndf.head()","bytes":1708136,"seconds":1.17,"family":"csv","image":"531775725781.dkr.ecr.us-east-1.amazonaws.com/scigantic-notebook-minimal:latest","rssDeltaMb":72.9,"library":"pandas","summary":"CSV/TSV: columns ['recnum', 'fc_num', 'contact_date', 'sex', 'race', 'age', 'build', 'hair_style', 'complexion', 'ethnicity', 'otherclothing'], dtypes ['int64', 'str', 'str', 'str', 'str', 'int64', 'str', 'str', 'str', 'str', 'str'], first 1000 rows read"}},"sourceType":"bucket","summary":"The Boston Police Department's record of Field Interrogation, Observation, Frisk and Search encounters, released by BPD for transparency and oversight of the FIO programme:…","title":"Boston Police Field Interrogation and Observation (FIO) records, 2011 to 2025","updatedAt":"2026-09-14T18:22:04.966Z","visibility":"public","websiteUrl":"https://www.boston.gov/departments/police","whiteLabel":"boston"}}