{"success":true,"data":{"author":"City of Boston (DoIT Data & Analytics)","bucketAccessType":"public","bucketListable":false,"bucketName":"data.boston.gov","category":"Urban & Civic","cloudProvider":"api","createdAt":"2026-08-15T00:00:00.000Z","description":"RentSmart Boston joins BOS:311 requests to Inspectional Services enforcement records so that a prospective tenant can see the complaint and violation history of a specific address: 491,953 rows across four DataStore-loaded resources, of which the main file holds 387,923. One row is one recorded issue at one address. Each carries the timestamp, the violation_type (Housing Violations, Building Violations, Enforcement Violations, Housing Complaints, Sanitation Requests, Civic Maintenance Requests), a free-text description of the specific problem (Rodent Activity, Illegal Occupancy, No Heat, Unsanitary Conditions), the address and ZIP, the neighborhood, the assessing parcel id, the property owner name, the year built and year remodeled, the property type (for example 'Residential 3-family'), and latitude and longitude. The parcel column joins directly to the Property Assessment rolls and to the 2025 parcel geometry, and the owner column makes it possible to aggregate conditions across a landlord's whole portfolio rather than one building at a time, which is what distinguishes this dataset from the raw 311 feed it partly derives from. It supports work on habitability, differential enforcement across neighborhoods, the relationship between building age or ownership structure and reported conditions, and the geography of rodent and sanitation complaints. Two things are documented in the schema card. The main resource is titled 'RENTSMART 2016 - PRESENT' but its earliest record is 7 September 2021, so the title overstates the coverage by five years and any claim about the 2016 to 2021 period drawn from this file is unfounded. And because the file is a union of two very different upstream sources, a 'Sanitation Request' row is a resident-reported 311 case with no finding attached, while a 'Housing Violation' row is an inspector-issued citation, so mixing the two into a single count of problems per building conflates complaint volume with enforcement outcome. Published by DoIT Data and Analytics under PDDL 1.0.","displayTitle":"Boston RentSmart housing conditions by address","dockerImageUrl":"531775725781.dkr.ecr.us-east-1.amazonaws.com/scigantic-notebook-minimal:latest","id":"2f0c0e42-b633-5514-84a6-f0abbd660d9d","license":"Open Data Commons Public Domain Dedication and License (PDDL) 1.0","referenceUrl":"https://data.boston.gov/dataset/rentsmart","region":"us-east-1","schemaCard":{"generatedAt":"2026-08-16T17:39:25.112Z","format":"v2-hybrid","sampleObjects":["dc615ff7-2ff3-416a-922b-f0f334f085d0  RENTSMART 2016 - PRESENT","f6b5aee9-8607-4ba8-ac6b-17b3ffb43084  1900-to-1919.csv","d401d262-6556-40d9-a31b-45f79144f638  1800-to-1849.csv","ecb8e8a6-b5cf-41ea-8cbe-73fd53e03219  1700-to-1799.csv"],"rowCount":491953,"accessHints":[{"snippet":"import io, json, urllib.error, urllib.parse, urllib.request\nimport pandas as pd\n\nCKAN = \"https://data.boston.gov/api/3/action\"\n\ndef ckan(action, **params):\n    \"\"\"Any CKAN action endpoint. Plain urllib works; no auth, no headers.\"\"\"\n    url = f\"{CKAN}/{action}?\" + urllib.parse.urlencode(params)\n    with urllib.request.urlopen(url, timeout=300) as r:\n        return json.load(r)[\"result\"]\n\ndef sql(query):\n    \"\"\"Server-side SQL over the DataStore. Warns when the 32,000-row cap clips.\"\"\"\n    res = ckan(\"datastore_search_sql\", sql=query)\n    if res.get(\"records_truncated\"):\n        print(\"WARNING: clipped at the 32,000-row cap; aggregate or page with OFFSET\")\n    return pd.DataFrame(res[\"records\"])\n","language":"python"},{"language":"python","snippet":"# Bulk CSV download. pd.read_csv(url) on this portal raises 403: the CKAN link\n# 302s to a presigned S3 URL that the redirect fails on. Resolve it by hand and\n# drop the explicit :443, which is the single change that makes it a 200.\nclass _NoRedirect(urllib.request.HTTPRedirectHandler):\n    def redirect_request(self, *a, **k):\n        return None\n\ndef boston_csv(url, **read_csv_kwargs):\n    try:\n        urllib.request.build_opener(_NoRedirect).open(url, timeout=120)\n        signed = url\n    except urllib.error.HTTPError as e:\n        if e.code not in (301, 302, 303, 307, 308):\n            raise\n        signed = e.headers[\"Location\"].replace(\"s3.amazonaws.com:443\", \"s3.amazonaws.com\")\n    with urllib.request.urlopen(signed, timeout=900) as r:\n        return pd.read_csv(io.BytesIO(r.read()), **read_csv_kwargs)\n"},{"language":"python","snippet":"# Resolve a resource id by package name rather than hardcoding it, and page\n# past the 32,000-row cap. Verified against a 282,836-row resource.\ndef resources(package_name):\n    pkg = ckan(\"package_show\", id=package_name)\n    return [\n        {\"id\": r[\"id\"], \"name\": r.get(\"name\"), \"file\": r[\"url\"].rsplit(\"/\", 1)[-1],\n         \"format\": r.get(\"format\"), \"live\": bool(r.get(\"datastore_active\"))}\n        for r in pkg[\"resources\"]\n    ]\n\ndef read_all(resource_id, page=32000, **filters):\n    \"\"\"Every row of a DataStore resource, in _id order.\"\"\"\n    out, offset = [], 0\n    while True:\n        res = ckan(\"datastore_search\", resource_id=resource_id,\n                   limit=page, offset=offset, **filters)\n        rows = res[\"records\"]\n        out.extend(rows)\n        if len(rows) < page:\n            return pd.DataFrame(out)\n        offset += page\n"}],"layout":{"pattern":"One row is one recorded issue at one address. 491,953 rows across four DataStore resources; the main file holds 387,923 and a second holds 103,231. RentSmart is a JOIN of two upstream sources, BOS:311 requests and Inspectional Services enforcement records, published so a prospective tenant can look up a specific address.\nCOLUMNS (14): date, violation_type, description, address, neighborhood, zip_code, parcel, owner, year_built, year_remodeled, property_type, latitude, longitude.\nviolation_type IS THE SOURCE DISCRIMINATOR and takes six values: Housing Violations, Building Violations, Enforcement Violations, Housing Complaints, Sanitation Requests, Civic Maintenance Requests. The first three are inspector-issued citations with a finding behind them. The last three are resident-reported 311 cases with no finding attached. Mixing them into one \"problems per building\" count conflates complaint volume with enforcement outcome, and neighborhoods differ systematically in how much they call 311.\ndescription carries the specific issue (\"Rodent Activity\", \"Illegal Occupancy\", \"No Heat\", \"Unsanitary Conditions\") and is the column that actually says what happened.\nTHE TWO JOINS THAT MAKE THIS MORE THAN A 311 FILTER: parcel is the assessing parcel identifier, joining to the Property Assessment rolls (GIS_ID / PID), building permits (parcel_id) and the 2025 parcel geometry (MAP_PAR_ID). owner is carried on every row, so conditions can be aggregated across a landlord's whole portfolio rather than one building at a time, which is the analysis the raw 311 feed cannot support.\nRESOURCES:\n  RentSmart (data.boston.gov/dataset/rentsmart): 4 DataStore resources of 13 total\n    dc615ff7-2ff3-416a-922b-f0f334f085d0     387923 rows  RENTSMART 2016 - PRESENT\n    f6b5aee9-8607-4ba8-ac6b-17b3ffb43084     103231 rows  1900-to-1919.csv\n    d401d262-6556-40d9-a31b-45f79144f638        762 rows  1800-to-1849.csv\n    ecb8e8a6-b5cf-41ea-8cbe-73fd53e03219         37 rows  1700-to-1799.csv","example":"Issue mix by neighborhood, split by whether it is a citation or a complaint:\n  SELECT \"neighborhood\", \"violation_type\", count(*) n\n  FROM \"dc615ff7-2ff3-416a-922b-f0f334f085d0\"\n  GROUP BY 1, 2 ORDER BY n DESC LIMIT 30"},"columns":[{"type":"int","name":"_id"},{"type":"text","name":"date"},{"name":"violation_type","type":"text"},{"name":"description","type":"text"},{"name":"address","type":"text"},{"name":"neighborhood","type":"text"},{"name":"zip_code","type":"text"},{"type":"text","name":"parcel"},{"name":"owner","type":"text"},{"type":"text","name":"year_built"},{"type":"text","name":"year_remodeled"},{"type":"text","name":"property_type"},{"type":"text","name":"latitude"},{"type":"text","name":"longitude"}],"sampleRows":[{"longitude":"-71.07775000173515","year_remodeled":null,"property_type":"Residential 3-family","description":"Rodent Activity","neighborhood":"Roxbury","__schema":"main file","date":"2026-08-11 02:04:34.067+00","violation_type":"Sanitation Requests","_id":1,"parcel":"0900562000","owner":"BETTENCOURT ARLENE J","latitude":"42.33945999996866","__resourceId":"dc615ff7-2ff3-416a-922b-f0f334f085d0","address":"75 Worcester St, 02118","zip_code":"02118","year_built":"1870"}],"caveats":["DO NOT look for a FUSE mount. This archive has cloudProvider \"api\": nothing is mounted, ~/archive and /mnt/archive do not exist, and there are no files on disk to walk. The generic archive preamble you may have been given says otherwise; it is wrong for this row. All data is read over HTTPS from the CKAN API at https://data.boston.gov/api/3/action. The notebook has outbound internet; no credentials, tokens or headers are required for any endpoint below.","DO NOT trust a result set of exactly 32,000 rows. Both datastore_search and datastore_search_sql cap a single response at 32,000 records no matter what LIMIT you write, so a query over a million-row resource returns a full-looking frame that is silently a fragment. datastore_search_sql sets records_truncated: true in the result when it clips; check it on every call. The fixes: aggregate server-side (GROUP BY in SQL returns few enough rows to fit), or page with ORDER BY _id plus OFFSET, which is verified to work past 280,000.","DO NOT do arithmetic or date comparison on a column without casting it. Almost every DataStore column on this portal is typed text regardless of what it holds, so avg(\"latitude\") returns HTTP 409 Conflict rather than a number, and min/max on a date column sorts lexically and returns a confident wrong answer. Cast explicitly in SQL: \"latitude\"::float, \"open_dt\"::timestamp, date_part('year', \"open_dt\"::timestamp). Where a column IS genuinely typed numeric or timestamp it is marked as such in the columns list below, and those are safe to use directly.","SQL AGGREGATES COME BACK AS STRINGS, so arithmetic on the resulting DataFrame silently fails or, worse, concatenates. PostgreSQL `numeric` results are serialized as JSON strings by the CKAN API (sum(\"OVERTIME\") returns '153220443.25', not 153220443.25) while `int`/`bigint` and `double precision` come back as JSON numbers. So count(*) is an int, avg() and sum() over a numeric column are strings, and dividing one such column by another raises \"unsupported operand type(s) for /: 'str' and 'str'\". Run pd.to_numeric on every aggregate column before doing arithmetic on it. Do NOT blanket-coerce every column: ward \"06\", precinct \"0104\" and ZIP \"02128\" are zero-padded identifiers that must stay strings.","A 403 FROM datastore_search_sql MEANS THE SQL TEXT TRIPPED AN EDGE FILTER, NOT THAT YOUR QUERY IS WRONG. Some perfectly valid PostgreSQL is rejected with HTTP 403 Forbidden before it ever reaches the database, so there is no SQL error message to read and retrying the identical query never helps. Verified blocked, deterministically, 5 of 5 attempts each (2026-08-15): agg(...) FILTER (WHERE ...) · extract(<part> FROM ...) · substring(x FROM a FOR b) · trim(BOTH ... FROM ...) · NULLIF(...) · a literal constant comparison such as 1=2. Verified working substitutes for each: sum(CASE WHEN cond THEN 1 ELSE 0 END) for FILTER · date_part('year', ts) and date_part('epoch', ts_a - ts_b) for extract · left(col, n) for substring · CASE WHEN d = 0 THEN NULL ELSE d END for NULLIF. Subqueries, CTEs, LEFT JOIN, IN (...), HAVING, window functions, percentile_cont WITHIN GROUP, the regex operator ~, and casts are all fine. When a query 403s, rewrite it; do not retry it.","DO NOT call pd.read_csv() on a data.boston.gov download URL. It raises HTTP 403. The CKAN download link 302-redirects to a presigned S3 URL on the bucket og-production-open-data-bostonma-892364687672, and the request fails when it is followed from urllib's redirect path, which is what pandas uses. Verified 2026-08-15: pd.read_csv(url) -> 403, while requests.get(url) on the identical URL -> 200. Two fixes, both verified: read through requests (pd.read_csv(io.BytesIO(requests.get(url).content))), or resolve the redirect yourself and strip the explicit \":443\" from the returned Location before fetching it. The bucket itself is private (anonymous ListObjectsV2 and GetObject both return AccessDenied), so there is no S3 path around this: the presigned link is the only way in. See accessHints for a copy-paste helper. None of this applies to the /api/3/action endpoints, which are plain JSON over HTTPS and work with bare urllib.","Resource ids, not names, are the addressing key, and the names are unreliable: CKAN's resource `name` field is null on many Boston resources, and several packages publish generated filenames (tmp4myyj_u8.csv) that carry no year. The ids listed in this card were captured on 2026-08-15. The package NAME is the stable handle, so resolve ids at runtime with GET /api/3/action/package_show?id=<package-name> and match on the resource `name` or `url` rather than hardcoding an index into the resource array, whose order is not guaranteed.","_id is CKAN's synthetic 1-based row number added by the DataStore loader, not a key from the source data. It is the right column to ORDER BY for stable paging and it is meaningless as a join key or an entity id.","THE RESOURCE TITLE OVERSTATES ITS COVERAGE BY FIVE YEARS. The main resource is labelled \"RENTSMART 2016 - PRESENT\" but its earliest record is 2021-09-07 (verified 2026-08-15). Any claim about 2016 to 2021 drawn from this file is unfounded. If you need the earlier period, go to the source datasets (311 service requests and the building and property violations packages) rather than trusting the label.","A CITATION AND A COMPLAINT ARE NOT THE SAME EVENT. \"Housing Violations\", \"Building Violations\" and \"Enforcement Violations\" are inspector findings. \"Housing Complaints\", \"Sanitation Requests\" and \"Civic Maintenance Requests\" are resident reports with no finding. Counting them together measures a mixture of building condition and resident reporting propensity, which vary independently across neighborhoods.","address is a single formatted string with the ZIP appended (\"75 Worcester St, 02118\"), not a structured address, and the same physical building appears with punctuation and unit-number variants. Join on parcel, not on address.","owner is free text copied from the assessing roll and carries the same problems: individuals, LLCs and trusts unnormalised, with no entity id. Portfolio analysis needs fuzzy matching and will both split and merge real owners, which matters because naming a landlord is a claim about a real party.","ABSENCE OF RECORDS IS NOT EVIDENCE OF GOOD CONDITION. A building with no rows may be well maintained, or may house tenants who do not call 311. RentSmart is explicitly a tenant information tool, and the city presents it as partial history rather than a condition rating.","year_built and year_remodeled come from the assessing roll and are text; year_remodeled is null for most rows, meaning no recorded remodel rather than an unknown one."],"totalObjects":4,"starterCell":"import io, json, urllib.error, urllib.parse, urllib.request\nimport pandas as pd\n\nCKAN = \"https://data.boston.gov/api/3/action\"\n\ndef ckan(action, **params):\n    \"\"\"Any CKAN action endpoint. Plain urllib works; no auth, no headers.\"\"\"\n    url = f\"{CKAN}/{action}?\" + urllib.parse.urlencode(params)\n    with urllib.request.urlopen(url, timeout=300) as r:\n        return json.load(r)[\"result\"]\n\ndef sql(query):\n    \"\"\"Server-side SQL over the DataStore. Warns when the 32,000-row cap clips.\"\"\"\n    res = ckan(\"datastore_search_sql\", sql=query)\n    if res.get(\"records_truncated\"):\n        print(\"WARNING: clipped at the 32,000-row cap; aggregate or page with OFFSET\")\n    return pd.DataFrame(res[\"records\"])\n\nRENTSMART = \"dc615ff7-2ff3-416a-922b-f0f334f085d0\"   # 387,923 rows, from 2021-09-07\n\n# Keep citations and complaints apart: they are different kinds of evidence.\nCITATION = (\"Housing Violations\", \"Building Violations\", \"Enforcement Violations\")\ndf = sql(f'''\n    SELECT \"neighborhood\", \"violation_type\", \"property_type\", count(*) AS n\n    FROM \"{RENTSMART}\"\n    WHERE \"neighborhood\" IS NOT NULL\n    GROUP BY 1, 2, 3\n''')\ndf[\"kind\"] = df[\"violation_type\"].isin(CITATION).map({True: \"citation\", False: \"complaint\"})\nprint(df.pivot_table(index=\"neighborhood\", columns=\"kind\", values=\"n\", aggfunc=\"sum\")\n        .fillna(0).astype(int).sort_values(\"citation\", ascending=False))\n\n# Confirm the real start date rather than trusting the resource title.\nprint(sql(f'SELECT min(\"date\"::timestamp) AS first, max(\"date\"::timestamp) AS last '\n          f'FROM \"{RENTSMART}\"').to_dict(\"records\"))\n"},"summary":"RentSmart Boston joins BOS:311 requests to Inspectional Services enforcement records so that a prospective tenant can see the complaint and violation history of a specific…","title":"Boston RentSmart housing conditions by address","updatedAt":"2026-08-16T17:40:37.581Z","visibility":"public","websiteUrl":"https://www.boston.gov/departments/analytics-team/rentsmart-boston","whiteLabel":"boston"}}