[shourya116]

Import Name - CensusACS5YearSurvey_SubjectTables_S2702

Summary

Adds the US Census ACS 5-Year Subject Table S2702 (Selected Characteristics of the Uninsured in the United States) import configuration, documentation, and unit test fixtures, refreshed to include the 2024 ACS 5-Year release (2013–2024) across all US geographic summary levels (country down to census tracts).

Files Added

• scripts/us_census/acs5yr/subject_tables/S2702/S2702_spec.json (1,172 lines):
  • Defines preprocess.find_and_replace, pvs, measurement, populationType, ignoreTokens, ignoreColumns, denominators, and inferredSpec rules for S2702 across 2013–2024.
  • Adds 2024 inflation-adjusted household income entries (IN 2024 INFLATION ADJUSTED DOLLARS) to ignoreColumns (for Median household income of householders MOE) and denominators (for Under $25,000 to $100,000 and over household income brackets).
  • Handles the 2017+ age bracket transition (Under 18 years / 18 to 64 years in 2013–2016 vs. Under 19 years / 19 to 64 years in 2017–2024).
  • Preserves historical 2013–2023 preprocessing and ignoreColumns mappings (including the $50,000 to $74,999 rule) so all 217 StatVars and 10,624,152 historical observations match the previous 20250211 release with zero deletions or modifications.
• scripts/us_census/acs5yr/subject_tables/S2702/README.md (57 lines):
  • Documents table scope (2013–2024), data download instructions using census_api_data_downloader.py, processing instructions using process.py, and dataset notes.
• scripts/us_census/acs5yr/subject_tables/S2702/testdata/:
  • S2702_test.zip: Sample Census ACS 5-Year input data covering 2013, 2017, and 2024.
  • column_map.json, S2702_cleaned.csv, S2702_output.mcf, S2702_output.tmcf: Expected golden outputs for unit testing.

Dataset Summary (2013–2024)

Year	Columns	StatVars Generated	Unique Geos	Observations
2013	422	205	6,377	1,307,285
2014	422	205	6,153	1,261,365
2015	422	205	5,676	1,163,580
2016	422	205	5,130	1,051,650
2017	422	205	4,603	943,615
2018	422	205	4,117	843,985
2019	422	205	3,920	803,600
2020	422	207	3,882	803,574
2021	422	207	3,961	819,927
2022	422	207	3,936	814,752
2023	422	207	3,917	810,819
2024 (New)	422	207	3,909	809,163
Total	—	217 unique	—	11,433,315

Validation & Testing

• Unit Tests: Executed python3 -m unittest scripts/us_census/acs5yr/subject_tables/subject_table_test.py (Ran 1 test in 58.1s — OK).
• Import Tool (dc-import genmcf):
  • Input files: S2702_output.mcf, S2702_output.tmcf, S2702_cleaned.csv
  • NumRowSuccesses: 11,433,315 (0 fatal, 0 errors)
  • Existence_MissingReference_variableMeasured: 0
• Import Differ (tools/import_differ/import_differ.py vs. 20250211 release):
  • Previous Observations (2013–2023): 10,624,152
  • Current Observations (2013–2024): 11,433,315
  • Added Observations (2024): 809,163
  • Deleted Observations: 0
  • Modified Observations: 0
  • Added / Deleted / Modified Schemas: 0

[gemini-code-assist]

Code Review

This pull request introduces the US Census ACS5YR S2702 Subject Table import, adding a README with instructions, test data, MCF/tMCF schemas, and a column mapping configuration. The reviewer identified two issues: first, the specification file S2702_spec.json referenced in the documentation is missing from the repository; second, the --all_summaries flag used in the download command is not supported by the downloader script and should be removed.

[saanikaaa]

I don't see equivalent entry for other years like 2020, 2021 and son on. Can you pls check

[shourya116]

The equivalent entries for 2020–2024 are present right below on lines 292–296:
Margin of Error!!Total Uninsured!!EARNINGS IN THE PAST 12 MONTHS (IN <YEAR> INFLATION ADJUSTED DOLLARS)!!Civilian noninstitutionalized population 16 years and over with earnings!!Median earnings (dollars)

Why 2019 has the extra entry on line 289 (with Civilian noninstitutionalized population 16 years and over with earnings repeated twice) and line 291 (16 years and older):

• Those extra 2019 entries were carried over from the existing baseline spec (common/testdata/spec_s2702.json).
• Line 289 was the post-find_and_replace form of line 290 (since find_and_replace replaces Median earnings (dollars) with Civilian noninstitutionalized population 16 years and over with earnings!!Median earnings (dollars)).
• However, generate_col_map.py checks ignoreColumns against the raw CSV column headers before running preprocess.find_and_replace. Because the raw CSV header only has the token once, line 290 (for 2019) and lines 292–296 (for 2020–2024) are the entries that actually match and ignore the column, while lines 289 and 291 are historical no-ops.

(Also, in case this was referring to lines 308–309 for 2019 Household Income $50,000 to $74,999: those are intentionally not ignored for 2020–2024 so they continue mapping to Count_Household_16OrMoreYears_Civilian_NoHealthInsurance_IncomeOf50000To74999USDollar_WithEarnings_NonInstitutionalized for backward compatibility with the 20250211 release, as documented in README.md Note 3.)