You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Rebuild diabetes from NHANES, and decide what to do about its imputed labels #137
data/diabetes/diabetes.csv is the GDFS repository's copy of Kachuee et al.'s NHANES 1999–2016 preprocessing (load_diabetes in mkachuee/Opportunistic/nhanes.py); nothing in AFABench rebuilds it (docs/reference/datasets/diabetes.md).
Two problems:
Labels are mostly imputed. The loader fills a missing fasting glucose (LBXGLU) with the mean before thresholding it at 100 and 125 mg/dL (preproc_impute), so every participant without a measurement gets the mean's class. 73,573 of the 92,062 instances (80%) are class 1 (prediabetes), which suggests most class 1 labels are imputed. The file does not record which.
Not reproducible or traceable. The loader shuffles with an unseeded permutation and the file has no participant ID (SEQN) and unnamed columns 0–44; Kachuee's preprocessed diabetes.pkl link is dead (Google Drive 404).
Decide whether to keep the dataset as is, drop it, or port the loader into DiabetesDataset.make_prepared_source (from the NHANES XPT files, keeping SEQN order, naming the columns, and dropping participants without a fasting glucose measurement instead of imputing it). Rebuilding would make a new dataset; results on the current one would not carry over.
Activity