Skip to content

Constant and near-constant features in actg, physionet_challenge_2012 and diabetes #140

Open

Description

@valterschutz

Problem

After dataset generation, these features hold one value for every instance, or almost every one:

  • actg feature zprior (ZDV before the trial): 1 for all 2,139 patients, so 0 after z-normalization.
  • physionet_challenge_2012 feature MechVent: the Challenge records it only as 1, so a stay's mean is 1 or missing, and mean-filling makes it 1 everywhere. Whether ventilation was recorded is informative (16.6% vs 10.2% in-hospital deaths), but the fill discards it (Mean imputation in TabularCSVDataset discards informative missingness and averages category codes #142).
  • diabetes columns 40 and 41 (one-hot smoking categories): 1 for 18 and 40 of 92,062 instances.

A selection of such a feature costs budget and reveals nothing (or, for the rare one-hots, almost nothing). Methods may learn to avoid them, but they inflate the feature count and make budgets less comparable across datasets.

Scope

  • Dataset generation for actg, physionet_challenge_2012 and diabetes; both budget settings.
  • Changes paper results if features are dropped: actg and physionet_challenge_2012 are in the kdd26 dataset list.

Options to decide

Acceptance criteria

  • No dataset has a constant feature, or its page says why it keeps one.
  • just qa passes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageMaintainer needs to evaluate this issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions