Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: "3.12"
- uses: pre-commit/action@v3.0.1
Expand All @@ -30,8 +30,8 @@ jobs:
env:
AUTODEEP_DEVICE: cpu
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
cache: pip
Expand All @@ -41,7 +41,7 @@ jobs:
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install -e ".[dev,files]"
- name: Cache TabICL weights
uses: actions/cache@v4
uses: actions/cache@v5
with:
path: ~/.cache/huggingface
key: hf-tabicl-v2
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -91,4 +91,4 @@ jspm_packages/


# benchmark outputs (summarised in benchmarks/README.md)
benchmarks/results/
benchmarks/results*/
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ repos:
rev: v0.6.9
hooks:
- id: ruff
args: [--fix, --select, "E,F,W,I,B", --ignore, "E501,B008,B904,B905"]
args: [--fix, --select, "E,F,W,I,B,UP", --ignore, "E501,B008,B904,B905", --target-version, py310]
files: ^(autodeep|tests|examples)/.*\.py$
exclude: ^autodeep/modelutils/igtdutilities\.py$

Expand Down
49 changes: 49 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,54 @@
# Changelog

## 0.5.0 (2026-11-13): competition tools, LightGBM

### New
- **LightGBM** (`lgbm`, CPU, native categorical columns) with a default search space; it is part of the `fast` preset and of the `models: auto` candidates (after CatBoost).
- **Offset columns (GLM-boost).** XGBoost and LightGBM accept `default_params: {offset_column: <name>}`: that column holds log-odds from another model (e.g. an out-of-fold logistic regression), boosting starts from it (`base_margin` / `init_score`) and predictions add it back. The column is not used as a feature. Binary classification only.
- **Residual MLP** (`resmlp`, PyTorch): with an offset column the output is `offset + net(x)` with a zero-initialised head, so training starts at the offset model and learns its residual; otherwise a plain MLP. Early stopping on validation AUC (binary) or loss.
- **`search_folds: K`** in a model's `default_params`: every trial is scored on K stratified folds (mean validation metric) instead of one split. Use it when candidates differ by less than the noise of a single validation split (K times the cost).
- **`autodeep.ensemble.Stacker`**: stacks out-of-fold predictions (from AutoDeep or anywhere else) with Caruana hill climbing or an L2 logistic regression on logit-ranks, scored by nested cross-validation (`nested_score_`, `weights_`, `predict()`).

### Also
- README: a demo clip of the web app, new screenshots and a minimal Python quick start.

## 0.4.0 (2026-11-13): use the models, sensible defaults

### New
- **Saved, ready-to-use models.** Every run keeps the best model of each dataset (chosen on the validation score, never the test score) and saves it with its preprocessing to `<output>/models/<dataset>/best`. `runner.best_model("churn").predict(new_rows)`, `autodeep.load_model(path)`, `autodeep predict <model> new.csv -o predictions.csv` and `autodeep info <model>`. New rows need the original columns: missing values, unseen categories and numbers stored as text are handled like during training, and predictions use the original class labels. `predict_proba` returns one column per class; `evaluate` scores labelled rows. In the web app, each run has a "Use the best models" card to download a model or predict an uploaded file.
- `save_models=True` now saves every model in the same loadable format (`models/<dataset>/<model>`); `save_best=False` turns saving off. The leaderboard has a `selected` column.
- All 17 models can be saved and loaded. Models trained on a GPU load on machines without one (checked for XGBoost, RealMLP, TabM, the 1D-CNN, TabICL and, before the port, the pytorch-tabular models).
- **`models: auto` (the new default) with a time budget.** AutoDeep measures the machine (cores, RAM, GPU, a ~0.5 s CPU benchmark) and picks the models, trials and per-model time limit that fit `time_budget` (default 15 minutes), explaining what it skipped and why. `AutoRunner(time_budget="20m")`, `autodeep run data.csv --target y --budget 20m`, `time_budget:` in YAML, and an Auto mode in the web app. With explicit models, a budget sets `timeout_per_model`.
- `autodeep run` / `estimate` accept a data file and `--target` directly, no YAML needed. `autodeep system` shows what AutoDeep sees on the machine.
- Runtime estimates scale the built-in cost curves by the measured CPU speed until timings from the machine itself are available.

- **No more pytorch-tabular dependency; pandas 3 supported.** The nine deep tabular architectures (CategoryEmbedding, TabNet, FT-Transformer, TabTransformer, AutoInt, GANDALF, GATE, NODE, DANet) were ported from pytorch-tabular 1.2.0 (MIT) to plain PyTorch modules in `autodeep/modelsdefinition/tabular_nets/` and are trained by AutoDeep's own loop, like the 1D-CNN and the ResNet. pytorch-tabular was the only dependency pinning `pandas<3`; the lock file now resolves pandas 3.1 (2.3 on Python 3.10). `pytorch-tabular`, `requests` and the `pytorch-tabnet<4.2` cap are gone, and so are omegaconf, rich and their transitive dependencies. hyperopt 0.3 no longer needs `pkg_resources`, so the `setuptools<81` pin is gone as well (no dependency is held back any more).

### Changed
- TabICL predicts the test set with every training row as context (the validation score still comes from the 80% split), and no longer reports train metrics, which scored the context rows against themselves. Models can opt in with `refit_on_full_train` and `compute_train_metrics = False`.
- Estimates warn when a model that is much faster on a GPU (TabICL, TabM, the 1D-CNN, ResNet) would take more than 5 minutes on CPU. `autodeep estimate` and the run log now print the per-job warnings, which only the web app showed before.
- **XGBoost and CatBoost search spaces are now competitive baselines.** XGBoost used to search 50-200 trees with learning rates down to 0.01 (badly underfit at low rates, and early stopping after 100 rounds could never trigger), and CatBoost only searched `iterations` between 100 and 500. Both now train up to 2000 trees with early stopping on the validation split (50 / 100 rounds) and search learning rate, depth and regularisation (XGBoost also row/column subsampling and `gamma`). Trials take longer; GBDT scores from 0.3.0 are not comparable.
- `AutoRunner(models=None)` means `"auto"` instead of the fixed `default` preset; pass `models="default"` for the old behaviour. `max_evals` defaults to the plan's value (25 with explicit models).
- Preprocessing is a fitted `TabularPreprocessor` object (identical output to 0.3.0 on four built-in datasets × all normalisation/encoding options). Mixed-type text columns are now always treated as categorical strings, and datetime columns with missing values no longer produce huge negative numbers.
- The deep tabular models keep pytorch-tabular's hyperparameter names and defaults, so existing `param_grid`s work unchanged. Scores differ slightly from 0.3.0: binary targets use one logit with a class-balanced BCE loss (as the other PyTorch models do) instead of two-class cross-entropy, categories are encoded on the whole training set, and the batch size falls back to `default_params.batch_size`.
- AutoInt's grid now searches `deep_layers` (true/false); its `layers` / `activation` / `dropout` entries had no effect without it.
- Two pytorch-tabular options that were silently ignored now take effect: TabTransformer's `out_ff_layers` / `out_ff_activation` / `out_ff_dropout` / `out_ff_initialization` configure its MLP head, and `transformer_head_dim` sets the attention head size of TabTransformer and FT-Transformer.
- Models saved with 0.3.0's pytorch-tabular trainers cannot be loaded; retrain them. All trained PyTorch models are now saved as weights plus build arguments.
- The hyperparameter search hands over to TPE after a third of the trials (5 to 10) instead of hyperopt's fixed 20. A 25-trial search used to be 20 random trials and only 5 guided ones.
- `autodeep.modelsdefinition.pytorch_tabular_models` is kept as an alias of the new `deep_tabular_models` module.
- The scikit-learn MLP's runtime prior was re-measured (it was ~15x too optimistic above 10,000 rows).

### Also
- `benchmarks/README.md`: results of `larger_datasets.py` (TabICLv2, RealMLP, TabM, CatBoost, XGBoost on three datasets of 20k-50k rows) and measured GPU speed-ups. The estimator's GPU factors for TabICLv2 (now 50x), TabM (12x) and RealMLP (1.5x) and its XGBoost/CatBoost CPU priors (for the new grids) were updated from these runs.
- `benchmarks/larger_datasets.py` has suites (`core`, `deep`), resumes interrupted runs and stops when the GPU fills up. `benchmarks/search_diagnostics.py` reports, per search, whether TPE improved on the start-up trials, when the best trial came, and which hyperparameters sit at the edge of their range.
- README screenshots of the web app.
- `examples/walkthrough.ipynb` was executed end to end in Jupyter (nbconvert).

### Fixed
- `openml:` datasets failed on pandas 3 with scikit-learn 1.9 ("expected string or bytes-like object, got 'int'" in the ARFF parser); AutoDeep now retries with scikit-learn's `liac-arff` parser.
- Web app: model checkboxes on cards with longer descriptions were squeezed into tiny dots.
- XGBoost trained on a GPU printed a "mismatched devices" warning on every prediction.

## 0.3.0: modern tabular models

### New
Expand Down
Loading
Loading