Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: CI

on:
push:
branches: [main, dev]
pull_request:
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- uses: pre-commit/action@v3.0.1

test:
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.12"]
env:
AUTODEEP_DEVICE: cpu
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: pip
- name: Install (CPU-only PyTorch keeps the download small)
run: |
python -m pip install --upgrade pip
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install -e ".[dev,files]"
- name: Cache TabICL weights
uses: actions/cache@v4
with:
path: ~/.cache/huggingface
key: hf-tabicl-v2
- name: Tests
run: pytest -q
33 changes: 16 additions & 17 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Custom
*IGTD/
*catboost_info/

#TODO REMOVE TXT LATER
*.txt
*.tsv

*/catboost_info/
*.csv
tracked_files.txt
current_req.txt
# AutoDeep outputs
IGTD/
catboost_info/
lightning_logs/
saved_models/
output/
autodeep_workspace/
autodeep_quickstart/
autodeep_results/
logfile.log
cuda_error_log.txt
hyperopt_results_*.csv

# Compiled Python files
__pycache__/
Expand All @@ -21,13 +21,13 @@ __init__.pyc
.pt_tmp/

# Python environment
.venv/
venv/
env/
*.egg-info/
.Python
pip-log.txt
pip-delete-this-directory.txt
.python-version
# Jupyter Notebook
.ipynb_checkpoints/

Expand Down Expand Up @@ -56,11 +56,7 @@ var/
sdist/
develop-eggs/
.installed.cfg
lib/
lib64/
__pycache__/
bin/
include/
.pip_cache/
.installed.cfg

Expand Down Expand Up @@ -93,3 +89,6 @@ node_modules/
bower_components/
jspm_packages/


# benchmark outputs (summarised in benchmarks/README.md)
benchmarks/results/
53 changes: 10 additions & 43 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,49 +1,16 @@
# .pre-commit-config.yaml
repos:
# isort configuration
- repo: https://github.com/PyCQA/isort
rev: 6.0.1 # Specify the version (you can change to latest stable)
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.9
hooks:
- id: isort
name: isort
args:
- "--profile"
- "black"
exclude: "^(tests|docs|autodeep/myexamples|autodeep/modelutils)/.*" # Exclude test and docs directories
files: ^autodeep/.*\.py$ # Only check Python files
- id: ruff
args: [--fix, --select, "E,F,W,I,B", --ignore, "E501,B008,B904,B905"]
files: ^(autodeep|tests|examples)/.*\.py$
exclude: ^autodeep/modelutils/igtdutilities\.py$


# # flake8 configuration
# - repo: https://github.com/PyCQA/flake8
# rev: 7.1.2 # Specify version (can be updated)
# hooks:
# - id: flake8
# name: flake8
# args:
# - "--max-line-length=150" # Max line length for flake8 (PEP 8 compatible)
# - "--exclude=.tox,.git,.venv,build,dist" # Commonly excluded folders
# exclude: "^(tests|docs)/.*" # Exclude test and docs directories
# files: ^autodeep/.*\.py$ # Only check Python files

# pylint configuration
- repo: https://github.com/PyCQA/pylint
rev: v3.3.5 # Specify version, update if needed
hooks:
- id: pylint
name: pylint
args:
- "--max-line-length=150" # Max line length for pylint (PEP 8 compatible)
- "--disable=missing-docstring,invalid-name,import-error,missing-module-docstring,missing-class-docstring" # Disable warnings for missing docstrings and invalid names (you can customize this)
exclude: "^(tests|docs|autodeep/myexamples|autodeep/modelutils)/.*" # Exclude test and docs directories
files: ^autodeep/.*\.py$ # Only check Python files

# black configuration
- repo: https://github.com/psf/black
rev: 25.1.0 # Specify version, can update to latest stable version
rev: 24.10.0
hooks:
- id: black
name: black
args:
- "--line-length=150"
exclude: "^(tests|docs|autodeep/myexamples|autodeep/modelutils)/.*" # Exclude test and docs directories
files: ^autodeep/.*\.py$ # Only check Python files
args: [--line-length=150]
files: ^(autodeep|tests|examples)/.*\.py$
exclude: ^autodeep/modelutils/igtdutilities\.py$
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
61 changes: 61 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Changelog

## 0.3.0: modern tabular models

### New
- **TabICLv2** (`tabicl`): a pretrained tabular foundation model (BSD-3). It predicts in one forward pass with the training rows as context, so its search runs a single trial. The weights are downloaded from Hugging Face on first use.
- **RealMLP** (`realmlp`) and **TabM** (`tabm`) from `pytabkit` (Apache-2.0). The first trial uses the published tuned defaults, and later trials search around them.
- New `modern` preset (`tabicl`, `realmlp`, `tabm`). `default` now includes `tabicl` and `realmlp`, and `deep` includes `realmlp` and `tabm`.
- A search can start from a fixed point: `first_trial_defaults` runs the first option of every choice first, with `null` meaning the library default (`run_hyperopt(points_to_evaluate=...)`).
- Models can declare `max_trials` (a cap on search trials) and run with an empty `param_grid`. The runtime estimate and progress bar follow that cap.
- Models can also set their own CPU thread count and a cap on the rows used for train metrics.
- GitHub Actions CI runs lint and tests on Python 3.10 and 3.12. The README has CI, Python and licence badges.
- Runtime-estimate priors for the new models, measured on CPU. A model's epoch cap now scales against that model's own typical number of epochs.
- `benchmarks/larger_datasets.py` compares the new models with XGBoost and CatBoost on three datasets with more than 20,000 rows.

### Fixed
- The built-in `california_housing` dataset failed with "target column not found", because scikit-learn names its target `MedHouseVal`.

### Also
- `examples/walkthrough.ipynb` replaces the old example notebook, which used the pre-0.2 API and did not run.

## 0.2.0: plug-and-play release

### New
- **Web app** (`autodeep ui`): upload or select datasets, pick models, see a live runtime estimate, launch runs in the background, follow progress and ETA, stop runs, view the leaderboard and download results.
- **CLI**: `autodeep ui | run | estimate | quickstart | models | datasets | profile`, plus experiment YAML files (`AutoRunner.from_yaml`).
- **Runtime estimation**: per-model cost curves, recalibrated from timings measured on your machine, and a live ETA during runs.
- **Datasets**: `DatasetSpec` with inference of the problem type, metric and evaluation metrics. Supports CSV/TSV/Parquet/Feather/JSON/Excel, built-in datasets that work offline, `openml:` datasets, `drop_columns`, `max_rows`, and a column profile that flags IDs and high-cardinality columns.
- **Models**: DANet added; model registry with presets (`fast`, `default`, `deep`, `all`).
- **Robustness**: a failing trial or model no longer stops a run; `timeout_per_model` and `max_epochs` caps; device selection that copes with a GPU shared by other programs.
- Test suite (unit, integration and API tests, plus a `slow` test that trains every model on each problem type).

### Fixed
- **PyTorch Tabular models only worked if the target column was literally named `target`.** It is now renamed internally.
- **ResNet: the hyperparameter search optimised the *training* score.** The validation score was overwritten by the train score, and train metrics were computed on concatenated validation and train predictions.
- **ResNet and 1D-CNN "restored the best epoch" from a live reference to `state_dict()`**, so the weights actually used were from the last epoch. Now a real copy is kept.
- ResNet binary `pos_weight` was inverted (positives/negatives).
- `ReduceLROnPlateau(verbose=True)` crashes on current PyTorch (ResNet, 1D-CNN).
- PyTorch Tabular best-checkpoint reload fails on PyTorch ≥ 2.6 (`weights_only` loading). The best weights are now kept in memory instead.
- FT-Transformer and AutoInt crashed when `embed_dim` was not a multiple of `num_heads`. These are now derived from a multiplier.
- `activation: TanH` in the configs (the torch class is `Tanh`).
- TabNet needs `pytorch-tabnet`, which was not installed. It failed with `create_group_matrix() takes no arguments`.
- `hyperopt` imports `pkg_resources`, which was removed in setuptools 81, so it could not be imported in fresh environments. The dependency is now constrained.
- `pytorch-tabular` imports `requests` without declaring it.
- CatBoost picked the GPU whenever CUDA existed, and a GPU out-of-memory error aborted the whole Python process.
- Recent PyTorch (seen with 2.15) creates a CUDA context in `optimizer.step()` even for CPU training, which crashed when another process filled the GPU. That check is now skipped for CPU-only optimizers. (Hiding the GPU is not an option: XGBoost 3.x CUDA builds then fail on multiclass problems.)
- CatBoost `best_score` had the wrong sign for metrics that should be maximised.
- `split_col` stayed in the feature matrix.
- Missing values were imputed with statistics from the full dataset (test leakage). Imputation now uses the training split only.
- Metrics named in the README and examples (`f1_score`, `lift`) were silently ignored. Asking to optimise a metric not in `eval_metrics` raised a `KeyError`.
- The small-dataset config was swapped in *after* the data had been loaded with the other config's preprocessing, and the shared default IGTD dict was mutated across datasets.
- `quniform` batch-size ranges could sample a batch size of 0.
- The IGTD cache was keyed only by dataset name, so stale orderings were reused after the columns changed. The cache is now keyed by the column set. IGTD now computes only the ordering it uses (previously two, at 1,000,000 steps each).
- Every trial's model was kept in memory. Now only the best one is kept.
- Runs littered the working directory with `logfile.log`, `catboost_info/`, `lightning_logs/`, `.pt_tmp/`, `cuda_error_log.txt` and `hyperopt_results_*.csv`. Everything now goes to `output_folder`.
- Results were written as Python reprs containing `<class ...>`. They are now JSON.

### Removed
- Dataset-specific loaders with hard-coded local paths (`/home/...`), which could not work on another machine. They are replaced by built-in datasets and `openml:`.
- Dead code: a duplicate IGTD loader, Keras IGTD models, old search-space builders, and seven copy-pasted PyTorch Tabular wrappers (now one class).
- Unused dependencies: `keras`, `ipython`, `seaborn`, `Scripts`. Exact pins replaced by tested version ranges plus `uv.lock`.
Loading
Loading