Skip to content

Add model training, export, and read-only inference - #2

Open
Kaushik-Kumar-CEG wants to merge 5 commits into
gsoc/package-foundationfrom
gsoc/training-pipeline
Open

Add model training, export, and read-only inference#2
Kaushik-Kumar-CEG wants to merge 5 commits into
gsoc/package-foundationfrom
gsoc/training-pipeline

Conversation

@Kaushik-Kumar-CEG

@Kaushik-Kumar-CEG Kaushik-Kumar-CEG commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Adds model training, export, and read only inference to the package

This includes:

  • DeBERTa BIOES/CRF training from the generated dataset
  • best-checkpoint selection using validation F1
  • strict dataset, model-state, and artifact validation
  • offline reload before publishing a final model
  • CRF and optional ONNX export
  • read-only phrase prediction without changing ScanCode rules or files

Final run results:

  • validation F1: 0.9008
  • test F1: 0.8766
  • test precision: 0.8748
  • test recall: 0.8784
  • test exact match: 0.8197
  • invalid BIOES paths: 0

Predictions are candidates and still require human review. The uploaded model was downloaded and validated again successfully

The ScanCode Toolkit version is in aboutcode-org/scancode-toolkit#5222

Package the DeBERTa BIOES and CRF training pipeline with strict dataset and checkpoint validation. Add ONNX export, focused tests, documentation, and installed command entry points.

Signed-off-by: Kaushik Kumar <kaushikrjpm10@gmail.com>
Signed-off-by: Kaushik <kaushikrjpm10@gmail.com>
Signed-off-by: Kaushik <kaushikrjpm10@gmail.com>
Signed-off-by: Kaushik <kaushikrjpm10@gmail.com>
@Kaushik-Kumar-CEG Kaushik-Kumar-CEG changed the title Add model training and export commands Add model training, export, and read-only inference Sep 7, 2026
Signed-off-by: Kaushik <kaushikrjpm10@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant