[chiajunglien]

Description

This PR introduces end-to-end (E2E) testing and validation pipelines for the GPT-OSS-20b model in MaxText.

Tests (Testing on v5p-128 cluster)

Pre-Training:

export RUN_ID=$(date +%Y-%m-%d-%H-%M)
bash tests/end_to_end/tpu/gpt-oss/20b/test_gpt_oss_to_mt.sh $RUN_ID
bash tests/end_to_end/tpu/gpt-oss/20b/test_gpt_oss.sh $RUN_ID

SFT:

export RUN_ID=$(date +%Y-%m-%d-%H-%M)
bash tests/end_to_end/tpu/gpt-oss/20b/test_gpt_oss_to_mt.sh $RUN_ID
bash tests/end_to_end/tpu/gpt-oss/20b/test_gpt_oss_sft.sh $RUN_ID

RL:

export RUN_ID=$(date +%Y-%m-%d-%H-%M)
bash tests/end_to_end/tpu/gpt-oss/20b/test_gpt_oss_to_mt.sh $RUN_ID
bash tests/end_to_end/tpu/gpt-oss/20b/test_gpt_oss_rl.sh $RUN_ID

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[gemini-code-assist]

Code Review

This pull request modularizes the GPT-OSS-20b end-to-end testing pipeline by splitting the original script into separate, targeted scripts for checkpoint conversion, pre-training/inference, RL, SFT, and Hugging Face conversion. The review feedback identifies critical issues: escaped single quotes around the dot_product attention argument will cause configuration mismatches, and defaulting run_id to a timestamp in downstream scripts will cause failures since they rely on pre-existing checkpoints. Additionally, some script comments describing a three-step flow need to be updated to accurately reflect the actual single-step execution.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!