build_release_files: predict translation sizes instead of building them on PRs - #2
Open
lynt-smitka wants to merge 27 commits into
Open
lynt-smitka wants to merge 27 commits into
lynt-smitka wants to merge 27 commits into
Conversation
The last buffer was padded by length_read % 4 bytes instead of 4 - length_read % 4, so a 5 or 7 byte tail still ended unaligned. Round caller-supplied buffer halves down to a multiple of 4 so the pad always fits, and add a unix coverage test.
C5 is the first dual-band ESP32. It differs from c6 in ways that build cleanly but fail at run time: - bootloader flash offset is 0x2000, not 0x0 - ADC1 channels start at GPIO1, not GPIO0 - efuse MAC registers are named like c61, not c6 - libphy.a references _rom_eco_version, so rom.version.ld must link - USB-Serial-JTAG pads (GPIO13/14) are cleared by gpio_ll_func_sel(), so reset_all_pins() must skip them along with the flash/PSRAM MSPI pins and VDD_SPI - boot button is GPIO28, not GPIO9 or GPIO0 - the 48MHz crystal leaves only 40/80/160/240MHz CPU valid - TWAI is the newer TWAI-FD, so canio is off pending a backend
8MB flash, 8MB quad PSRAM, WS2812B on GPIO27, UART0 on GPIO11/12. Flash mode is dio: qio locks the CPU during bootloader startup on this board. PSRAM stays qio, which is the only mode C5 offers.
The comment was copied from the c6 block. The 8MB flash has a 2048K app partition, and enabling both leaves 42 KB free.
The C5 has an I2S peripheral and the DevKitC breaks out 19 GPIOs, so external I2S hardware works the same as on the S3 DevKitC, which ships with these on and has no onboard audio either. Costs 78160 bytes, leaving 42928 free in the 2048K app partition.
Seven ports never write firmware.size.json, so build_release_files.py cannot tell whether the en_US build left room and builds all 17 languages for every board of those ports on every pull request. They cannot use this script as it stands: it reads the region size out of a linker script, which needs the size to be a literal there. raspberrypi writes LENGTH = firmware_size, a symbol the board can override; broadcom, cxd56 and silabs link a script from a submodule or an SDK; zephyr-cp has no script of its own at all. The linker map has the same numbers with the symbols already resolved, and every one of these ports either writes a map already or is one flag away from it. So read the Memory Configuration table when the file has one, and keep the linker script path for the ports that pass one. The region name is FLASH_FIRMWARE or FLASH, or --region for anything else. Wired into raspberrypi, renode, mimxrt10xx, broadcom, cxd56 and silabs, which only needed -Wl,-Map added. A missing map or an unknown region name prints what was found and writes nothing, so the language skip falls back to today's behaviour instead of failing the build. zephyr-cp still needs doing; its Makefile has none of the toolchain variables.
Its makefile has none of the toolchain variables the other ports get from circuitpy_mkenv.mk, so there is no size(1) to pipe in. There does not need to be: what the image occupies in flash is the size of the binary, which is how ports/espressif has always measured it. Pass --image and the region still comes from the map. Measured on rpi_pico: 502256 bytes used of a 1040128 byte region, against the 502000 and 1040128 Zephyr prints itself -- the difference is the 256 byte second stage, which the .bin includes and the FLASH region does not.
Rather than rounding each half of a caller-supplied buffer down to a multiple of 4, reject lengths that are not a multiple of 8 up front so the buffer the user passes is the buffer that gets used.
…padding audiocore: fix WaveFile 8-bit sample padding at end of file
espressif: add ESP32-C5 support and the ESP32-C5-DevKitC-1-N8R8 board
Neither has a flash firmware region to measure against. cxd56 links the image into RAM through the Spresense SDK script, so its map has one region and it is called ram; broadcom boots kernel.img off an SD card and its regions are RAM and READONLY. Nothing there constrains the firmware, so there is no headroom to compute and the step only printed what it could not find.
atmel-samd, analog, litex, nordic and stm were still passing their linker script, which left the script answering the same question two ways. Their maps carry the same numbers -- the script computed them, ld resolved them -- so point those five at the map as well and the linker script parser, the K and M suffix handling and the eval that needed them all come out. What a port supplies is now one rule: the map for the region, size(1) or --image for what is used.
…fallback build_release_files: read the linker memory table when size.json is missing
A display test with scheduled captures holds its last frame in `while True: time.sleep(1)`, so native_sim never exits on its own and wait_until_done() waits out the whole wall-clock timeout: every capture test takes exactly its duration, 346 s for the 26 display tests. Pass the duration as -stop_at so the simulator ends the run itself. In -no-rt the idle loop is fast-forwarded: gifio 60 s -> 2 s, the display suite 346 s -> 19 s, goldens unchanged. Realtime tests are left alone.
GitHub allows 256 configurations per matrix and build-boards.yml runs one matrix per port. espressif reached 257 boards, so every full build fails with "Strategy for job 'board' produced 257 configurations". ci_set_matrix.py now splits such a port into equal parts listed like ports (espressif-1, espressif-2) and a "split_ports" map lets build.yml hand the real port name to build-boards.yml, so the toolchain setup is unchanged. Ports below the limit and the per-board job names are not affected. Fixes adafruit#11324
Each part covered the whole alphabet, so finding a board meant looking at every part. Slice the sorted list into consecutive runs instead.
The iLabs Challenger RP2040 NFC is an RP2040 board in the Adafruit Feather form factor with 8MB of flash and an on-board NXP PN7150 NFC controller. It is the only Challenger RP2040 variant that has no CircuitPython board definition, and the vendor currently ships no CircuitPython support for it. Pin mapping is taken from the official pinout sheet (98-00241-1 V0.1) and cross-checked against iLabs' Arduino variant (arduino-pico, variants/challenger_2040_nfc/pins_arduino.h). USB VID/PID 0x2e8a/0x1036 is the pair iLabs registered for this board in that same core. The PN7150 is wired to a second I2C bus on GPIO10/11, and that bus has no external pull-up resistors: iLabs' Arduino core enables the RP2040's internal pull-ups in TwoWire::begin() instead. This board therefore defines CIRCUITPY_I2C_ALLOW_INTERNAL_PULL_UP, following the existing espressif idiom, and this adds support for that option to the rp2 port. Two details are specific to rp2. The pull-ups are enabled only after the pins are handed to the I2C peripheral, because shared_module_bitbangio_i2c_construct() runs first and leaves both pads as open-drain outputs with no pull (see scl_release() and sda_read()); gpio_set_function() deliberately does not touch pad pull settings, so anything set earlier would already be gone. And the CIRCUITPY_REQUIRE_I2C_PULLUPS timing check is compiled out when the internal pull-ups are in use: RP2xxx pull-ups are weak, and that check waits a fixed 3us regardless of frequency, so it passes even at bus speeds the internal pull-ups cannot sustain -- measured on this board, it passes 300/300 at both 100kHz and 400kHz although only 100kHz actually works. Tested on hardware: enumerates as CIRCUITPY, NeoPixel status LED on GPIO14 and LED on GPIO24 confirmed, and the on-board PN7150 answers NCI at 100kHz (CORE_RESET_RSP 40:00:03:00:11:01). NDEF was read from NTAG/Ultralight, MIFARE Classic and ISO15693 tags, and FeliCa cards were identified, through that bus. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Dan Halbert <halbert@halwitz.org>
ci: split a port with more than 256 boards into several matrix jobs
zephyr-cp tests: let a capture test stop the simulator at its duration
Add a comment regarding pull-up checks for I2C.
…040-nfc Add Challenger RP2040 NFC board
…em on PRs On pull requests the translations other than en_US are built only to prove that they still fit in flash. The compiled code is identical for every translation; only three generated data files differ: the compressed strings, the compression dictionary and the terminal font. When the first language leaves less than 10 KB of headroom, run only the generators (the make targets for translations-<lang>.c and autogen_display_resources-<lang>.c, about 4 s, no compiler), count the bytes of the generated tables and predict the flash usage. Only translations predicted within LANGUAGE_MARGIN (1 KB) of the limit, or whose build configuration differs (clean build), are really built. LANGUAGE_PREDICT=dryrun builds everything and prints the prediction error; LANGUAGE_PREDICT=off restores the old behaviour. Measured on feather_m4_can, pybadge and metro_m4_express with every language (41 predictions): error -5..+174 bytes, almost always an over-estimate. On metro_m4_express the prediction flagged fr as not fitting; the real build then overflowed by 56 bytes. feather_m4_can board job: 443 s -> 228 s; the remainder is the two clean builds. Push and release builds are unchanged.
lynt-smitka
force-pushed
the
ci-language-predict
branch
from
September 8, 2026 03:26
41a5731 to
a09cc62
Compare
prediction, predict_flash and usage were three near-identical names for
three different things, and two of them were tuples read by index. Name
them by their role instead: baseline_flash and flash_region for what the
first language's build establishes, baseline_translation_bytes and
translation_growth for the translation data, predicted_flash and
actual_flash for the two sides of the check. flash_usage() returns two
values so every caller unpacks them.
Say flash size in the messages too:
Predicted flash size for itsybitsy_m0_express es: 253527 of 253696
bytes (169 free, +1479 vs en_US) -> build
Flash size check itsybitsy_m0_express es: predicted 253527,
actual 253504, error +23
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test run in the fork, not for upstream in this shape.
Rebased onto main now that adafruit#11314 is in: flash_usage no longer falls back to the ld
table, it just reads firmware.size.json, which after adafruit#11314 exists on ten ports
instead of six.
What to read out of this run:
Predict <board> for <lang>: ... -> skip|build— the prediction and what it decidedPredict check <board> for <lang>: predicted X, actual Y, error N— only where abuild did happen, so the error can be checked against the real number
zephyr-cp is excluded: it has no make target for the translation generators and the
generated files live under circuitpython-prefix, not in the build dir.