Impossible to profile binary with exit status 1

My process exits with status 1, but I want to be able to profile its execution. codspeed exec fails in this case with:

Error: failed to execute the benchmark process: exit status: 1

It would be great to add a flag to allow non-zero exit codes, just like hyperfine does for example.

That would definitely be something we should do, thanks for the feedback!

I'd be happy to pick this up. Two quick design questions before I send a PR:

Flag name — following hyperfine's convention cited above, my default would be -i / --ignore-failure. Any
preference for a different naming?
Scope — the cmd_status != 0 bail lives in src/executor/valgrind/measure.rs, and there's a symmetric
!status.success() check in src/executor/wall_time/executor.rs. Would you like the flag to apply to both executors,
or keep it scoped to codspeed exec only?

Happy to open a draft PR first if you'd prefer to review the shape before I add tests.

feat(cli): add --ignore-failure to allow non-zero exit codes

I've opened #286 as a draft to show the shape, happy to rename the flag or narrow the scope based on your preference before marking it ready.

feat(cli): add --ignore-failure to exec command

Navigation Menu