This benchmark suite is deterministic (fixed seed for synthetic noise) and reusable across machines/commits.
julia --project=benchmark -e 'using Pkg; Pkg.instantiate()'From the repository root:
julia --project=benchmark benchmark/run_benchmarks.jl --backend=cpu --samples=20 --output=benchmark/results/cpu.csvTo try CUDA too:
julia --project=benchmark -e 'using Pkg; Pkg.add("CUDA")'
julia --project=benchmark benchmark/run_benchmarks.jl --backend=both --samples=20 --output=benchmark/results/both.csvIf CUDA is not installed/functioning (or extension is inactive), CUDA cases are skipped automatically.
--backend=cpu|cuda|both--samples=N--evals=N--seed=N--quick--output=PATH--no-output--no-gpu(alias for--backend=cpu)
Use --help for all options:
julia --project=benchmark benchmark/run_benchmarks.jl --helpFor CUDA-only comparison across batch sizes (1,4,...,1024):
julia --project=benchmark -e 'using Pkg; Pkg.add("CUDA")'
julia --project=benchmark benchmark/compare_batching_cuda.jl \
--batch-sizes=1,4,8,16,32,64,128,256,512,1024 \
--output=benchmark/results/cuda_batching_vs_no_batching.csv