Skip to content

Support emitting a generated function - #786

Open
vchuravy wants to merge 1 commit into
mainfrom
vc/generated-main
Open

Support emitting a generated function#786
vchuravy wants to merge 1 commit into
mainfrom
vc/generated-main

Conversation

@vchuravy

@vchuravy vchuravy commented Sep 15, 2026

Copy link
Copy Markdown
Member

Port of #667 (which targets release-0.9) to main for 0.10.

Adds a generated=true configuration to @kernel. The transformed kernel body is quoted and run through macroexpand, so $ interpolation is available inside the kernel and where-parameters are bound to their values. This makes it possible to write e.g.

@kernel generated = true function kernel!(a, ::Val{N}) where {N}
    @unroll $N for i in 1:5
        @inbounds a[i] = i * $N
    end
end

since @unroll N requires a literal count.

Differences from #667:

  • main only emits the GPU function, so only that path is wrapped.
  • __kernel now takes __module__ alongside __source__.
  • The @kernel docstring documents the new option.
  • The test lives in test/unroll.jl and actually launches the kernel and checks the result (the test added in support emitting a generated function #667 called an undefined test_generated_kernel).

Locally verified on the CPU (POCL) backend: the unroll, linenumbers and CPU codegen testsuites pass, and @device_code_llvm on a generated kernel works.

🤖 Generated with Claude Code

@vchuravy
Support emitting a generated function
425aa9a 
Add a `generated=true` configuration to `@kernel`. The transformed kernel
body is quoted and expanded via `macroexpand`, so `$` interpolation is
available and `where`-parameters are bound to their values. This makes it
possible to e.g. unroll a loop `$N` times with `@unroll $N for ...`, which
requires a literal count.

Port of #667 (release-0.9) to main / 0.10.

Assisted-by: Claude Code (Fable 5.1)
@github-actions

github-actions Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Show table
main 425aa9a... main / 425aa9a...
const/@Const/Float32/262144 0.365 ± 0.02 ms 0.365 ± 0.02 ms 1 ± 0.078
const/@Const/Float32/65536 0.136 ± 0.021 ms 0.134 ± 0.024 ms 1.02 ± 0.24
const/@Const/Float64/262144 0.444 ± 0.03 ms 0.45 ± 0.028 ms 0.987 ± 0.09
const/@Const/Float64/65536 0.163 ± 0.014 ms 0.156 ± 0.013 ms 1.05 ± 0.12
const/unmarked/Float32/262144 1.28 ± 0.03 ms 1.27 ± 0.027 ms 1 ± 0.032
const/unmarked/Float32/65536 0.354 ± 0.033 ms 0.351 ± 0.03 ms 1.01 ± 0.13
const/unmarked/Float64/262144 1.45 ± 0.041 ms 1.45 ± 0.039 ms 1 ± 0.039
const/unmarked/Float64/65536 0.416 ± 0.021 ms 0.418 ± 0.02 ms 0.996 ± 0.068
launch/3D static workgroup, dynamic ndrange 0.0467 ± 0.032 ms 0.0487 ± 0.033 ms 0.959 ± 0.93
launch/3D static workgroup, static ndrange 0.048 ± 0.032 ms 0.0415 ± 0.033 ms 1.16 ± 1.2
launch/dynamic workgroup, dynamic ndrange 0.0478 ± 0.033 ms 0.0487 ± 0.033 ms 0.983 ± 0.95
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 0.0464 ± 0.031 ms 0.0423 ± 0.033 ms 1.1 ± 1.1
launch/static workgroup, dynamic ndrange 0.0464 ± 0.032 ms 0.0467 ± 0.031 ms 0.993 ± 0.95
launch/static workgroup, static ndrange 0.0451 ± 0.032 ms 0.0415 ± 0.032 ms 1.09 ± 1.2
partition/dynamic workgroup, dynamic ndrange 0.063 ± 0.003 μs 0.0591 ± 0.0025 μs 1.07 ± 0.068
partition/static workgroup, dynamic ndrange 0.0694 ± 0.014 μs 0.0696 ± 0.014 μs 0.998 ± 0.29
partition/static workgroup, static ndrange 1.74 ± 0.001 ns 1.74 ± 0.001 ns 1 ± 0.00081
saxpy/default/Float16/1024 0.0571 ± 0.029 ms 0.0529 ± 0.03 ms 1.08 ± 0.82
saxpy/default/Float16/1048576 0.874 ± 0.031 ms 0.868 ± 0.032 ms 1.01 ± 0.051
saxpy/default/Float16/16384 0.0753 ± 0.03 ms 0.0727 ± 0.03 ms 1.04 ± 0.6
saxpy/default/Float16/2048 0.0565 ± 0.033 ms 0.0567 ± 0.032 ms 0.996 ± 0.8
saxpy/default/Float16/256 0.0531 ± 0.034 ms 0.0552 ± 0.031 ms 0.962 ± 0.81
saxpy/default/Float16/262144 0.263 ± 0.032 ms 0.259 ± 0.032 ms 1.02 ± 0.18
saxpy/default/Float16/32768 0.0672 ± 0.029 ms 0.0678 ± 0.03 ms 0.991 ± 0.61
saxpy/default/Float16/4096 0.0585 ± 0.025 ms 0.0574 ± 0.025 ms 1.02 ± 0.61
saxpy/default/Float16/512 0.0518 ± 0.034 ms 0.057 ± 0.03 ms 0.907 ± 0.76
saxpy/default/Float16/64 0.0538 ± 0.032 ms 0.0532 ± 0.032 ms 1.01 ± 0.86
saxpy/default/Float16/65536 0.102 ± 0.03 ms 0.0985 ± 0.031 ms 1.04 ± 0.45
saxpy/default/Float32/1024 0.0589 ± 0.029 ms 0.0527 ± 0.032 ms 1.12 ± 0.88
saxpy/default/Float32/1048576 0.507 ± 0.025 ms 0.5 ± 0.019 ms 1.01 ± 0.063
saxpy/default/Float32/16384 0.0493 ± 0.03 ms 0.0479 ± 0.029 ms 1.03 ± 0.89
saxpy/default/Float32/2048 0.0534 ± 0.028 ms 0.0533 ± 0.027 ms 1 ± 0.72
saxpy/default/Float32/256 0.0537 ± 0.033 ms 0.0517 ± 0.033 ms 1.04 ± 0.92
saxpy/default/Float32/262144 0.18 ± 0.021 ms 0.176 ± 0.021 ms 1.02 ± 0.17
saxpy/default/Float32/32768 0.0588 ± 0.03 ms 0.0547 ± 0.03 ms 1.07 ± 0.81
saxpy/default/Float32/4096 0.0562 ± 0.03 ms 0.056 ± 0.029 ms 1 ± 0.74
saxpy/default/Float32/512 0.0557 ± 0.03 ms 0.0586 ± 0.03 ms 0.95 ± 0.71
saxpy/default/Float32/64 0.0533 ± 0.034 ms 0.0541 ± 0.032 ms 0.984 ± 0.85
saxpy/default/Float32/65536 0.0858 ± 0.03 ms 0.0757 ± 0.03 ms 1.13 ± 0.59
saxpy/default/Float64/1024 0.0525 ± 0.032 ms 0.0538 ± 0.03 ms 0.975 ± 0.8
saxpy/default/Float64/1048576 0.556 ± 0.1 ms 0.57 ± 0.19 ms 0.975 ± 0.36
saxpy/default/Float64/16384 0.063 ± 0.03 ms 0.0591 ± 0.03 ms 1.07 ± 0.75
saxpy/default/Float64/2048 0.0567 ± 0.028 ms 0.0548 ± 0.027 ms 1.03 ± 0.72
saxpy/default/Float64/256 0.0501 ± 0.033 ms 0.0522 ± 0.033 ms 0.958 ± 0.88
saxpy/default/Float64/262144 0.177 ± 0.019 ms 0.175 ± 0.018 ms 1.01 ± 0.15
saxpy/default/Float64/32768 0.0654 ± 0.03 ms 0.0726 ± 0.03 ms 0.901 ± 0.56
saxpy/default/Float64/4096 0.0565 ± 0.029 ms 0.0563 ± 0.029 ms 1 ± 0.73
saxpy/default/Float64/512 0.0536 ± 0.032 ms 0.047 ± 0.034 ms 1.14 ± 1.1
saxpy/default/Float64/64 0.0539 ± 0.033 ms 0.0531 ± 0.032 ms 1.02 ± 0.88
saxpy/default/Float64/65536 0.0913 ± 0.027 ms 0.0897 ± 0.027 ms 1.02 ± 0.43
saxpy/static workgroup=(1024,)/Float16/1024 0.0504 ± 0.03 ms 0.0547 ± 0.028 ms 0.922 ± 0.72
saxpy/static workgroup=(1024,)/Float16/1048576 0.869 ± 0.03 ms 0.865 ± 0.03 ms 1 ± 0.049
saxpy/static workgroup=(1024,)/Float16/16384 0.0728 ± 0.03 ms 0.0707 ± 0.03 ms 1.03 ± 0.61
saxpy/static workgroup=(1024,)/Float16/2048 0.0551 ± 0.032 ms 0.0543 ± 0.031 ms 1.01 ± 0.82
saxpy/static workgroup=(1024,)/Float16/256 0.0501 ± 0.032 ms 0.0516 ± 0.029 ms 0.97 ± 0.84
saxpy/static workgroup=(1024,)/Float16/262144 0.261 ± 0.032 ms 0.257 ± 0.031 ms 1.01 ± 0.17
saxpy/static workgroup=(1024,)/Float16/32768 0.0669 ± 0.03 ms 0.0648 ± 0.029 ms 1.03 ± 0.65
saxpy/static workgroup=(1024,)/Float16/4096 0.054 ± 0.024 ms 0.0541 ± 0.024 ms 0.998 ± 0.63
saxpy/static workgroup=(1024,)/Float16/512 0.0522 ± 0.031 ms 0.054 ± 0.03 ms 0.967 ± 0.78
saxpy/static workgroup=(1024,)/Float16/64 0.0539 ± 0.029 ms 0.0525 ± 0.03 ms 1.03 ± 0.8
saxpy/static workgroup=(1024,)/Float16/65536 0.102 ± 0.03 ms 0.0971 ± 0.03 ms 1.05 ± 0.45
saxpy/static workgroup=(1024,)/Float32/1024 0.0501 ± 0.033 ms 0.0535 ± 0.03 ms 0.937 ± 0.82
saxpy/static workgroup=(1024,)/Float32/1048576 0.502 ± 0.033 ms 0.502 ± 0.032 ms 1 ± 0.091
saxpy/static workgroup=(1024,)/Float32/16384 0.0466 ± 0.027 ms 0.0465 ± 0.028 ms 1 ± 0.84
saxpy/static workgroup=(1024,)/Float32/2048 0.0508 ± 0.027 ms 0.0522 ± 0.026 ms 0.974 ± 0.71
saxpy/static workgroup=(1024,)/Float32/256 0.0536 ± 0.03 ms 0.0515 ± 0.029 ms 1.04 ± 0.82
saxpy/static workgroup=(1024,)/Float32/262144 0.175 ± 0.022 ms 0.174 ± 0.022 ms 1.01 ± 0.18
saxpy/static workgroup=(1024,)/Float32/32768 0.0584 ± 0.03 ms 0.054 ± 0.029 ms 1.08 ± 0.8
saxpy/static workgroup=(1024,)/Float32/4096 0.0553 ± 0.029 ms 0.0539 ± 0.028 ms 1.03 ± 0.75
saxpy/static workgroup=(1024,)/Float32/512 0.0521 ± 0.031 ms 0.0535 ± 0.03 ms 0.974 ± 0.79
saxpy/static workgroup=(1024,)/Float32/64 0.0529 ± 0.029 ms 0.0534 ± 0.029 ms 0.99 ± 0.76
saxpy/static workgroup=(1024,)/Float32/65536 0.083 ± 0.029 ms 0.0719 ± 0.029 ms 1.16 ± 0.61
saxpy/static workgroup=(1024,)/Float64/1024 0.0557 ± 0.028 ms 0.0508 ± 0.031 ms 1.09 ± 0.87
saxpy/static workgroup=(1024,)/Float64/1048576 0.556 ± 0.13 ms 0.514 ± 0.16 ms 1.08 ± 0.42
saxpy/static workgroup=(1024,)/Float64/16384 0.0496 ± 0.029 ms 0.0482 ± 0.027 ms 1.03 ± 0.84
saxpy/static workgroup=(1024,)/Float64/2048 0.0537 ± 0.027 ms 0.0528 ± 0.026 ms 1.02 ± 0.72
saxpy/static workgroup=(1024,)/Float64/256 0.0576 ± 0.027 ms 0.0489 ± 0.03 ms 1.18 ± 0.91
saxpy/static workgroup=(1024,)/Float64/262144 0.177 ± 0.022 ms 0.174 ± 0.018 ms 1.02 ± 0.16
saxpy/static workgroup=(1024,)/Float64/32768 0.0689 ± 0.029 ms 0.0617 ± 0.029 ms 1.12 ± 0.71
saxpy/static workgroup=(1024,)/Float64/4096 0.0566 ± 0.028 ms 0.0554 ± 0.027 ms 1.02 ± 0.71
saxpy/static workgroup=(1024,)/Float64/512 0.0579 ± 0.029 ms 0.0487 ± 0.032 ms 1.19 ± 0.99
saxpy/static workgroup=(1024,)/Float64/64 0.0539 ± 0.029 ms 0.052 ± 0.029 ms 1.04 ± 0.81
saxpy/static workgroup=(1024,)/Float64/65536 0.0891 ± 0.027 ms 0.088 ± 0.028 ms 1.01 ± 0.44
time_to_load 0.818 ± 0.024 s 0.852 ± 0.0023 s 0.961 ± 0.029
main 425aa9a... main / 425aa9a...
const/@Const/Float32/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/@Const/Float32/65536 1 allocs: 16 B 1 allocs: 16 B 1
const/@Const/Float64/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/@Const/Float64/65536 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float32/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float32/65536 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float64/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float64/65536 1 allocs: 16 B 1 allocs: 16 B 1
launch/3D static workgroup, dynamic ndrange 5 allocs: 0.141 kB 5 allocs: 0.141 kB 1
launch/3D static workgroup, static ndrange 6 allocs: 0.172 kB 6 allocs: 0.172 kB 1
launch/dynamic workgroup, dynamic ndrange 15 allocs: 0.312 kB 15 allocs: 0.312 kB 1
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 1 allocs: 16 B 1 allocs: 16 B 1
launch/static workgroup, dynamic ndrange 1 allocs: 16 B 1 allocs: 16 B 1
launch/static workgroup, static ndrange 2 allocs: 0.0469 kB 2 allocs: 0.0469 kB 1
partition/dynamic workgroup, dynamic ndrange 2 allocs: 0.0625 kB 2 allocs: 0.0625 kB 1
partition/static workgroup, dynamic ndrange 2 allocs: 32 B 2 allocs: 32 B 1
partition/static workgroup, static ndrange 0 allocs: 0 B 0 allocs: 0 B
saxpy/default/Float16/1024 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/1048576 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/16384 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/2048 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/256 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float16/262144 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/32768 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/4096 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/512 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/64 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float16/65536 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/1024 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/1048576 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/16384 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/2048 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/256 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float32/262144 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/32768 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/4096 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/512 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/64 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float32/65536 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/1024 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/1048576 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/16384 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/2048 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/256 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float64/262144 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/32768 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/4096 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/512 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/64 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float64/65536 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/static workgroup=(1024,)/Float16/1024 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/1048576 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/16384 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/2048 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/256 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float16/262144 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/32768 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/4096 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/512 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/64 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float16/65536 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/1024 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/1048576 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/16384 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/2048 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/256 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float32/262144 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/32768 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/4096 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/512 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/64 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float32/65536 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/1024 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/1048576 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/16384 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/2048 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/256 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float64/262144 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/32768 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/4096 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/512 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/64 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float64/65536 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
time_to_load 0.201 k allocs: 11.8 kB 0.201 k allocs: 11.8 kB 1

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vchuravy