Optimize SIMD impl of `lines_fwd` and `lines_bwd` by heiher · Pull Request #535 · microsoft/edit

hev (heiher) · 2025-06-26T15:17:39Z

Benchmark results on AMD Zen4:

 simd/lines_fwd/1        time:   [1.4801 ns 1.4803 ns 1.4804 ns]
                         thrpt:  [644.19 MiB/s 644.26 MiB/s 644.34 MiB/s]
                  change:
                         time:   [−38.958% −38.933% −38.898%] (p = 0.00 < 0.05)
                         thrpt:  [+63.661% +63.755% +63.821%]
                         Performance has improved.

 simd/lines_fwd/8        time:   [3.8571 ns 3.8607 ns 3.8641 ns]
                         thrpt:  [1.9282 GiB/s 1.9299 GiB/s 1.9316 GiB/s]
                  change:
                         time:   [−19.458% −19.423% −19.394%] (p = 0.00 < 0.05)
                         thrpt:  [+24.060% +24.105% +24.159%]
                         Performance has improved.

 simd/lines_fwd/128      time:   [14.064 ns 14.092 ns 14.120 ns]
                         thrpt:  [8.4426 GiB/s 8.4592 GiB/s 8.4764 GiB/s]
                  change:
                         time:   [−6.0955% −5.8911% −5.6706%] (p = 0.00 < 0.05)
                         thrpt:  [+6.0115% +6.2599% +6.4912%]
                         Performance has improved.

 simd/lines_fwd/1024     time:   [18.160 ns 18.178 ns 18.195 ns]
                         thrpt:  [52.415 GiB/s 52.462 GiB/s 52.516 GiB/s]
                  change:
                         time:   [−4.1174% −3.9973% −3.8859%] (p = 0.00 < 0.05)
                         thrpt:  [+4.0430% +4.1637% +4.2942%]
                         Performance has improved.

 simd/lines_fwd/131072   time:   [871.09 ns 871.14 ns 871.21 ns]
                         thrpt:  [140.12 GiB/s 140.13 GiB/s 140.14 GiB/s]
                  change:
                         time:   [−10.451% −10.405% −10.358%] (p = 0.00 < 0.05)
                         thrpt:  [+11.555% +11.613% +11.670%]
                         Performance has improved.

 simd/lines_fwd/134217728
                         time:   [1.7326 ms 1.7332 ms 1.7338 ms]
                         thrpt:  [72.094 GiB/s 72.120 GiB/s 72.146 GiB/s]
                  change:
                         time:   [−0.4091% −0.2348% −0.0670%] (p = 0.00 < 0.05)
                         thrpt:  [+0.0671% +0.2353% +0.4108%]
                         Change within noise threshold.

hev (heiher) · 2025-06-26T15:23:04Z

@microsoft-github-policy-service agree company="Loongson"

Leonard Hecker (lhecker) · 2025-06-26T15:27:21Z

+        let off = beg.align_offset(32);
+        if off != 0 && off < remaining {
+            (beg, line) = lines_fwd_fallback(beg, beg.add(off), line, line_stop);
+            remaining -= off;
+        }


We can use the SIMD code to do this instead of doing a byte-wise search here. This will hurt the throughput of the lines_fwd/1 case, but that doesn't matter much, because the latency is much more important (which should be within a few ns). Basically, modify the SIMD loop to this C pseudo-code:

beg = (beg + 128) & ~31; remaining = /* some equivalent of this, but I haven't though it through */

I'm curious what difference this would make.

I see what you mean. In the first iteration, we scan 128 bytes, and beg moves forward to the next 32-byte aligned point. However, the actual distance covered may be less than 128 bytes. The tricky part here is figuring out how to discard the unnecessary entries in line_next.

For the leading segment smaller than 16 or 31 bytes, the byte-wise search is more latency-friendly.

Benchmark results on AMD Zen4: ``` simd/lines_fwd/1 time: [1.4801 ns 1.4803 ns 1.4804 ns] thrpt: [644.19 MiB/s 644.26 MiB/s 644.34 MiB/s] change: time: [−38.958% −38.933% −38.898%] (p = 0.00 < 0.05) thrpt: [+63.661% +63.755% +63.821%] Performance has improved. simd/lines_fwd/8 time: [3.8571 ns 3.8607 ns 3.8641 ns] thrpt: [1.9282 GiB/s 1.9299 GiB/s 1.9316 GiB/s] change: time: [−19.458% −19.423% −19.394%] (p = 0.00 < 0.05) thrpt: [+24.060% +24.105% +24.159%] Performance has improved. simd/lines_fwd/128 time: [14.064 ns 14.092 ns 14.120 ns] thrpt: [8.4426 GiB/s 8.4592 GiB/s 8.4764 GiB/s] change: time: [−6.0955% −5.8911% −5.6706%] (p = 0.00 < 0.05) thrpt: [+6.0115% +6.2599% +6.4912%] Performance has improved. simd/lines_fwd/1024 time: [18.160 ns 18.178 ns 18.195 ns] thrpt: [52.415 GiB/s 52.462 GiB/s 52.516 GiB/s] change: time: [−4.1174% −3.9973% −3.8859%] (p = 0.00 < 0.05) thrpt: [+4.0430% +4.1637% +4.2942%] Performance has improved. simd/lines_fwd/131072 time: [871.09 ns 871.14 ns 871.21 ns] thrpt: [140.12 GiB/s 140.13 GiB/s 140.14 GiB/s] change: time: [−10.451% −10.405% −10.358%] (p = 0.00 < 0.05) thrpt: [+11.555% +11.613% +11.670%] Performance has improved. simd/lines_fwd/134217728 time: [1.7326 ms 1.7332 ms 1.7338 ms] thrpt: [72.094 GiB/s 72.120 GiB/s 72.146 GiB/s] change: time: [−0.4091% −0.2348% −0.0670%] (p = 0.00 < 0.05) thrpt: [+0.0671% +0.2353% +0.4108%] Change within noise threshold. ```

hev (heiher) · 2025-06-26T16:38:47Z

Benchmark results on Intel i7-7700k (e20e006):

simd/lines_fwd/1        time:   [2.9242 ns 2.9293 ns 2.9350 ns]
                        thrpt:  [324.93 MiB/s 325.56 MiB/s 326.13 MiB/s]
                 change:
                        time:   [−39.147% −38.969% −38.816%] (p = 0.00 < 0.05)
                        thrpt:  [+63.443% +63.850% +64.331%]
                        Performance has improved.

simd/lines_fwd/8        time:   [7.7873 ns 7.7982 ns 7.8092 ns]
                        thrpt:  [976.98 MiB/s 978.35 MiB/s 979.72 MiB/s]
                 change:
                        time:   [−17.301% −17.160% −17.018%] (p = 0.00 < 0.05)
                        thrpt:  [+20.508% +20.714% +20.921%]
                        Performance has improved.

simd/lines_fwd/128      time:   [27.227 ns 27.327 ns 27.488 ns]
                        thrpt:  [4.3368 GiB/s 4.3623 GiB/s 4.3783 GiB/s]
                 change:
                        time:   [−4.5703% −4.0676% −3.4233%] (p = 0.00 < 0.05)
                        thrpt:  [+3.5446% +4.2401% +4.7892%]
                        Performance has improved.

simd/lines_fwd/1024     time:   [38.485 ns 38.561 ns 38.646 ns]
                        thrpt:  [24.677 GiB/s 24.732 GiB/s 24.780 GiB/s]
                 change:
                        time:   [−5.3727% −4.8970% −4.2502%] (p = 0.00 < 0.05)
                        thrpt:  [+4.4389% +5.1492% +5.6777%]
                        Performance has improved.

simd/lines_fwd/131072   time:   [1.8623 µs 1.8700 µs 1.8804 µs]
                        thrpt:  [64.917 GiB/s 65.279 GiB/s 65.547 GiB/s]
                 change:
                        time:   [−14.964% −14.690% −14.387%] (p = 0.00 < 0.05)
                        thrpt:  [+16.804% +17.220% +17.598%]
                        Performance has improved.

simd/lines_fwd/134217728
                        time:   [5.6005 ms 5.6123 ms 5.6264 ms]
                        thrpt:  [22.217 GiB/s 22.272 GiB/s 22.319 GiB/s]
                 change:
                        time:   [−3.4977% −3.2437% −2.9432%] (p = 0.00 < 0.05)
                        thrpt:  [+3.0324% +3.3524% +3.6244%]
                        Performance has improved.

Leonard Hecker (lhecker) · 2025-07-02T14:39:55Z

+        let off = beg.align_offset(32);
+        if off != 0 && off < end.offset_from_unsigned(beg) {
+            (beg, line) = lines_fwd_fallback(beg, beg.add(off), line, line_stop);
+        }


I've benchmarked this locally and I have to say that I don't understand why and how llvm optimizes the code here. Even something as benign as inlining this call and doing nothing otherwise breaks the performance uplift of this PR in realistic scenarios (tested via cargo bench TextBuffer). I suspect that this code is somewhat brittle and may break at any point in the future.

It seems that in cargo bench TextBuffer, the proportions of lines_fwd and lines_bwd are too small, which might explain why the improvements appear to have no visible effect. Below is the perf sampling data:

37.16% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] _ZN4edit7unicode11measurement17MeasurementConfig15measure_forward17hbf272b50487f6abfE.llvm.14076153525412774890 18.54% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] criterion::stats::univariate::kde::Kde<A,K>::estimate 6.82% lib-e65f69f9e00 libm.so.6 [.] exp 3.26% lib-e65f69f9e00 libc.so.6 [.] 0x00000000001684d3 2.15% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] _ZN4edit6buffer10TextBuffer10edit_begin17h671f29e926b8daa4E.llvm.4491080591852468380 1.94% lib-e65f69f9e00 libc.so.6 [.] 0x00000000001685dd 1.88% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] _ZN4edit6buffer10TextBuffer31cursor_move_to_logical_internal17hf5aa8894874da6fcE.llvm.4491080591852468380 >1.78% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] _ZN4edit4simd9lines_bwd14lines_bwd_avx217h133dca1f4a656c38E.llvm.8779832433927300399 1.60% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] alloc::raw_vec::RawVecInner<A>::grow_amortized 1.52% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] rayon::slice::quicksort::shift_tail 1.43% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] edit::buffer::TextBuffer::edit_write 1.20% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] _ZN4edit6buffer10TextBuffer5write17h1eec3d2968088c69E.llvm.4491080591852468380 1.14% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] rayon::slice::quicksort::partition_in_blocks 0.98% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] _ZN4edit6buffer10TextBuffer26cursor_move_delta_internal17h997986d1c8de8030E.llvm.4491080591852468380 0.93% lib-e65f69f9e00 libc.so.6 [.] cfree 0.86% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] <edit::buffer::gap_buffer::GapBuffer as edit::document::ReadableDocument>::read_forward 0.82% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] _ZN4edit6buffer10TextBuffer30cursor_move_to_offset_internal17h1858e2b540126b9cE.llvm.4491080591852468380 0.81% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] oorandom::Rand64::rand_range 0.75% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] alloc::raw_vec::RawVecInner<A>::deallocate 0.71% lib-e65f69f9e00 libc.so.6 [.] 0x000000000016858b >0.69% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] _ZN4edit4simd9lines_fwd14lines_fwd_avx217h76579f1bcf554c93E.llvm.6941329016930144062 0.69% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] <alloc::collections::linked_list::LinkedList<T,A> as core::ops::drop::Drop>::drop 0.62% lib-e65f69f9e00 libc.so.6 [.] 0x00000000001685a0 0.58% lib-e65f69f9e00 libc.so.6 [.] 0x00000000001685a8 0.57% lib-e65f69f9e00 lib-e65f69f9e0071761 [.] criterion::bencher::Bencher<M>::iter ...

It actually does have a reproducible impact of about 3% for me.

It is however correct that measure_forward is not well optimized yet. (No ASCII fast-pass for instance. It should also be split up into separate word-wrap "layout" and regular text measurement functions.)

...on the other hand, it only takes 8ms to run through the entire set of changes. I suspect that this already qualifies this editor as being among the fastest. 😄 For this reason I'm currently focusing on more important areas, such as syntax highlighting.

Leonard Hecker (lhecker) reviewed Jun 26, 2025

View reviewed changes

Leonard Hecker (lhecker) approved these changes Jul 2, 2025

View reviewed changes

Leonard Hecker (lhecker) enabled auto-merge (squash) July 2, 2025 14:40

Leonard Hecker (lhecker) merged commit d27ba98 into microsoft:main Jul 2, 2025

hev (heiher) deleted the opt-simd-lines branch July 2, 2025 16:02

Navigation Menu

Repository navigation

Optimize SIMD impl of `lines_fwd` and `lines_bwd` - #535

Optimize SIMD impl of `lines_fwd` and `lines_bwd`#535
Leonard Hecker (lhecker) merged 1 commit into
microsoft:mainfrom
heiher:opt-simd-lines

hev (heiher) commented Jun 26, 2025 •

edited

Loading

Uh oh!

hev (heiher) commented Jun 26, 2025

Uh oh!

Leonard Hecker (lhecker) Jun 26, 2025

Uh oh!

hev (heiher) Jun 26, 2025

Uh oh!

hev (heiher) Jun 26, 2025

Uh oh!

hev (heiher) commented Jun 26, 2025

Uh oh!

Leonard Hecker (lhecker) Jul 2, 2025 •

edited

Loading

Uh oh!

hev (heiher) Jul 2, 2025 •

edited

Loading

Uh oh!

Leonard Hecker (lhecker) Jul 2, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

hev (heiher) commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hev (heiher) commented Jun 26, 2025

Uh oh!

Leonard Hecker (lhecker) Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

hev (heiher) Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

hev (heiher) Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

hev (heiher) commented Jun 26, 2025

Uh oh!

Leonard Hecker (lhecker) Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hev (heiher) Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Leonard Hecker (lhecker) Jul 2, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hev (heiher) commented Jun 26, 2025 •

edited

Loading

Leonard Hecker (lhecker) Jul 2, 2025 •

edited

Loading

hev (heiher) Jul 2, 2025 •

edited

Loading