Skip to content

optimization: Build the G-code preview in C++ - #4481

Merged
BsAtHome merged 2 commits into
LinuxCNC:masterfrom
alex-pres:pr-render-preview-cpp
Sep 15, 2026
Merged

BsAtHome merged 2 commits into
LinuxCNC:masterfrom
alex-pres:pr-render-preview-cpp

Conversation

@alex-pres

@alex-pres alex-pres commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Follows PR##4293

Adds C++ GCodeRenderer, which builds the whole preview - transform, arcs, taps, suppression, vertices, extents, lengths and event records - during gcode.parse and hands it over once, replacing the Python rendering engine. C++ handles Vec[9] -> X,Y,Z transform completely, following all interpretator details, and Python renders it into OpenGL using shaders.

Parsing is ~2.9x faster on move-shaped programs and peaks at less memory; and specially for arcs, the last per-move Python call kept in code, go from 1.58 s to 0.075 s at 1.28M segments.

Even without speed benefit it is anyway better because resulting code is much more clean and easy to follow. Previous code was hard to read, understand and test. Right now we can test GCodeRenderer engine directly, it is independent of presentation.

Benchmark results, compare to current master

Headless, fractal-1M.ngc (1,000,149 moves):

master this branch
parse+build 2.705 s 0.732 s (3.7x)
total load 2.712 s 0.745 s
peak RSS 166.8 MB 163.5 MB
VBO bytes 20,003,000 20,003,000
CPU geometry 28,004,200 28,004,200
extents identical identical

Byte-identical geometry out, ~3.7x less time to produce it.

Results could be much better if ngc file use arcs (current master calculates arcs in Python which was bad idea, after moving to C++ it is 100x faster).

@alex-pres
alex-pres marked this pull request as draft August 27, 2026 15:50
@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from bd037e7 to c0bf905 Compare August 27, 2026 16:01
@alex-pres
alex-pres marked this pull request as ready for review August 27, 2026 16:06
@BsAtHome

Copy link
Copy Markdown
Contributor

What are all those json files doing in there? It seems you want to do some "bake" test, but I did not see a test{.hal,.sh,} in that directory. Did I miss it?
You also need to disable any test(s) that cannot run in the headless CI run. All GUI/Graphics related dependencies are not installed in the headless CI run.

Another point, you are "redecorating" the gcodemodule. Would it be an idea if we move this to pybind11 at the same time? Doing this work twice may be more work and may also be error prone.

@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from c0bf905 to 0b8fa5f Compare August 27, 2026 16:20
@alex-pres

alex-pres commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor Author

What are all those json files doing in there? It seems you want to do some "bake" test, but I did not see a test{.hal,.sh,} in that directory. Did I miss it? You also need to disable any test(s) that cannot run in the headless CI run. All GUI/Graphics related dependencies are not installed in the headless CI run.

My fault, forgot to add test.sh - done now. JSON files are reference results for related .ngc files from another folder.

No problem with headless - these tests are for C++ routines that do math only - no GUI, no graphics.

Another point, you are "redecorating" the gcodemodule. Would it be an idea if we move this to pybind11 at the same time? Doing this work twice may be more work and may also be error prone.

Any WIP on this or samples?

Yes, I can work on pybind11 conversion, but any existing refrerences to such change in LinuxCNC will be helpful. If any

@alex-pres

Copy link
Copy Markdown
Contributor Author

I agree test files do not look good. I will rework it to easily see what each case is testing.

In time being, I'd appreciate feedback on the rest of PR.

@alex-pres

Copy link
Copy Markdown
Contributor Author

Another point, you are "redecorating" the gcodemodule. Would it be an idea if we move this to pybind11 at the same time? > Doing this work twice may be more work and may also be error prone.

OK, I have found halquery is using pybind now. I will try to rework gcodemodule to use it too in this branch.

@alex-pres
alex-pres marked this pull request as draft August 27, 2026 19:49
@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from 0b8fa5f to 7597f74 Compare August 28, 2026 12:44
@alex-pres
alex-pres marked this pull request as ready for review August 28, 2026 12:45
@alex-pres

alex-pres commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Reworked with pybind11 , it is a pleasure to work with. Resulting code is much easier to read and follow! Added some modern C++ which gcodemodule.cc deserve )

Tests reworked too.

Comment thread src/emc/rs274ngc/gcodemodule.cc Resolved
Comment thread src/emc/rs274ngc/Submakefile Resolved
@alex-pres

Copy link
Copy Markdown
Contributor Author

Last commit is breaking compatibility - returns tuples instead of arrays is not good idea, breaks code on Python side. I will revert this at Monday.

@alex-pres
alex-pres marked this pull request as draft August 28, 2026 18:58
@alex-pres
alex-pres marked this pull request as ready for review August 31, 2026 08:25
@alex-pres

Copy link
Copy Markdown
Contributor Author

Last commit is breaking compatibility - returns tuples instead of arrays is not good idea, breaks code on Python side. I will revert this at Monday.

It was false alarm, problem not caused by this code. Everything seems to be OK and ready for review and testing.

Current live testing results: loading of 418Mb .ngc file (real work file sent by customer, with G2 arcs): 25 seconds to first render.

@grandixximo

Copy link
Copy Markdown
Contributor

418Mb .ngc file (real work file sent by customer, with G2 arcs): 25 seconds

🤤

@alex-pres alex-pres changed the title Optimization: Build the G-code preview in C++ optimization: Build the G-code preview in C++ Aug 31, 2026
@alex-pres

alex-pres commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor Author

418Mb .ngc file (real work file sent by customer, with G2 arcs): 25 seconds

🤤

Together with PR #4491 : 12 seconds!

P.S. As said before AXIS [specially AXIS] is using not optimised file editor code that now eats more memory and CPU than graphical preview. So for benchmarking I cut it off. Worth to be optimised too, may be later.

@alex-pres

alex-pres commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor Author

And with both #4491 and #4492 - 5.7 seconds to first frame on same file.
Peak RSS after file load: +1.04GB.

@grandixximo

grandixximo commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Could you please fetch master, then merge or rebase and push, just to let the CI run again, and we shall get the shots of all the UIs to make sure nothing regressed.

@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from 2564163 to 075e6b8 Compare September 2, 2026 14:27
@alex-pres

Copy link
Copy Markdown
Contributor Author

Could you please fetch master, then merge or rebase and push, just to let the CI run again, and we shall get the shots of all the UIs to make sure nothing regressed.

Done. And squashed to one commit.

@BsAtHome

BsAtHome commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

And the screenshots seem to be similar to what we see elsewhere.

Comment thread src/rtapi/rtapi_string.h Resolved
Comment thread lib/python/rs274/glcanon.py Resolved
Comment thread lib/python/rs274/glcanon.py Resolved
Comment thread lib/python/rs274/glcanon.py Resolved
Comment thread src/emc/rs274ngc/gcode_renderer.cc Resolved
Comment thread src/emc/rs274ngc/gcode_renderer.cc Resolved
Comment thread src/emc/rs274ngc/gcode_renderer.cc Resolved
Comment thread src/emc/rs274ngc/gcode_renderer.cc Resolved
Comment thread src/emc/rs274ngc/gcodemodule.cc Resolved
@alex-pres

Copy link
Copy Markdown
Contributor Author

Finished with changes, hope all mentioned is fixed now

@grandixximo

Copy link
Copy Markdown
Contributor

You might want to look at CI the clang is reporting failures, we have it fail on warnings, gotta clean those up.

@alex-pres

Copy link
Copy Markdown
Contributor Author

You might want to look at CI the clang is reporting failures, we have it fail on warnings, gotta clean those up.

fixed with last commit.

Comment thread src/emc/rs274ngc/gcode_renderer.cc Resolved

@NTULINUX NTULINUX left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very awesome, thank you!

@alex-pres
alex-pres requested a review from BsAtHome September 7, 2026 08:49
@alex-pres

Copy link
Copy Markdown
Contributor Author

BsAtHome sorry, I clicked "request review" by mistake.

@alex-pres

Copy link
Copy Markdown
Contributor Author

Pushed documentation update, must be good to go now.

@alex-pres

Copy link
Copy Markdown
Contributor Author

Last commit adds ability for Python code to request actual machine axis positions array along with usually returned visual XYZ points. It is optional and disabled by default. It may be useful for advanced rendering or program analysis.

@alex-pres

Copy link
Copy Markdown
Contributor Author

Tool offsets is useful to build correct re-presentation based on AXIS coordinates [ like vismach ] . So it is returned too as sorted tuples, records mark line range affected. ProgramGeometry in python knowns how to parse it.

@alex-pres

Copy link
Copy Markdown
Contributor Author

Tool offsets is useful to build correct re-presentation based on AXIS coordinates [ like vismach ] . So it is returned too as sorted tuples, records mark line range affected. ProgramGeometry in python knowns how to parse it.

Just pushed and understood that implementation is invalid - it must index not line# but idx# in move array. It'd be incorrect if tool offsets change mid line. Not sure if that is possible but anyway better to fix. I will submit better implementation next week.

@BsAtHome BsAtHome self-assigned this Sep 13, 2026
Comment thread src/emc/rs274ngc/gcode_renderer.cc Resolved
Comment thread src/emc/rs274ngc/gcode_renderer.cc Resolved
Comment thread src/emc/rs274ngc/gcode_renderer.cc Resolved
Comment thread src/emc/rs274ngc/gcode_renderer.cc Resolved
Comment thread tests/gcode-renderer/test_bake.py Resolved
Comment thread tests/gcode-renderer/test.sh Resolved
@BsAtHome

Copy link
Copy Markdown
Contributor

BTW, you need to rebase because there is a conflict now.

@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from 07e4603 to a46e1c7 Compare September 14, 2026 11:37
@alex-pres

Copy link
Copy Markdown
Contributor Author

BTW, you need to rebase because there is a conflict now.

done

@alex-pres

alex-pres commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor Author

Probably need one more step here: I almost implemented TimeEstimateCanon in C++ which does close-to-real time estimate during parse and return it to GUI.
Compared in SIM and got ~98% time match which I believe is quite enough.

It would deserve separate PR but because it changes gcodemodule.cc internals it may be here too. Let me know if separate PR is better choice.

@BsAtHome

Copy link
Copy Markdown
Contributor

Probably need one more step here: I almost implemented TimeEstimateCanon in C++ which does close-to-real time estimate during parse and return it to GUI. Compared in SIM and got ~98% time match which I believe is quite enough.

A 98% match is better than most people can guess, I guess.
Unless we're talking about a long tail that is "off by progress dialogs from a know Redmond software firm pretending it could calculate time". If that is not the case, then we'd be stupid to complain at this stage.

It would deserve separate PR but because it changes gcodemodule.cc internals it may be here too. Let me know if separate PR is better choice.

I think this PR has been boiling quite enough for now. The speed improvement is a good thing. Improving on it can be the next update.

@alex-pres

alex-pres commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor Author

I completely agree: this PR is ready to be served.

I'll add one small refactoring : split GcodeRenderer to 2 classes, one that is base and second is preview-targeted. And it will be enough for this branch and ready to merge.

Next PR for time estimate will use the base class to do its own job, and it will be add-on only not touching the preview itself, I think it will be best that way.

And time estimate definitely will work much better than Win 3.11 install progress )

@BsAtHome

Copy link
Copy Markdown
Contributor

And please squash the commits, if you can.

Adds C++ GCodeRenderer, which builds the whole preview - transform, arcs, taps,
suppression, vertices, extents, lengths and event records - during gcode.parse
and hands it over once, replacing the Python rendering engine. C++ handles Vec[9] -> X,Y,Z transform completely, and Python renders it into OpenGL using shaders.

2 protocols to interact with python: CallbackCanon (compat) and GCodeRenderer (new). Selected when "parse" method called based on canon properties.
@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from 9dd8985 to dad6bc1 Compare September 15, 2026 09:39
@alex-pres

alex-pres commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor Author

After trying I decided to abandon idea of the class split - code becomes too noisy and hard to follow. So I squashed all commits already reviewed.

Last problem I found accidentally - LineCode pybind11 rewrite slow down it. It does not affect new rendering code, but those who will use old python canon protocol (where all moves are reported to python methods) will be affected. And it is not cosmetic (numbers below). So I rewrote it back to C++ struct shared to python without pybind11. I made this a separate commit. After review, it is ready to merge [ I can do squash again ]

Benchmark

8M-move pure-G1 file, dev container, both gcode.so builds kept side by side and alternated rep by rep so drift lands on both. Minimum of 5 reps.

canon's next_line before after delta
drops the object 5.04 s 4.24 s −0.80 s (−16 %)
reads sequence_number 6.06 s 4.40 s −1.67 s (−27 %)
reads four fields 8.60 s 4.71 s −3.88 s (−45 %)

The first row is exactly the 0.80 s the plan attributed to the object. The other two are the part the plan listed as unmeasured, and it was the larger half: a field read went from ~125 ns (pybind11 property lambda) to ~19 ns (descriptor over a struct offset).
.

@BsAtHome
BsAtHome merged commit aaa4b35 into LinuxCNC:master Sep 15, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants