Skip to content

latency-histogram: port BLT chart to Tk canvas for Tcl 9 - #3968

Merged
andypugh merged 1 commit into
LinuxCNC:masterfrom
grandixximo:latency-histogram-tcl9
Apr 26, 2026
Merged

andypugh merged 1 commit into
LinuxCNC:masterfrom
grandixximo:latency-histogram-tcl9

Conversation

@grandixximo

@grandixximo grandixximo commented Apr 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • BLT is dead since 2002 with no Tcl/Tk 9 port; this blocks latency-histogram on Fedora 42+.
  • Replaces blt::barchart with a small in-script lh_chart namespace on plain Tk canvas; call sites stay identical.
  • Also stubs TclX (signal no-op) and replaces blt::winop snap with a Tk 8.7+/9 native photo, ImageMagick fallback.

Refs #3793.

Test plan

  • Debian 13 / Tcl 8.6 RIP build, both threads with setuid, side-by-side compared with system BLT version.
  • Fedora 43 / Tcl 9.x (would appreciate @NTULINUX testing this).

@NTULINUX

Copy link
Copy Markdown
Contributor

This commit fixes: #3793

Can confirm on Tcl/Tk 9 setup. Removed TclX and blt packages, still works!

@BsAtHome

Copy link
Copy Markdown
Contributor

I think you will need to adapt the code to prevent the RT/non-RT race of the data transfer from RT into non-RT.

Both latencybins and timedelta were dropped from my version and rewrote them as timedeltalatch.comp and latencybinstream.comp (see attached zip). These new versions solve the problem by introducing data latching. This also solves the (exorbitant) delay you get when you wanted to read the bins from RT because you had to wait one cycle for each +/- bin pair after setting the index (and in that time, the histogram changed every cycle).
latchcomps.zip

The latency bins are conveyed using the HAL stream mechanism, which is why I needed to fix that first. The data is pushed into the stream in RT as one set when you set the latch pin. Non-RT can then look and wait until the right amount of values are in the stream and read them all. Below are the code snippets I used to get the data (python). You'll have to adapt it to Tcl...

Getting the latency bin data from the stream:

    def readBins(self):
        if not self.stream:
            return
        mod = self.name + "hist."    # 'basehist', 'servohist'
        dly = self.period / 1e9
        if dly < 1e-3: dly = 1e-3
        hal.set_p(mod+"stream", "1")   # Signal to stream
        cnt = 0;
        req = 2*self.nbins + 1 + 2
        # Wait for the data to arrive
        while True:
            time.sleep(dly)
            depth = self.stream.depth
            if depth >= req:
                break
            cnt += 1
            if cnt >= 100:
                err = hal.get_value(mod+"stream-error")
                if err:
                    warning("Stream data for {} was not sent".format(self.name))
                    return
                else:
                    error("Failed to get enough stream data ({} < {}) for {} thread".format(depth, req, self.name))

        # Get the data from the steam
        cnt = 1
        maxi = -1
        bins = []
        for i in range(0, req):
            if not self.stream.readable:
                error("Stream became not readable while reading element {} from {} elements".format(i, req))
            tup = self.stream.read()[0]
            if tup > maxi:
                maxi = tup
            if tup <= 0:
                tup = 1
            bins.append(tup)
        if cnt <= 0:
            error("No samples counted")
#        cnt /= 100  # We want range 1...100
        cnt = maxi / 100
        self.bins = [bins[i] / cnt for i in range(0, req)]
        self.timeval = hal.get_value(mod+"stream-time")
        if self.timeval >= self.period:
            self.timewarn += 1
        # Get the statistics
        self.latavg  = hal.get_value(mod+"latency-avg")
        self.latmin  = hal.get_value(mod+"latency-min")
        self.latmax  = hal.get_value(mod+"latency-max")
        self.stdev   = math.sqrt(math.fabs(hal.get_value(mod+"variance")))
        hal.set_p(mod+"stream", "0")   # Reset for next rising edge

Getting the time delta values:

    def readStats(self):
        mod = self.name + "time."
        dly = self.period / 1000000000
        if dly < 1e-3:  # Don't sleep less than one millisecond
            dly = 1e-3
        hal.set_p(mod+"latch", "1")
        cnt = 0
        while cnt < 100:
            time.sleep(dly)
            if hal.get_value(mod+"latched"):
                break
        if cnt >= 100:
            error("Failed to retrieve {} statistics, 'latched' never became active".format(self.name))
        hal.set_p(mod+"latch", "0")
        self.count= hal.get_value(mod+"count")
        self.pmin = hal.get_value(mod+"period-min")
        self.pmax = hal.get_value(mod+"period-max")
        self.psum = hal.get_value(mod+"period-sum")
        self.pssq = hal.get_value(mod+"period-ssq")
        self.jmin = hal.get_value(mod+"jitter-min")
        self.jmax = hal.get_value(mod+"jitter-max")
        self.jsum = hal.get_value(mod+"jitter-sum")
        self.jssq = hal.get_value(mod+"jitter-ssq")
        self.amin = hal.get_value(mod+"absjit-min")
        self.amax = hal.get_value(mod+"absjit-max")
        self.asum = hal.get_value(mod+"absjit-sum")
        self.assq = hal.get_value(mod+"absjit-ssq")
        self.pstdev = self.stdev(self.psum, self.pssq, self.count)
        self.jstdev = self.stdev(self.jsum, self.jssq, self.count)
        self.astdev = self.stdev(self.asum, self.assq, self.count)
        if self.count > 0:
            self.pavg = self.psum / self.count
            self.javg = self.jsum / self.count
            self.aavg = self.asum / self.count
        else:
            self.pavg = self.javg = self.aavg = 0.0
            # The min/max would otherwise show +/- MAXINT
            self.pmax = self.pmin = self.jmax = self.jmin = self.amax = self.amin = 0
        if self.pmax > self.pmaxall: self.pmaxall = self.pmax
        if self.pmin < self.pminall: self.pminall = self.pmin
        if self.jmax > self.jmaxall: self.jmaxall = self.jmax
        if self.jmin < self.jminall: self.jminall = self.jmin
        if self.amax > self.amaxall: self.amaxall = self.amax
        if self.amin < self.aminall: self.aminall = self.amin

grandixximo added a commit to grandixximo/linuxcnc that referenced this pull request Apr 26, 2026
Old latencybins comp transfers bins one cycle at a time. Snapshot
smears across hundreds of ms, stats desync from bins. Race noted by
@BsAtHome on PR LinuxCNC#3968.

- Add latencybinstream.comp (B.Stultiens): atomic histogram + stats
  snapshot pushed into HAL stream FIFO on `stream` rising edge.
- Add scripts/latency-bin-reader: long-running Python helper since
  Tcl has no hal_stream binding. Tcl pipes READ/QUIT to it.
- Wire into latency-histogram as default. Per-thread unique stream
  keys derived from PID for coexistence.
- Keep old behavior under --legacy flag for comparison.

Refs LinuxCNC#3793
Refs LinuxCNC#3968
@grandixximo

Copy link
Copy Markdown
Contributor Author

Pushed f600c1d.

  • Brings in @BsAtHome latencybinstream.comp as the default. Confirmed the race: bins transferred one cycle at a time get smeared across hundreds of ms, stats desync from bins.
  • Tcl has no hal_stream binding, so a small Python helper (scripts/latency-bin-reader) drains the FIFO on demand over a stdin/stdout pipe. One helper per thread, kept open for the session.
  • Old latencybins path kept behind --legacy for direct comparison.

Did not pull timedeltalatch.comp in this round, the script's existing min/max/sdev come from latencybinstream pins (latched on stream rising edge) so no extra comp needed for the GUI. Happy to add it in a follow-up if there's appetite for period/jitter readouts.

Tested on Debian 13 / Tcl 8.6 with both default and --legacy.

@grandixximo
grandixximo force-pushed the latency-histogram-tcl9 branch from 2a89b36 to 0ff9d2f Compare April 26, 2026 11:51
@grandixximo

Copy link
Copy Markdown
Contributor Author

Force-pushed 0ff9d2f (squashed).

Replaced the Python sidecar with a native Tcl hal_stream binding in halsh.c (attach/read/drain/depth/detach). Pure Tcl now, no extra runtime dep. latency-histogram drains the latencybinstream FIFO directly.

Comment thread src/hal/utils/halsh.c Resolved
Comment thread .gitignore Resolved
Comment thread src/hal/utils/halsh.c Resolved
Comment thread scripts/latency-histogram Resolved
@grandixximo
grandixximo force-pushed the latency-histogram-tcl9 branch from 0ff9d2f to 8d0c092 Compare April 26, 2026 12:15
@grandixximo

grandixximo commented Apr 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks for the review @BsAtHome. 8d0c092 addresses the four points:

  • halsh.c: HAL_S64 / HAL_U64 now use v.l / v.k. Good catch, the s32/s64 truncation would have bitten any wider stream.
  • halsh.c: #include <hal.h> for the public layer.
  • script: in addition to the depth-reach poll before reading, also verify hal_stream depth == 0 after drain so we abort on out-of-phase reads, and check stream-error pin on timeout.
  • .gitignore: I went through all 204 .9 files in docs/man/man9/ and each has either a .comp, a .adoc, or a kinematics .c source. None are tracked. Happy to switch to ignoring the directory if you prefer.

BLT dead since 2002, no Tcl/Tk 9 port - blocks Fedora 42+.

- Replace blt::barchart with in-script lh_chart on Tk canvas
- Stub TclX signal no-op when missing
- Replace blt::winop snap with Tk 8.7+/9 native photo / ImageMagick
- Sci notation for axis labels >= 5 digits, nice ceilings (1/2/2.5/5)
- Screenshot dir ~/Pictures (was /tmp)

Atomic snapshot via @BsAtHome latencybinstream.comp (default).
Old latencybins kept under --legacy flag.

Add Tcl hal_stream binding to halsh.c (attach/read/drain/depth/detach)
so the FIFO can be drained from Tcl directly without a Python sidecar.

Refs LinuxCNC#3793
@grandixximo
grandixximo force-pushed the latency-histogram-tcl9 branch from 8d0c092 to a3271af Compare April 26, 2026 12:21
@grandixximo

grandixximo commented Apr 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Reverted root .gitignore change. Added man9/latencybinstream.9 to docs/man/.gitignore matching the existing per-file convention. a3271af

There are no non-generated files, maybe another PR could do the switch, from write one by one to a simple *.9

@andypugh
andypugh merged commit bf7fa26 into LinuxCNC:master Apr 26, 2026
14 checks passed
@grandixximo
grandixximo deleted the latency-histogram-tcl9 branch April 27, 2026 02:48
This was referenced Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants