Skip to content

Commit c89ae87

Browse files
boxerabboxerab
authored
add the football analyzer pipeline (#19)
player, referee and ball detection from one model, camera motion compensation in the tracker, and a broadcast-style overlay. weights come from the hugging face hub via demo/football/fetch_models.py.
2 parents 2613cfa + b633652 commit c89ae87

34 files changed

Lines changed: 3985 additions & 45 deletions

‎data/COLLABORA_02_RGB.png‎

20.9 KB
Loading

‎data/Chinedu-Obasi_2684938.jpg‎

44.2 KB
Loading

‎demo/football/README.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Football demo
2+
3+
Real-time football broadcast overlay: **detection → tracking → overlay**
4+
(`pyml_yolo`/`pyml_objectdetector` -> `pyml_tracker` -> `pyml_football_overlay`).
5+
6+
The overlay draws a foot ellipse per player coloured by team (red/blue, voted
7+
from jersey hue), a gold ellipse for referees, motion trails (off by default),
8+
and a focal-player HUD with headshot, ball contacts, and distance travelled.
9+
Players whose team isn't decided yet (and unclassifiable kits, e.g. the
10+
goalkeeper) are left unmarked rather than drawn in a placeholder colour. The
11+
ball is tracked for contact counting but its marker is off by default.
12+
13+
## Models
14+
15+
The detector weights (`football.pt`, `football.onnx`, `football_fp16.onnx`,
16+
`football_int8.onnx`) are hosted on the Hugging Face Hub at
17+
`collabora/gst-python-ml-football`, not in git. `run.sh` downloads the one its
18+
`BACKEND` needs into `models/football/` on first use. To fetch by hand:
19+
20+
```bash
21+
python demo/football/fetch_models.py # pt + fp16
22+
python demo/football/fetch_models.py all
23+
```
24+
25+
## Run
26+
27+
```bash
28+
# file -> annotated MP4
29+
demo/football/run.sh
30+
demo/football/run.sh 08fd33_4.mp4 demo/football/out.mp4 1280x720
31+
32+
# file -> live on-screen
33+
demo/football/run.sh display
34+
demo/football/run.sh display 08fd33_4.mp4 1280x720
35+
36+
# live camera -> on-screen
37+
demo/football/run.sh camera /dev/video0
38+
```
39+
40+
## Environment knobs
41+
42+
| Var | Default | Meaning |
43+
|------------|---------|---------|
44+
| `BACKEND` | `pt` | `pt` = PyTorch `pyml_yolo`; `fp16` = ONNX FP16 via `pyml_objectdetector` (CUDA). |
45+
| `INTERVAL` | `3` | Run detection every Nth frame; the tracker/overlay still update every frame, so it stays smooth at ~N× less inference cost. The main real-time lever. |
46+
47+
```bash
48+
BACKEND=fp16 demo/football/run.sh display # faster inference path
49+
INTERVAL=5 demo/football/run.sh display # detect every 5th frame
50+
INTERVAL=1 demo/football/run.sh # detect every frame (max accuracy)
51+
```
52+
53+

‎demo/football/fetch_models.py‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
#!/usr/bin/env python3
2+
# Football demo model download
3+
# Copyright (C) 2026 Collabora Ltd.
4+
#
5+
# This library is free software; you can redistribute it and/or
6+
# modify it under the terms of the GNU Library General Public
7+
# License as published by the Free Software Foundation; either
8+
# version 2 of the License, or (at your option) any later version.
9+
#
10+
# This library is distributed in the hope that it will be useful,
11+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
12+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
13+
# Library General Public License for more details.
14+
#
15+
# You should have received a copy of the GNU Library General Public
16+
# License along with this library; if not, write to the
17+
# Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
18+
# Boston, MA 02110-1301, USA.
19+
#
20+
# Download the football detector weights from the Hugging Face Hub into
21+
# models/football/ (gitignored). Usage:
22+
# python demo/football/fetch_models.py # pt + fp16, what run.sh uses
23+
# python demo/football/fetch_models.py int8 onnx # named variants
24+
# python demo/football/fetch_models.py all
25+
26+
import os
27+
import sys
28+
29+
from huggingface_hub import hf_hub_download
30+
31+
REPO_ID = "collabora/gst-python-ml-football"
32+
LOCAL_DIR = os.path.join(
33+
os.path.dirname(os.path.abspath(__file__)), "..", "..", "models", "football"
34+
)
35+
# run.sh's BACKEND value -> the file it loads
36+
VARIANTS = {
37+
"pt": "football.pt",
38+
"fp16": "football_fp16.onnx",
39+
"onnx": "football.onnx",
40+
"int8": "football_int8.onnx",
41+
}
42+
43+
44+
def main(argv):
45+
wanted = argv[1:] or ["pt", "fp16"]
46+
if wanted == ["all"]:
47+
wanted = list(VARIANTS)
48+
for variant in wanted:
49+
if variant not in VARIANTS:
50+
sys.exit(
51+
f"unknown model variant {variant!r}; "
52+
f"choose from {', '.join(VARIANTS)} or all"
53+
)
54+
local = os.path.join(LOCAL_DIR, VARIANTS[variant])
55+
if os.path.isfile(local):
56+
print(local)
57+
continue
58+
path = hf_hub_download(
59+
repo_id=REPO_ID, filename=VARIANTS[variant], local_dir=LOCAL_DIR
60+
)
61+
print(path)
62+
63+
64+
if __name__ == "__main__":
65+
main(sys.argv)

‎demo/football/onnx_loop.py‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
#!/usr/bin/env python3
2+
# Run a video through the ONNX (fp16) football pipeline.
3+
#
4+
# detector (onnx) -> pyml_tracker -> pyml_football_overlay
5+
#
6+
# Usage:
7+
# python demo/football/onnx_loop.py INPUT.mp4 # live display, looping
8+
# python demo/football/onnx_loop.py INPUT.mp4 OUTPUT.mp4 # write annotated mp4
9+
# (self-contained: finds the repo venv + plugins and re-execs into them)
10+
import os
11+
import subprocess
12+
import sys
13+
import glob
14+
15+
REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
16+
VENV = os.path.join(REPO, ".venv")
17+
MODEL = os.path.join(REPO, "models/football/football_fp16.onnx")
18+
os.environ["GST_PLUGIN_PATH"] = (
19+
os.path.join(REPO, "plugins") + os.pathsep + os.environ.get("GST_PLUGIN_PATH", "")
20+
)
21+
if not os.environ.get("_ONNX_LOOP_REEXEC") and os.path.isdir(VENV):
22+
os.environ["VIRTUAL_ENV"] = VENV
23+
os.environ["PATH"] = (
24+
os.path.join(VENV, "bin") + os.pathsep + os.environ.get("PATH", "")
25+
)
26+
libs = sorted(
27+
set(
28+
glob.glob(
29+
os.path.join(
30+
VENV, "lib", "python*", "site-packages", "nvidia", "*", "lib"
31+
)
32+
)
33+
)
34+
)
35+
if libs:
36+
os.environ["LD_LIBRARY_PATH"] = os.pathsep.join(
37+
[*libs, os.environ.get("LD_LIBRARY_PATH", "")]
38+
)
39+
os.environ["_ONNX_LOOP_REEXEC"] = "1"
40+
pybin = os.path.join(VENV, "bin", "python")
41+
exe = pybin if os.path.exists(pybin) else sys.executable
42+
os.execv(exe, [exe, *sys.argv])
43+
44+
if not os.path.isfile(MODEL):
45+
subprocess.check_call(
46+
[
47+
sys.executable,
48+
os.path.join(REPO, "demo", "football", "fetch_models.py"),
49+
"fp16",
50+
]
51+
)
52+
53+
import gi # noqa: E402
54+
55+
gi.require_version("Gst", "1.0")
56+
from gi.repository import Gst, GLib # noqa: E402
57+
58+
Gst.init(None)
59+
60+
61+
def on_message(bus, message, loop, pipeline, do_loop):
62+
t = message.type
63+
64+
if t == Gst.MessageType.EOS:
65+
if do_loop:
66+
# Display mode: seek back to the start to loop the clip.
67+
print("Looping...")
68+
if not pipeline.seek_simple(
69+
Gst.Format.TIME, Gst.SeekFlags.FLUSH | Gst.SeekFlags.KEY_UNIT, 0
70+
):
71+
print("Failed to seek back to start", file=sys.stderr)
72+
loop.quit()
73+
else:
74+
# mp4 mode: end of file, the muxer has finalized the file.
75+
loop.quit()
76+
77+
elif t == Gst.MessageType.ERROR:
78+
err, debug = message.parse_error()
79+
print(f"ERROR: {err}", file=sys.stderr)
80+
if debug:
81+
print(f"DEBUG: {debug}", file=sys.stderr)
82+
loop.quit()
83+
84+
85+
def main():
86+
if len(sys.argv) < 2:
87+
print(f"usage: {sys.argv[0]} INPUT.mp4 [OUTPUT.mp4]", file=sys.stderr)
88+
print(
89+
" no OUTPUT -> live display (looping); OUTPUT -> write annotated mp4",
90+
file=sys.stderr,
91+
)
92+
sys.exit(1)
93+
video = os.path.abspath(sys.argv[1])
94+
out = os.path.abspath(sys.argv[2]) if len(sys.argv) > 2 else None
95+
96+
# Shared detection + overlay chain. Feed the ORIGINAL resolution:
97+
# pyml_objectdetector letterboxes to the model's 640 internally for
98+
# inference and maps boxes back, so the overlay stays full-res.
99+
chain = (
100+
f"filesrc location={video} ! "
101+
"decodebin ! videoconvert ! video/x-raw,format=RGB ! "
102+
"queue max-size-buffers=8 max-size-time=0 max-size-bytes=0 ! "
103+
"pyml_objectdetector engine-name=onnx "
104+
f" model-name={MODEL} device=cuda:0 "
105+
" input-format=nchw post-process=anchor_free interval=1 "
106+
" confidence=0.1 nms-iou=0.7 ! "
107+
"queue max-size-buffers=8 max-size-time=0 max-size-bytes=0 ! "
108+
"pyml_tracker tracker-type=bytetrack new-track-confidence=0.25 ! "
109+
"videoconvert ! video/x-raw,format=RGBA ! "
110+
"queue max-size-buffers=8 max-size-time=0 max-size-bytes=0 ! "
111+
"pyml_football_overlay class-names=ball,goalkeeper,player,referee "
112+
" team-colors=true trails=false show-ids=false show-labels=false "
113+
" draw-from-detections=true min-confidence=0 merge-iou=0.5 "
114+
" position-smoothing=0.7 highlight-focal=false ! "
115+
)
116+
if out:
117+
pipeline_description = (
118+
chain + "queue max-size-buffers=8 max-size-time=0 max-size-bytes=0 ! "
119+
"videoconvert ! openh264enc ! h264parse ! mp4mux ! "
120+
f"filesink location={out}"
121+
)
122+
do_loop = False
123+
else:
124+
# Pre-roll buffer absorbs inference jitter for smooth real-time display.
125+
pipeline_description = (
126+
chain + "queue max-size-buffers=600 max-size-time=0 max-size-bytes=0 "
127+
" min-threshold-buffers=30 ! "
128+
"videoconvert ! autovideosink sync=true"
129+
)
130+
do_loop = True
131+
132+
print(pipeline_description)
133+
print(f"writing -> {out}" if out else "live display (looping)")
134+
135+
try:
136+
pipeline = Gst.parse_launch(pipeline_description)
137+
except GLib.Error as e:
138+
print(f"Failed to create pipeline: {e}", file=sys.stderr)
139+
sys.exit(1)
140+
141+
loop = GLib.MainLoop()
142+
143+
bus = pipeline.get_bus()
144+
bus.add_signal_watch()
145+
bus.connect("message", on_message, loop, pipeline, do_loop)
146+
147+
pipeline.set_state(Gst.State.PLAYING)
148+
149+
try:
150+
loop.run()
151+
except KeyboardInterrupt:
152+
if out:
153+
# Finalize the mp4 on Ctrl-C: send EOS and wait for the muxer to
154+
# flush its trailer, otherwise the file is left unplayable.
155+
pipeline.send_event(Gst.Event.new_eos())
156+
bus.timed_pop_filtered(
157+
5 * Gst.SECOND, Gst.MessageType.EOS | Gst.MessageType.ERROR
158+
)
159+
finally:
160+
pipeline.set_state(Gst.State.NULL)
161+
if out:
162+
print(f"Done: {out}")
163+
164+
165+
if __name__ == "__main__":
166+
main()

‎demo/football/run.sh‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
#!/usr/bin/env bash
2+
# Football broadcast-overlay demo.
3+
#
4+
# Ppipeline:
5+
# detector -> pyml_tracker (ByteTrack) -> pyml_football_overlay
6+
#
7+
# Usage:
8+
# demo/football/run.sh [INPUT.mp4] [OUTPUT.mp4] [WxH] # file -> annotated mp4
9+
# demo/football/run.sh display [INPUT.mp4] [WxH] # file -> live on-screen
10+
# demo/football/run.sh camera [/dev/videoN] [WxH] # live camera -> on-screen
11+
set -euo pipefail
12+
13+
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
14+
cd "$REPO"
15+
source .venv/bin/activate
16+
export GST_PLUGIN_PATH="$REPO/plugins:${GST_PLUGIN_PATH:-}"
17+
18+
BACKEND="${BACKEND:-pt}"
19+
# The weights live on the Hugging Face Hub; this is a no-op once cached.
20+
python demo/football/fetch_models.py "$BACKEND"
21+
INTERVAL="${INTERVAL:-3}" # run detection every Nth frame; tracker/overlay stay per-frame
22+
CONF="${CONF:-0.1}" # detector confidence threshold (low = more detections)
23+
IOU="${IOU:-0.7}" # NMS IoU (ultralytics/football_analyzer default)
24+
NEWTRACK="${NEWTRACK:-0.25}" # min confidence to START a new track (ByteTrack gate; kills ghosts)
25+
DRAWCONF="${DRAWCONF:-0}" # min confidence to DRAW a detection (0 = draw all; raise to trim weak boxes)
26+
MERGE="${MERGE:-0.5}" # collapse overlapping boxes (lower=merge more; 0 disables) so one player=one circle
27+
SMOOTH="${SMOOTH:-0.6}" # temporal EMA on circle positions (0=off, higher=smoother but more lag)
28+
CLASSES="ball,goalkeeper,player,referee"
29+
TRACK="pyml_tracker tracker-type=bytetrack new-track-confidence=$NEWTRACK"
30+
# Detection-based overlay: circles sit on the raw per-frame detections (no
31+
# tracking drift/phantoms/doubles); merge collapses overlaps and
32+
# position-smoothing low-passes the positions. DRAWCONF defaults 0 so no
33+
# detection is hidden; the tracker still runs so the HUD keeps its stats.
34+
OVERLAY="pyml_football_overlay class-names=$CLASSES team-colors=true trails=false show-ids=false show-labels=false draw-from-detections=true min-confidence=$DRAWCONF merge-iou=$MERGE position-smoothing=$SMOOTH highlight-focal=false"
35+
36+
if [[ "$BACKEND" == "fp16" ]]; then
37+
export LD_LIBRARY_PATH="$(python -c "import os,nvidia,glob;b=os.path.dirname(nvidia.__file__);print(':'.join(sorted(set(glob.glob(b+'/*/lib')))))"):${LD_LIBRARY_PATH:-}"
38+
DETECT="pyml_objectdetector engine-name=onnx model-name=models/football/football_fp16.onnx device=cuda:0 input-format=nchw post-process=anchor_free interval=$INTERVAL"
39+
IN_FMT="RGB"; FORCE_SQUARE=1
40+
else
41+
DETECT="pyml_yolo model-name=models/football/football device=cuda:0 interval=$INTERVAL confidence=$CONF nms-iou=$IOU"
42+
IN_FMT="RGBA"; FORCE_SQUARE=0
43+
fi
44+
45+
POST_DETECT="$TRACK"
46+
[[ "$IN_FMT" == "RGB" ]] && POST_DETECT="$TRACK ! videoconvert ! video/x-raw,format=RGBA"
47+
48+
# A queue at each stage boundary turns the serial chain into a threaded
49+
# pipeline: while inference runs on frame N, the sink renders N-1 and the
50+
# decoder reads N+1. Nothing is dropped (leaky=no, the default).
51+
Q="queue max-size-buffers=8 max-size-time=0 max-size-bytes=0"
52+
# Pre-roll buffer before the display sink: build a head start of processed
53+
# frames so real-time playback (sync=true) rides out per-frame inference
54+
# jitter without stuttering. Smooths jitter, not a sustained throughput
55+
# deficit -- if inference can't keep up on average, playback just lags
56+
# (still no drops). Lower INTERVAL/raise the head start if it falls behind.
57+
PREROLL="queue max-size-buffers=600 max-size-time=0 max-size-bytes=0 min-threshold-buffers=30"
58+
59+
# detector -> tracker -> overlay, with a thread boundary at each hop.
60+
CHAIN="$Q ! $DETECT ! $Q ! $POST_DETECT ! $Q ! $OVERLAY"
61+
62+
MODE="${1:-file}"
63+
if [[ "$MODE" == "camera" ]]; then
64+
DEV="${2:-/dev/video0}"; SIZE="${3:-1280x720}"
65+
[[ "$FORCE_SQUARE" == "1" ]] && SIZE="640x640"
66+
W="${SIZE%x*}"; H="${SIZE#*x}"
67+
echo "[$BACKEND] live camera $DEV @ ${W}x${H} -> autovideosink (needs a display)"
68+
exec gst-launch-1.0 -e \
69+
v4l2src device="$DEV" ! videoconvert ! videoscale \
70+
! "video/x-raw,width=${W},height=${H},format=${IN_FMT}" \
71+
! $CHAIN \
72+
! $Q ! videoconvert ! autovideosink sync=false
73+
elif [[ "$MODE" == "display" ]]; then
74+
IN="${2:-data/soccer_tracking.mp4}"
75+
SIZE="${3:-1280x720}"
76+
[[ "$FORCE_SQUARE" == "1" ]] && SIZE="640x640"
77+
W="${SIZE%x*}"; H="${SIZE#*x}"
78+
[[ -f "$IN" ]] || { echo "input not found: $IN" >&2; exit 1; }
79+
echo "[$BACKEND] '$IN' @ ${W}x${H} -> live display (real-time, sync=true)"
80+
exec gst-launch-1.0 -e \
81+
filesrc location="$IN" ! decodebin ! videoconvert ! videoscale \
82+
! "video/x-raw,width=${W},height=${H},format=${IN_FMT}" \
83+
! $CHAIN \
84+
! $PREROLL ! videoconvert ! autovideosink sync=true
85+
else
86+
IN="${1:-data/soccer_tracking.mp4}"
87+
OUT="${2:-demo/football/out.mp4}"
88+
SIZE="${3:-1280x720}"
89+
[[ "$FORCE_SQUARE" == "1" ]] && SIZE="640x640"
90+
W="${SIZE%x*}"; H="${SIZE#*x}"
91+
[[ -f "$IN" ]] || { echo "input not found: $IN" >&2; exit 1; }
92+
echo "[$BACKEND] '$IN' @ ${W}x${H} -> '$OUT'"
93+
gst-launch-1.0 -e \
94+
filesrc location="$IN" ! decodebin ! videoconvert ! videoscale \
95+
! "video/x-raw,width=${W},height=${H},format=${IN_FMT}" \
96+
! $CHAIN \
97+
! $Q ! videoconvert ! openh264enc ! h264parse ! mp4mux ! filesink location="$OUT"
98+
echo "Done: $OUT"
99+
fi

0 commit comments

Comments
 (0)