Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/workflows/fsh-vet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@
# 1. Is upstream's security hygiene getting worse? (scorecard vs a baseline)
# 2. Do our dependencies have known vulnerabilities? (osv-scanner)
# 3. Is our fork still additive-only? (diff vs upstream)
# 4. Does the code we are about to build still build and pass? (go test)
#
# (4) is here because upstream's own ci.yml only triggers on PRs to main/master
# — a PR into `fsh` gets none of it, so without this job our own patches would
# merge with no build, vet or test gate at all.
#
# None of this proves upstream code is benign — Scorecard measures practices,
# not intent. It is a prior and a drift detector, not a verdict. See FORK.md.
Expand Down Expand Up @@ -56,7 +61,22 @@ jobs:
# Files we are allowed to add or change on top of upstream. Anything
# else means we have started editing his tree — which is a decision,
# not an accident, so it has to be made here, in this list.
#
# P4, the SQL Server driver, is the first entry that edits upstream's
# own files rather than only adding ours: an engine family touches the
# engine registry, the config schema, prepare, resolve and the docs.
# The FORK.md row carries the why and the drop-when; this list carries
# the blast radius. Keep the two in step.
allowed='^(FORK\.md|PATCHES\.md|\.fsh/|\.github/workflows/fsh-)'
allowed="$allowed"'|^(internal/db/mssql/|e2e/mssql/|internal/db/ident/mssql\.go)'
allowed="$allowed"'|^(go\.mod|go\.sum|justfile|README\.md|docs/|schemas/|e2e/README\.md)'
allowed="$allowed"'|^internal/(engine/engine(_test)?\.go|schema/schema\.go)'
allowed="$allowed"'|^internal/config/(config|connections|validate|connections_test|dsn_test)\.go'
allowed="$allowed"'|^internal/db/(engineconn/|reachability/reachability\.go)'
allowed="$allowed"'|^internal/prepare/(prepare|branchstate|branchreap|coldbuild_invariant_test)\.go'
allowed="$allowed"'|^internal/(resolve/resolve\.go|snapshot/orphans\.go|initgen/initgen\.go)'
allowed="$allowed"'|^internal/mcp/(tools_engine|tools_write|server)\.go'
allowed="$allowed"'|^cmd/treeman/cmd/(misc|doctor_engines)\.go'
base="$(git merge-base HEAD upstream/master)"
unexpected="$(git diff --name-only "$base"..HEAD | grep -Ev "$allowed" || true)"
if [ -n "$unexpected" ]; then
Expand Down Expand Up @@ -105,6 +125,47 @@ jobs:
python3 .fsh/osv.py compare .fsh/upstream-osv.json /tmp/osv.json \
>> "$GITHUB_STEP_SUMMARY"

# Build, vet and test. Deliberately NOT a copy of upstream's full ci.yml:
# no lint (the flake-pinned golangci-lint is a nix download this job's
# egress policy would have to admit, and a lint finding in upstream's tree
# is not ours to gate on), no macOS matrix, no e2e tier — e2e needs docker
# engines and is run by hand. What it does cover is the gate a patch into
# `fsh` otherwise has none of.
build-test:
runs-on: ubuntu-latest
steps:
- uses: step-security/harden-runner@ccd8616d44fd3846e67624a50d5aad6d37bf2d25 # v2.22.1
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
objects.githubusercontent.com:443
proxy.golang.org:443
release-assets.githubusercontent.com:443
storage.googleapis.com:443
sum.golang.org:443
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6
with:
go-version-file: go.mod
cache: true
- run: go build ./...
- run: go vet ./...
# The e2e suites are behind `//go:build e2e`, so the untagged vet above
# never type-checks them — a signature change reaches them only here.
- run: go vet -tags e2e ./e2e/...
- run: go test ./...
- name: gofmt
run: |
set -euo pipefail
unformatted="$(gofmt -l .)"
if [ -n "$unformatted" ]; then
echo "::error::gofmt would rewrite:"
echo "$unformatted"
exit 1
fi

# Has upstream's hygiene regressed since we last looked? A drop in
# Token-Permissions, Dangerous-Workflow or Pinned-Dependencies is exactly the
# shape a hostile or careless change to his release path would take.
Expand Down
25 changes: 20 additions & 5 deletions FORK.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ Our diff against upstream is **new files**, plus the smallest possible edits to
We do **not** delete the parts we don't use (the MCP server, the TUI, the MySQL/Mongo drivers,
`treeman git`). Dead code costs disk; a 20k-line deletion costs us every future merge.

A new engine family is the one shape that cannot honour "new files only" — the engine registry,
the config schema, prepare, resolve and the generated docs all have to learn it. The patch index
below names each such patch and `.github/workflows/fsh-vet.yml` carries the matching allowlist, so
the blast radius is written down in both places rather than discovered during a merge.

- `master` — untouched mirror of upstream. Never commit here; it only fast-forwards.
- `fsh` — our branch, and the one we release from. Upstream arrives by **pull request**
(`fsh-sync.yml`), never by an unattended rebase, so somebody reads the delta before we ship it.
Expand All @@ -23,14 +28,14 @@ We do **not** delete the parts we don't use (the MCP server, the TUI, the MySQL/
been abused — it is the same mutable-tag pattern most repos have, ours included — but it means
a published binary is not cryptographically tied to the source. Building it ourselves makes
that question moot.
2. **We need a Meilisearch driver**, which upstream has no support for and 7 of our 8 local
stacks require.
2. **We need drivers upstream does not have** — Meilisearch, which 7 of our 8 local stacks
require, and SQL Server, which one client's whole ERP is.
3. **Two safety fixes we would rather not wait for** (see the patch index).

Everything in (1) is fixable upstream with ~5 lines (`actions/attest-build-provenance`, pinning
`xilo@v1`, a `dist/` gitignore entry). We intend to offer those. **If upstream takes them, this
fork's reason to exist shrinks to the Meilisearch driver, and we should say so here and shrink
it.** A fork that outlives its justification is just technical debt with a remote.
fork's reason to exist shrinks to the drivers, and we should say so here and shrink it.** A fork
that outlives its justification is just technical debt with a remote.

## What we actually run

Expand All @@ -45,7 +50,7 @@ Every patch is one commit, and one row here. A row without a **Drop when** condi

| # | Patch | Why | Upstream | Drop when |
|---|---|---|---|---|
| — | _none yet_ | the branch currently carries only the additive files below | — | — |
| P4 | SQL Server (`mssql`) engine family | Groomershop's ERP is Subiekt GT on SQL Server. Its test tier runs against one shared container with hardcoded ids, so two worktrees — or two jest workers — cannot run it at once; Postgres gets per-worktree databases and per-worker clones from treeman, SQL Server got nothing. Upstream supports Postgres/MySQL/Mongo/Redis/ES-OpenSearch/S3 and has a clean driver interface to extend | offer once it has run a few weeks for us | upstream merges it |

Planned, not yet written:

Expand All @@ -68,6 +73,9 @@ Planned, not yet written:
| `.fsh/upstream-osv.json` | baseline of upstream's known-vulnerable dependencies |
| `.fsh/osv.py` | writes that baseline and compares against it |
| `.fsh/socket/go.mod` | a manifest nothing builds, so Socket scores upstream's own code — see [Vetting](#vetting) |
| `internal/db/mssql/` | the SQL Server driver (P4) |
| `internal/db/ident/mssql.go` | bracket-quoting for SQL Server identifiers (P4) |
| `e2e/mssql/` | its docker-tier suite: the prepare cycle, plus the driver's clone path, spare claim and rename race (P4) |

## Vetting

Expand Down Expand Up @@ -97,6 +105,13 @@ source imports no `x/crypto` package, so reachability is doubtful, and a `go get
clear all 37). A check that is red on day one teaches everyone to ignore it. These go red only when
something *moves*.

The one dependency we have added is `github.com/microsoft/go-mssqldb` (P4). It carries no OSV
advisory of its own, and it drags `golang.org/x/crypto` from 0.33.0 up to 0.55.0 — which clears 34
of the 37 advisories that single stale indirect contributes to the baseline. The three that remain
on 0.55.0 are all baselined already, so the deps check stays green without a re-baseline. Its
Azure, Key Vault and Kerberos modules are in its `go.mod` but not in ours: they belong to packages
we do not import, and module-graph pruning keeps them out of `go.sum`.

Two honest limits, so nobody mistakes a green run for an audit:

- **Scorecard measures practices, not intent.** 3.3/10 means an attack on this project would not
Expand Down
23 changes: 17 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@ databases, search indices, and parallel test clones for each git
worktree — torn down on delete, kept in sync as your migrations and
fixtures change. Language- and framework-agnostic.

Pure wire-protocol DB access (Go `database/sql` for MySQL +
PostgreSQL, the official Mongo / Redis / Elasticsearch SDKs) — no
shelling out to `mysql` / `psql` / `mongosh` / `redis-cli`. A single
Pure wire-protocol DB access (Go `database/sql` for MySQL,
PostgreSQL + SQL Server, the official Mongo / Redis / Elasticsearch
SDKs) — no shelling out to `mysql` / `psql` / `sqlcmd` / `mongosh` /
`redis-cli`. A single
user-mode daemon, a thin CLI client, and a SQLite-backed event log.

---
Expand All @@ -28,14 +29,24 @@ commands you type.
|---|---|---|
| MySQL | MariaDB, TiDB | Separate database |
| PostgreSQL | — | Separate database |
| SQL Server | — | Separate database |
| MongoDB | — | Separate database |
| Redis | Valkey, DragonflyDB | Key-prefix in DB 0 (cluster-safe, no 16-DB cap) |
| Elasticsearch | OpenSearch | Index-name prefix |

Variants are first-class aliases — declare `engine: mariadb`, `tidb`,
`valkey`, `dragonfly`, or `opensearch` and treeman routes it to the
parent engine's driver (Valkey/DragonflyDB ride the Redis wire
protocol).
`sqlserver`, `valkey`, `dragonfly`, or `opensearch` and treeman routes
it to the parent engine's driver (Valkey/DragonflyDB ride the Redis
wire protocol).

SQL Server clones through BACKUP + RESTORE, the only database copy a
client can drive there — so a template is the template database plus
one `.bak` file beside it, captured once and restored per clone. Two
consequences worth knowing before you declare one: the connection
needs BACKUP/RESTORE rights (`sysadmin` in practice, since restoring a
database that does not yet exist is sysadmin-only), and `dump:` is
rejected, because RESTORE reads a path on the *server* — load a dump
from `seed:` instead.

## Features

Expand Down
6 changes: 6 additions & 0 deletions cmd/treeman/cmd/doctor_engines.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,12 @@ func engineProbes(cfg *config.Config) []engineProbe {
engineProbe{name: "postgres", ref: p.ContainerRef, host: orDefault(p.Host, "127.0.0.1"), port: orDefaultPort(p.Port, 5432)},
)
}
if ms := c.Mssql; ms != nil {
out = append(
out,
engineProbe{name: "mssql", ref: ms.ContainerRef, host: orDefault(ms.Host, "127.0.0.1"), port: orDefaultPort(ms.Port, 1433)},
)
}
if mg := c.Mongodb; mg != nil {
out = append(out, engineProbe{name: "mongodb", ref: mg.ContainerRef, url: mg.URI})
}
Expand Down
10 changes: 6 additions & 4 deletions cmd/treeman/cmd/misc.go
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,7 @@ func DbCmd() *cli.Command {
&cli.StringFlag{Name: "repo", Aliases: []string{"r"}},
&cli.StringFlag{
Name: "engine",
Usage: "restrict the reset to one engine family (mysql, postgres, mongodb, redis, elasticsearch; aliases like mariadb/postgresql/valkey/dragonfly accepted)",
Usage: "restrict the reset to one engine family (mysql, postgres, mssql, mongodb, redis, elasticsearch; aliases like mariadb/postgresql/sqlserver/valkey/dragonfly accepted)",
},
&cli.BoolFlag{Name: "json"},
&cli.BoolFlag{
Expand Down Expand Up @@ -452,7 +452,7 @@ func DbCmd() *cli.Command {
&cli.StringFlag{Name: "repo", Aliases: []string{"r"}},
&cli.StringFlag{
Name: "engine",
Usage: "restrict the save to one engine family (mysql, postgres, mongodb, redis, elasticsearch; aliases like mariadb/postgresql/valkey/dragonfly accepted)",
Usage: "restrict the save to one engine family (mysql, postgres, mssql, mongodb, redis, elasticsearch; aliases like mariadb/postgresql/sqlserver/valkey/dragonfly accepted)",
},
&cli.BoolFlag{Name: "json"},
&cli.BoolFlag{
Expand Down Expand Up @@ -710,6 +710,7 @@ func ConfigCmd() *cli.Command {
_, _ = fmt.Fprintln(ui.Out, "# resolved connections")
printResolved("mysql", r.Mysql)
printResolved("postgres", r.Postgres)
printResolved("mssql", r.Mssql)
printResolved("mongodb", r.Mongodb)
printResolved("redis", r.Redis)
printResolved("elasticsearch", r.Elasticsearch)
Expand Down Expand Up @@ -1478,7 +1479,7 @@ func InitCmd() *cli.Command {
},
&cli.StringFlag{
Name: "engine",
Usage: "comma-separated engines to activate in databases: (e.g. mysql,postgres,redis) — emits minimal blocks when detection found no framework",
Usage: "comma-separated engines to activate in databases: (e.g. mysql,postgres,mssql,redis) — emits minimal blocks when detection found no framework",
},
&cli.BoolFlag{
Name: "interactive",
Expand Down Expand Up @@ -1637,11 +1638,12 @@ func pickEngines() ([]string, error) {
items := []string{
"mysql — MySQL / MariaDB / TiDB",
"postgres — PostgreSQL",
"mssql — Microsoft SQL Server",
"mongodb — MongoDB",
"redis — Redis / Valkey / Dragonfly",
"elasticsearch — Elasticsearch / OpenSearch",
}
values := []string{"mysql", "postgres", "mongodb", "redis", "elasticsearch"}
values := []string{"mysql", "postgres", "mssql", "mongodb", "redis", "elasticsearch"}
res, err := tui.Select(items, tui.Options{
Prompt: "engines to activate in databases: (space marks, enter confirms, esc = none)",
Values: values,
Expand Down
6 changes: 3 additions & 3 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -523,7 +523,7 @@ re-sync branch_scoped databases from the live base branch (defaults to the cwd's
| Flag | Usage |
|---|---|
| `-r`, `--repo` | |
| `--engine` | restrict the reset to one engine family (mysql, postgres, mongodb, redis, elasticsearch; aliases like mariadb/postgresql/valkey/dragonfly accepted) |
| `--engine` | restrict the reset to one engine family (mysql, postgres, mssql, mongodb, redis, elasticsearch; aliases like mariadb/postgresql/sqlserver/valkey/dragonfly accepted) |
| `--json` | |
| `--dry-run` | list the branch_scoped namespaces that would be dropped + re-seeded; change nothing, connect nowhere |
| `-f`, `--wait`, `--foreground` | stream the daemon's live progress and block until done (default: dispatch and return) |
Expand All @@ -535,7 +535,7 @@ capture branch_scoped databases into the current branch's durable copies without
| Flag | Usage |
|---|---|
| `-r`, `--repo` | |
| `--engine` | restrict the save to one engine family (mysql, postgres, mongodb, redis, elasticsearch; aliases like mariadb/postgresql/valkey/dragonfly accepted) |
| `--engine` | restrict the save to one engine family (mysql, postgres, mssql, mongodb, redis, elasticsearch; aliases like mariadb/postgresql/sqlserver/valkey/dragonfly accepted) |
| `--json` | |
| `-f`, `--wait`, `--foreground` | stream the daemon's live progress and block until done (default: dispatch and return) |

Expand Down Expand Up @@ -866,7 +866,7 @@ print the slug treeman derives for a worktree
| `--force` | |
| `--json` | |
| `--global` | scaffold the user-global ~/.config/treeman/config.yaml (machine-wide defaults) instead of a per-repo .treeman.yaml |
| `--engine` | comma-separated engines to activate in databases: (e.g. mysql,postgres,redis) — emits minimal blocks when detection found no framework |
| `--engine` | comma-separated engines to activate in databases: (e.g. mysql,postgres,mssql,redis) — emits minimal blocks when detection found no framework |
| `--interactive` | pick engines from a terminal picker instead of --engine (declines gracefully when stdin is not a TTY) |

### `treeman doctor`
Expand Down
13 changes: 12 additions & 1 deletion docs/config-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ daemon:
connections:
mysql: '...'
postgres: '...'
mssql: '...'
mongodb: '...'
redis: '...'
elasticsearch: '...'
Expand Down Expand Up @@ -154,6 +155,7 @@ include:
connections:
mysql: '...'
postgres: '...'
mssql: '...'
mongodb: '...'
redis: '...'
elasticsearch: '...'
Expand Down Expand Up @@ -530,6 +532,11 @@ create clones, dump templates.
PostgreSQL connection. Required when any `databases:` entry
uses `engine: postgres` (or the `postgresql` alias).

#### `mssql` *(one of: [MssqlConn](#mssqlconn), map of name → [MssqlConn](#mssqlconn))*

Microsoft SQL Server connection. Required when any `databases:`
entry uses `engine: mssql` (or the `sqlserver` alias).

#### `mongodb` *(one of: [MongoConn](#mongoconn), map of name → [MongoConn](#mongoconn))*

MongoDB connection. URI form (`mongodb://...`); host/port get
Expand Down Expand Up @@ -662,7 +669,7 @@ for `redis` (same wire protocol, same key-prefix scoping).
The rendered schema's enum is injected from engine.Known by
schema.Reflect, so the alias list lives in exactly one place.

_Allowed: `mysql`, `mariadb`, `tidb`, `postgres`, `postgresql`, `mongodb`, `redis`, `valkey`, `dragonfly`, `elasticsearch`, `opensearch`, `s3`, `sqlite`, `duckdb`_
_Allowed: `mysql`, `mariadb`, `tidb`, `postgres`, `postgresql`, `mssql`, `sqlserver`, `mongodb`, `redis`, `valkey`, `dragonfly`, `elasticsearch`, `opensearch`, `s3`, `sqlite`, `duckdb`_

#### `connection` *(string)*

Expand Down Expand Up @@ -1091,6 +1098,10 @@ declared count are a config error caught at load time.

mongodb connection — bare URL string OR structured object.

### MssqlConn

Microsoft SQL Server connection — bare DSN string OR structured object.

### MysqlConn

MySQL connection — bare DSN string OR structured object.
Expand Down
Loading
Loading