diff --git a/.changeset/validate-skill-references.md b/.changeset/validate-skill-references.md new file mode 100644 index 00000000..259da828 --- /dev/null +++ b/.changeset/validate-skill-references.md @@ -0,0 +1,5 @@ +--- +'@tanstack/intent': patch +--- + +`intent validate` now checks every Markdown file under a skill's `references/` directory: the file must not begin with frontmatter, the `SKILL.md` body must link to it directly, and its TypeScript and JavaScript examples are checked with the skill's examples. A `skill_tree.yaml` entry for a skill with reference files must list them in a `references` key, and the list must match the files. Existing repositories now fail validation for reference files that `SKILL.md` does not link to directly, reference frontmatter, reference examples that do not typecheck, and tree entries that do not list their skill's reference files. diff --git a/docs/cli/intent-validate.md b/docs/cli/intent-validate.md index 917ce973..b331e6f7 100644 --- a/docs/cli/intent-validate.md +++ b/docs/cli/intent-validate.md @@ -100,6 +100,39 @@ TypeScript 5.0 or newer must be available in the repository for code checking. I Relative Markdown links outside fenced examples must point to an existing file or directory. External URLs and anchors are not checked. Link checks still run when TypeScript is unavailable. Repeated validations read current source files and link targets. +### Reference files + +Every `.md` file under a skill's `references/` directory, at any depth, is checked with its skill. No planning artifacts are required. + +- The file is plain Markdown: it does not begin with YAML frontmatter +- The body of the owning `SKILL.md` links to the file with a direct relative Markdown link outside fenced examples, such as `[retry behavior](references/retries.md)`. A path in backticks or a link from another reference file does not count, because an agent finds a reference only through that link. A reference file can also link to another reference file +- TypeScript and JavaScript fences are checked with the skill's fences, under the rules in [Code examples and links](#code-examples-and-links). Errors name the reference file and line. Relative links inside a reference file are not checked + +A `skill_tree.yaml` entry lists the skill's reference files in a `references` key, as paths relative to the skill directory: + +```yaml +skills: + - name: React Table State + slug: table-state + package: packages/react-table + path: skills/table-state/SKILL.md + references: + - references/reactivity.md +``` + +Intent reads the tree from `/_artifacts` and, in a monorepo, from `_artifacts` at the workspace root. It matches an entry to a skill by the resolved `path`, so two packages can use the same slug. When a skill has a tree entry, the entry must agree with the files: + +- An entry for a skill that has reference files has a `references` key. The error shows the YAML lines to add +- `references`, when present, is a list of strings + +- Each path has the form `references/.md`, stays inside the skill directory (no absolute path and no `..` segment), and is listed once +- Each listed file exists +- Each `.md` file under `references/` is listed + +A skill without reference files needs no `references` key. A skill without a tree entry, or a repository without planning artifacts, gets only the file checks above. + +Upgrade impact: a repository that already has reference files fails validation when a reference file is not linked directly from its `SKILL.md`, begins with frontmatter, or contains a code example that does not typecheck, or when a matching `skill_tree.yaml` entry does not list the skill's reference files. + ### Artifacts When `/_artifacts` exists, Intent also checks: diff --git a/packages/intent/meta/generate-skill/SKILL.md b/packages/intent/meta/generate-skill/SKILL.md index 81b6e4fa..8ac783d7 100644 --- a/packages/intent/meta/generate-skill/SKILL.md +++ b/packages/intent/meta/generate-skill/SKILL.md @@ -19,7 +19,7 @@ Work in the maintainer's library repository with their existing coding agent. Pr Use the current conversation, supplied diff, identified code/docs change, review report, or selected skill tree entry. Reuse the task, decisions, and evidence already established in this session; do not ask the maintainer to paste them again. Identify what developers need help doing and which package owns that task. If there is no usable task or review input, ask one question: “What do developers need help doing with this library?” Wait for the answer. -Check repository instructions and Git status before edits; preserve unrelated changes. Read [maintainer commands](references/maintainer-commands.md) when setting up the repository, adding a skill, or preparing a batch for review. Use those commands for registration, metadata synchronization, and completion checks. Discover the package name, version, repository, skill root, and vocabulary from the repository. Use an established custom root; otherwise use `skills/` inside the owning package, including in monorepos. +Check repository instructions and Git status before edits; preserve unrelated changes. Read [maintainer commands](references/maintainer-commands.md) when setting up the repository, adding a skill, or preparing a batch for review. When installing or updating CI, read [workflow security](references/workflow-security.md) before changing pins, triggers, or permissions. Use those commands for registration, metadata synchronization, and completion checks. Discover the package name, version, repository, skill root, and vocabulary from the repository. Use an established custom root; otherwise use `skills/` inside the owning package, including in monorepos. Before choosing or changing repository exports, follow [repository distribution](references/distribution.md). For an export-only change, complete that procedure against existing skill records; use the authoring steps below only if guidance must change. diff --git a/packages/intent/meta/generate-skill/references/planning-records.md b/packages/intent/meta/generate-skill/references/planning-records.md index 89a8133e..2a6f8e12 100644 --- a/packages/intent/meta/generate-skill/references/planning-records.md +++ b/packages/intent/meta/generate-skill/references/planning-records.md @@ -35,11 +35,13 @@ skills: purpose: '[descriptive explanation; synchronized from metadata.purpose]' requires: ['[other skill slugs]'] # omit if none sources: ['src/[path].ts', 'owner/repo:docs/[path].md'] + references: ['references/[file].md'] # required when the skill has reference files; relative to the skill directory ``` - `domain_map.yaml` owns the domain/task relationships, supported failure modes, cross-references, tensions, and knowledge gaps. Keep task slugs and package ownership aligned with the skills. - `skill_spec.md` is the human-readable coverage and decision record. Retain the existing inventories and add a **Coverage and batch history** section recording the assessed scope, each completed batch or behavior change, its source revision/version, consequential decisions and reasons, check outcomes, and remaining work. Keep entries concise; do not store transcripts. Distinguish implemented guidance from planned work and unassessed areas. - `skill_tree.yaml` owns skill identities, paths, prerequisites, source mappings, and package placement. Keep existing planned entries and their status or explanatory notes. Add each new skill, and update entries when their names, paths, dependencies, or sources change. Resolve `path` relative to the owning package when `package` is present; otherwise resolve it relative to the library root. Keep `generated_from` links accurate for the actual artifact location. +- `references` lists the `.md` files under the skill's `references/` directory. `intent validate` requires the key when the skill has reference files, and requires the list to match those files: each path has the form `references/.md`, is listed once, and exists, and no file is left out. Omit the key, or use an empty list, for a skill without reference files. `intent validate` also checks every reference file as described in [the skill format](skill-format.md). Keep each tree entry's `description` aligned with its skill's activation description and its `purpose` aligned with `metadata.purpose`. Existing map/spec task explanations retain their meaning; do not replace them with trigger text. Record a purpose migration and its preserved source text in the batch review without discarding prior decisions. diff --git a/packages/intent/meta/generate-skill/references/skill-format.md b/packages/intent/meta/generate-skill/references/skill-format.md index 56355c6e..541828a7 100644 --- a/packages/intent/meta/generate-skill/references/skill-format.md +++ b/packages/intent/meta/generate-skill/references/skill-format.md @@ -61,6 +61,8 @@ Example pointer (create the reference only for a real retry branch): When configuring retries, read [retry behavior](references/retries.md) before choosing a policy. ``` +`intent validate` checks every `.md` file under `references/`. Write each reference as plain Markdown without frontmatter. Link to it from the `SKILL.md` body with a direct relative Markdown link; a link from another reference file is not sufficient. Its TypeScript and JavaScript fences are checked with the skill's fences. + For a verification task (security, go-live, migration audit), use checks instead of setup/patterns: state what to inspect, the expected result, the failure condition, and the remediation. Include sourced mistakes and a final completion check. Use the applicable checklist template in [tree-generator's writing reference](../../tree-generator/references/write-skills.md#step-7--write-checklistaudit-skills-where-applicable) only when that detailed format is needed. For an entry selected from a full-library tree, retain its package placement, dependencies, failure-mode status, and cross-skill relationships. Read the applicable type template in [the tree writing reference](../../tree-generator/references/write-skills.md) when generating overview registries, framework trees, compositions, or cross-domain tension notes. The focused procedure remains in [generate-skill](../SKILL.md). diff --git a/packages/intent/src/commands/validate.ts b/packages/intent/src/commands/validate.ts index 099f5efb..5448ed9d 100644 --- a/packages/intent/src/commands/validate.ts +++ b/packages/intent/src/commands/validate.ts @@ -1,10 +1,21 @@ -import { appendFileSync, existsSync, readFileSync } from 'node:fs' +import { + appendFileSync, + existsSync, + readFileSync, + readdirSync, + statSync, +} from 'node:fs' import { basename, dirname, join, relative, resolve } from 'node:path' import { fail, isCliFailure } from '../shared/cli-error.js' import { resolveProjectContext } from '../core/project-context.js' import { findWorkspacePackages } from '../setup/workspace-patterns.js' import { createIntentFsCache } from '../discovery/fs-cache.js' -import { checkSkillBlocks, summarizeSkillExamples } from '../validate/blocks.js' +import { + checkSkillBlocks, + skillLinks, + summarizeSkillExamples, +} from '../validate/blocks.js' +import { readIntentArtifacts } from '../staleness/artifact-coverage.js' import { writeChanges } from '../maintainer/files.js' import { repositoryWritePath } from '../shared/write-path.js' import { @@ -15,6 +26,7 @@ import { printWarnings } from './support.js' import type { SkillBlockCache } from '../validate/blocks.js' import type { FileChange } from '../maintainer/files.js' import type { ProjectContext } from '../core/project-context.js' +import type { IntentArtifactSkill } from '../shared/types.js' interface ValidationError { file: string @@ -184,6 +196,47 @@ function collectPackagingWarnings( return warnings } +function findReferenceFiles(skillDir: string): Array { + const referencesDir = join(skillDir, 'references') + if (!statSync(referencesDir, { throwIfNoEntry: false })?.isDirectory()) + return [] + return readdirSync(referencesDir, { recursive: true, withFileTypes: true }) + .filter((entry) => entry.isFile() && entry.name.endsWith('.md')) + .map((entry) => + relative(skillDir, join(entry.parentPath, entry.name)).replaceAll( + '\\', + '/', + ), + ) + .sort() +} + +function readTreeEntries( + context: ProjectContext, + skillsDir: string, + artifacts: Map>, +): Map { + const entries = new Map() + for (const [artifactRoot, root] of [ + [skillsDir, context.packageRoot ?? context.cwd], + [context.workspaceRoot, context.workspaceRoot], + ]) { + if (!artifactRoot || !root) continue + if (!artifacts.has(artifactRoot)) + artifacts.set(artifactRoot, readIntentArtifacts(artifactRoot)) + for (const entry of artifacts.get(artifactRoot)?.skills ?? []) { + if (entry.artifactKind !== 'skill-tree' || !entry.path) continue + entries.set( + entry.package && !entry.path.startsWith(`${entry.package}/`) + ? resolve(root, entry.package, entry.path) + : resolve(root, entry.path), + entry, + ) + } + } + return entries +} + function formatWarning({ file, message }: ValidationWarning): string { return `${file}: ${message}` } @@ -404,6 +457,7 @@ async function runValidateCommandInternal( const setVersionPlans: Array = [] let validatedCount = 0 const validatedFiles = new Set() + const artifacts = new Map>() if (skillsDirs.length === 0) { console.log('No skills/ directory found — skipping validation.') @@ -425,7 +479,9 @@ async function runValidateCommandInternal( file: string content: string library: string | undefined + references: Array<{ file: string; content: string }> }> = [] + const treeEntries = readTreeEntries(validateContext, skillsDir, artifacts) for (const filePath of skillFiles) { const rel = relative(process.cwd(), filePath) const content = readFileSync(filePath, 'utf8') @@ -538,10 +594,101 @@ async function runValidateCommandInternal( ...collectAgentSkillSpecWarnings({ fm, rel }).map(formatWarning), ) + const skillDir = dirname(filePath) + const skillName = basename(skillDir) + const referenceFiles = findReferenceFiles(skillDir) + const linked = new Set( + referenceFiles.length + ? skillLinks(match[2]!).map((link) => resolve(skillDir, link.path)) + : [], + ) + const references = referenceFiles.map((reference) => { + const referencePath = join(skillDir, reference) + const file = relative(process.cwd(), referencePath) + const referenceContent = readFileSync(referencePath, 'utf8') + if ( + /^---\r?\n(?:[\s\S]*?\r?\n)?---[ \t]*(?:\r?\n|$)/.test( + referenceContent, + ) + ) + errors.push({ + file, + message: + 'Reference files must be plain Markdown without YAML frontmatter', + }) + if (!linked.has(referencePath)) + errors.push({ + file, + message: `Not linked from SKILL.md — add a direct relative Markdown link such as [${basename(reference, '.md')}](${reference}) so an agent can find it`, + }) + return { file, content: referenceContent } + }) + + const declared = treeEntries.get(filePath)?.references + if (declared === undefined) { + if (treeEntries.has(filePath) && referenceFiles.length) + errors.push({ + file: rel, + message: `The skill_tree.yaml entry for skill "${skillName}" must list its reference files. Add to the entry:\n${[ + 'references:', + ...referenceFiles.map((reference) => ` - ${reference}`), + ] + .map((line) => ` ${line}`) + .join('\n')}`, + }) + } else if ( + !Array.isArray(declared) || + declared.some((reference) => typeof reference !== 'string') + ) { + errors.push({ + file: rel, + message: `The "references" value of the skill_tree.yaml entry for skill "${skillName}" must be a list of reference file paths`, + }) + } else { + const seen = new Set() + for (const reference of declared as Array) { + if (seen.has(reference)) { + errors.push({ + file: rel, + message: `Reference "${reference}" is declared more than once in skill_tree.yaml`, + }) + continue + } + seen.add(reference) + if ( + !/^references\/.+\.md$/.test(reference) || + reference.includes('\\') || + reference + .split('/') + .some((part) => part === '' || part === '.' || part === '..') + ) { + errors.push({ + file: rel, + message: `Declared reference "${reference}" must be a references/.md path inside the skill directory`, + }) + } else if (!existsSync(join(skillDir, reference))) { + errors.push({ + file: relative(process.cwd(), join(skillDir, reference)), + message: + 'Declared in skill_tree.yaml but the file does not exist', + }) + } + } + for (const reference of referenceFiles) { + if (!seen.has(reference)) + errors.push({ + file: relative(process.cwd(), join(skillDir, reference)), + message: + 'Not declared in the "references" list of its skill_tree.yaml entry', + }) + } + } + checkedSkills.push({ file: rel, content, library: readScalarField(fm, 'library'), + references, }) const lineCount = content.split(/\r?\n/).length @@ -576,6 +723,7 @@ async function runValidateCommandInternal( packageDir: validateContext.packageRoot, library, skills, + references: skills.flatMap((skill) => skill.references), cache: blockCache, }) if (exampleSummaries) diff --git a/packages/intent/src/shared/types.ts b/packages/intent/src/shared/types.ts index 76322aa7..f2e1a388 100644 --- a/packages/intent/src/shared/types.ts +++ b/packages/intent/src/shared/types.ts @@ -136,6 +136,7 @@ export interface IntentArtifactSkill { packages: Array sources: Array covers: Array + references?: unknown } export interface IntentArtifactCoverageIgnore { diff --git a/packages/intent/src/staleness/artifact-coverage.ts b/packages/intent/src/staleness/artifact-coverage.ts index 9c2f2a6a..5084345e 100644 --- a/packages/intent/src/staleness/artifact-coverage.ts +++ b/packages/intent/src/staleness/artifact-coverage.ts @@ -97,6 +97,7 @@ function parseSkills( packages: [...new Set(packages)].sort((a, b) => a.localeCompare(b)), sources: stringArray(skill.sources), covers: stringArray(skill.covers), + ...('references' in skill ? { references: skill.references } : {}), }) } diff --git a/packages/intent/src/validate/blocks.ts b/packages/intent/src/validate/blocks.ts index 2a232857..d1befa2b 100644 --- a/packages/intent/src/validate/blocks.ts +++ b/packages/intent/src/validate/blocks.ts @@ -432,13 +432,10 @@ async function splitExamples( return { content: lines.join(''), suggestions } } -function checkSkillLinks( - root: string, - file: string, +export function skillLinks( content: string, -): Array { - const findings: Array = [] - const absolute = resolve(root, file) +): Array<{ target: string; path: string; line: number }> { + const links: Array<{ target: string; path: string; line: number }> = [] // Blank out fenced examples, keeping newlines so line numbers still match. const lines = content.split(/\r?\n/) for (const fence of codeFences(content)) @@ -454,10 +451,27 @@ function checkSkillLinks( if (/^[a-z][a-z0-9+.-]*:/i.test(target) || target.startsWith('#')) continue const path = target.replace(/[#?].*$/, '') if (!path) continue + links.push({ + target, + path, + line: prose.slice(0, match.index).split('\n').length, + }) + } + return links +} + +function checkSkillLinks( + root: string, + file: string, + content: string, +): Array { + const findings: Array = [] + const absolute = resolve(root, file) + for (const { target, path, line } of skillLinks(content)) { if (!existsSync(resolve(dirname(absolute), path))) findings.push({ file, - line: prose.slice(0, match.index).split('\n').length, + line, message: `Link target not found: ${target}`, severity: 'error', }) @@ -610,6 +624,7 @@ export async function checkSkillBlocks( packageDir: string library: string skills: Array<{ file: string; content: string }> + references?: Array<{ file: string; content: string }> cache?: SkillBlockCache }, ts?: typeof TS | null, @@ -618,8 +633,8 @@ export async function checkSkillBlocks( const findings = options.skills.flatMap((skill) => checkSkillLinks(root, skill.file, skill.content), ) - const blocks = options.skills.flatMap((skill) => - extractCodeBlocks(skill.file, skill.content), + const blocks = [...options.skills, ...(options.references ?? [])].flatMap( + (skill) => extractCodeBlocks(skill.file, skill.content), ) const result = (skipped?: string): SkillBlockCheck => ({ blocks: blocks.length, diff --git a/packages/intent/tests/cli.test.ts b/packages/intent/tests/cli.test.ts index 01c6f6a2..a31c764e 100644 --- a/packages/intent/tests/cli.test.ts +++ b/packages/intent/tests/cli.test.ts @@ -3595,6 +3595,282 @@ describe('cli commands', () => { expect(output).toContain('"!skills/_artifacts" is not in the "files" array') }) + describe('validate reference files', () => { + function writeReferenceFixture({ + body, + references = {}, + tree, + }: { + body: string + references?: Record + tree?: string + }): void { + const root = mkdtempSync(join(realTmpdir, 'intent-cli-validate-refs-')) + tempDirs.push(root) + writeJson(join(root, 'package.json'), { + name: '@acme/library', + devDependencies: { '@tanstack/intent': '^0.0.18' }, + keywords: ['tanstack-intent'], + files: ['skills', '!skills/_artifacts'], + }) + mkdirSync(join(root, 'src')) + writeFileSync( + join(root, 'src', 'index.ts'), + 'export declare function retry(options: { max: number }): void\n', + ) + const skillDir = join(root, 'skills', 'core') + mkdirSync(skillDir, { recursive: true }) + writeFileSync( + join(skillDir, 'SKILL.md'), + `---\nname: core\ndescription: Core guidance\n---\n\n${body}\n`, + ) + for (const [path, content] of Object.entries(references)) { + mkdirSync(dirname(join(skillDir, path)), { recursive: true }) + writeFileSync(join(skillDir, path), content) + } + if (tree !== undefined) { + const artifacts = join(root, 'skills', '_artifacts') + mkdirSync(artifacts) + writeFileSync(join(artifacts, 'domain_map.yaml'), 'skills: []\n') + writeFileSync(join(artifacts, 'skill_spec.md'), '# Spec\n') + writeFileSync( + join(artifacts, 'skill_tree.yaml'), + `skills:\n - name: Core\n slug: core\n path: skills/core/SKILL.md\n${tree}`, + ) + } + process.chdir(root) + } + + const retries = join('skills', 'core', 'references', 'retries.md') + const linked = 'Read [retries](references/retries.md) before retrying.' + const passed = '✅ Validated 1 skill files — all passed' + + it('rejects a reference file that begins with frontmatter', async () => { + writeReferenceFixture({ + body: linked, + references: { + 'references/retries.md': '---\nname: retries\n---\n\n# Retries\n', + }, + }) + + expect(await main(['validate'])).toBe(1) + expect(errorSpy.mock.calls.flat().join('\n')).toContain( + `${retries}: Reference files must be plain Markdown without YAML frontmatter`, + ) + }) + + it('rejects a reference file that SKILL.md does not link to', async () => { + writeReferenceFixture({ + body: 'See `references/retries.md` and\n\n```md\n[retries](references/retries.md)\n```', + references: { 'references/retries.md': '# Retries\n' }, + }) + + expect(await main(['validate'])).toBe(1) + expect(errorSpy.mock.calls.flat().join('\n')).toContain( + `${retries}: Not linked from SKILL.md — add a direct relative Markdown link such as [retries](references/retries.md) so an agent can find it`, + ) + }) + + it('rejects a reference file linked only from another reference', async () => { + writeReferenceFixture({ + body: linked, + references: { + 'references/retries.md': 'Then read [backoff](backoff.md).\n', + 'references/backoff.md': '# Backoff\n', + }, + }) + + expect(await main(['validate'])).toBe(1) + const errors = errorSpy.mock.calls.flat().join('\n') + expect(errors).toContain('Validation failed with 1 error(s)') + expect(errors).toContain( + `${join('skills', 'core', 'references', 'backoff.md')}: Not linked from SKILL.md — add a direct relative Markdown link such as [backoff](references/backoff.md) so an agent can find it`, + ) + }) + + it('accepts references that link to each other when SKILL.md links to both', async () => { + writeReferenceFixture({ + body: `${linked} Then read [backoff](references/backoff.md).`, + references: { + 'references/retries.md': 'Then read [backoff](backoff.md).\n', + 'references/backoff.md': '# Backoff\n', + }, + }) + + expect(await main(['validate'])).toBe(0) + expect(logSpy.mock.calls.flat().join('\n')).toBe(passed) + }) + + it('typechecks code blocks in a reference file', async () => { + writeReferenceFixture({ + body: linked, + references: { + 'references/retries.md': + "# Retries\n\n```ts\nimport { retry } from '@acme/library'\nretry({ max: 'many' })\n```\n", + }, + }) + + expect(await main(['validate'])).toBe(1) + expect(errorSpy.mock.calls.flat().join('\n')).toMatch( + /references[\\/]retries\.md:5: TS2322/, + ) + }) + + it('rejects declared reference paths outside references/ and duplicates', async () => { + writeReferenceFixture({ + body: linked, + references: { 'references/retries.md': '# Retries\n' }, + tree: ' references:\n - references/retries.md\n - references/retries.md\n - ../other/SKILL.md\n - /tmp/notes.md\n - references/../notes.md\n - references/notes.txt\n', + }) + + expect(await main(['validate'])).toBe(1) + const errors = errorSpy.mock.calls.flat().join('\n') + const skill = join('skills', 'core', 'SKILL.md') + expect(errors).toContain('Validation failed with 5 error(s)') + expect(errors).toContain( + `${skill}: Reference "references/retries.md" is declared more than once in skill_tree.yaml`, + ) + for (const path of [ + '../other/SKILL.md', + '/tmp/notes.md', + 'references/../notes.md', + 'references/notes.txt', + ]) + expect(errors).toContain( + `${skill}: Declared reference "${path}" must be a references/.md path inside the skill directory`, + ) + }) + + it('rejects a declared reference file that does not exist', async () => { + writeReferenceFixture({ + body: linked, + references: { 'references/retries.md': '# Retries\n' }, + tree: ' references:\n - references/retries.md\n - references/caching.md\n', + }) + + expect(await main(['validate'])).toBe(1) + const errors = errorSpy.mock.calls.flat().join('\n') + expect(errors).toContain('Validation failed with 1 error(s)') + expect(errors).toContain( + `${join('skills', 'core', 'references', 'caching.md')}: Declared in skill_tree.yaml but the file does not exist`, + ) + }) + + it('rejects a reference file missing from a declared references list', async () => { + writeReferenceFixture({ + body: `${linked} Then read [caching](references/caching.md).`, + references: { + 'references/retries.md': '# Retries\n', + 'references/caching.md': '# Caching\n', + }, + tree: ' references:\n - references/retries.md\n', + }) + + expect(await main(['validate'])).toBe(1) + const errors = errorSpy.mock.calls.flat().join('\n') + expect(errors).toContain('Validation failed with 1 error(s)') + expect(errors).toContain( + `${join('skills', 'core', 'references', 'caching.md')}: Not declared in the "references" list of its skill_tree.yaml entry`, + ) + }) + + it('accepts declared, linked, plain references whose examples compile', async () => { + writeReferenceFixture({ + body: `${linked} Then read [caching](./references/nested/caching.md#keys).`, + references: { + 'references/retries.md': + "# Retries\n\n```ts\nimport { retry } from '@acme/library'\nretry({ max: 3 })\n```\n", + 'references/nested/caching.md': '# Caching\n\n---\n\n## Keys\n', + }, + tree: ' references:\n - references/retries.md\n - references/nested/caching.md\n', + }) + + expect(await main(['validate'])).toBe(0) + expect(logSpy.mock.calls.flat().join('\n')).toBe(passed) + }) + + it('rejects a tree entry that does not list the reference files of its skill', async () => { + writeReferenceFixture({ + body: `${linked} Then read [caching](references/caching.md).`, + references: { + 'references/retries.md': '# Retries\n', + 'references/caching.md': '# Caching\n', + }, + tree: '', + }) + + expect(await main(['validate'])).toBe(1) + const errors = errorSpy.mock.calls.flat().join('\n') + expect(errors).toContain('Validation failed with 1 error(s)') + expect(errors).toContain( + `${join('skills', 'core', 'SKILL.md')}: The skill_tree.yaml entry for skill "core" must list its reference files. Add to the entry:\n references:\n - references/caching.md\n - references/retries.md`, + ) + }) + + it.each(['references/retries.md', '[{ path: references/retries.md }]'])( + 'rejects a tree entry whose references value is %s', + async (value) => { + writeReferenceFixture({ + body: linked, + references: { 'references/retries.md': '# Retries\n' }, + tree: ` references: ${value}\n`, + }) + + expect(await main(['validate'])).toBe(1) + const errors = errorSpy.mock.calls.flat().join('\n') + expect(errors).toContain('Validation failed with 1 error(s)') + expect(errors).toContain( + `${join('skills', 'core', 'SKILL.md')}: The "references" value of the skill_tree.yaml entry for skill "core" must be a list of reference file paths`, + ) + }, + ) + + it('leaves a skill without a references directory unaffected', async () => { + writeReferenceFixture({ body: 'Skill content here.', tree: '' }) + + expect(await main(['validate'])).toBe(0) + expect(logSpy.mock.calls.flat().join('\n')).toBe(passed) + }) + + it('matches tree entries by path when two packages share a slug', async () => { + const root = mkdtempSync(join(realTmpdir, 'intent-cli-validate-refs-')) + tempDirs.push(root) + writeJson(join(root, 'package.json'), { private: true }) + writeFileSync( + join(root, 'pnpm-workspace.yaml'), + 'packages:\n - packages/*\n', + ) + mkdirSync(join(root, '_artifacts')) + writeFileSync( + join(root, '_artifacts', 'skill_tree.yaml'), + 'skills:\n - slug: state\n package: packages/react\n path: packages/react/skills/state/SKILL.md\n references:\n - references/reactivity.md\n - slug: state\n package: packages/solid\n path: skills/state/SKILL.md\n references: []\n', + ) + for (const name of ['react', 'solid']) { + const skillDir = join(root, 'packages', name, 'skills', 'state') + writeJson(join(root, 'packages', name, 'package.json'), { + name: `@acme/${name}`, + }) + mkdirSync(join(skillDir, 'references'), { recursive: true }) + writeFileSync( + join(skillDir, 'SKILL.md'), + '---\nname: state\ndescription: State guidance\n---\n\nRead [reactivity](references/reactivity.md).\n', + ) + writeFileSync( + join(skillDir, 'references', 'reactivity.md'), + '# Reactivity\n', + ) + } + process.chdir(root) + + expect(await main(['validate'])).toBe(1) + const errors = errorSpy.mock.calls.flat().join('\n') + expect(errors).toContain('Validation failed with 1 error(s)') + expect(errors).toContain( + `${join('packages', 'solid', 'skills', 'state', 'references', 'reactivity.md')}: Not declared in the "references" list of its skill_tree.yaml entry`, + ) + }) + }) + it('skips cleanly when validate is run without a skills directory', async () => { const root = mkdtempSync(join(realTmpdir, 'intent-cli-missing-skills-')) tempDirs.push(root)