Conversation
Add an advisory `surface` label to every reported finding: `code`, `instructions` (`SKILL.md`), `docs`, `tests`, `comments`, or `config`. It is derived from the finding's path, plus the matched line for the `comments` case, and it is exposed rather than filtered on: severity, confidence, the risk score, and the finding count are unchanged. Both report formats carry it -- JSON `issues[].surface` and SARIF `properties.surface` -- so a consumer can sort a report by surface, diff two scans per surface, or write a local suppression policy that takes on the risk explicitly instead of the scanner taking it on their behalf. Exact-match dedup now records a label per occurrence, so aggregating the same match across files does not mislabel the occurrences that live elsewhere. Fixes NVIDIA#326. Signed-off-by: Harbor404 <2657212322@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #326.
Reports said which file and line a finding landed on, but not which surface
that line belongs to. The issue author manually labelled all 4415 findings from
65 real skill packages: code 1442, tests 981, docs 980, comments 412,
instructions 325, config 275.
The request is to expose, not filter. The reasoning is sound:
tests/lookslike noise but is also one of the easiest places to hide a payload, suppressing
it would trade real detections for cosmetics; a rule like SC1 hits config 100%
of the time and config is exactly where it should fire; and
instructionsmust stay distinct from
docsbecauseSKILL.mdis what the agent executes,not prose about it. Making the classification data lets consumers own their own
ranking and suppression risk, rather than baking that judgement into the
scanner.
Changes
src/skillspector/surface.py—infer_surface(file_path, line_text=None)returningcode | instructions | docs | tests | comments | config. Precedence:tests→instructions(SKILL.md) →comments→docs→config→code.commentsonly applies withincode/config: a comment inside a test fileis still
tests, and an HTML comment inREADME.mdis stilldocs.docs, sorequirements.txtis notread as prose because of
.txt.scripts/install.shis not read as a docbecause its name contains
install.models.py—Finding.surface, emitted byto_dict().nodes/analyzers/static_runner.py— pass the matched line text intoinfer_surface.nodes/deduplicate.py— record a surface per occurrence.nodes/report.py—properties.surfacein SARIF; expansion uses eachoccurrence's own label.
tests/test_finding_surface.py— 34 parametrised cases.A real bug found while testing
The first version passed unit tests but failed end-to-end: the same trigger
string placed in
SKILL.md,docs/guide.md,tests/test_skill.py,config/settings.jsonand a comment line inscripts/helper.pylabelled thehelper as
code.infer_surfacereturnedcommentswhen called directly, sothe label was being lost after conversion. Instrumenting
_expand_occurrencesshowed the cause:
deduplicate()groups byrule_id + fingerprint + classification_metadata, so all five files collapsed into one findingrepresented by
SKILL.md, with the other four hanging off as occurrences. Whenthe report expanded them, only the representative's label survived.
Fix: each occurrence records its own label (same-file reuses the finding's,
cross-file falls back to pure path inference), and both expansion and SARIF
prefer the occurrence's own. The label is stored in the occurrence dict's
value, not its key — as a key, two labels on one
(file, line)could add arow and break the "do not change finding counts" constraint.
Testing
(
mypyreports one error instatic_runner.py—_paragraph_ranges'stuple[()]typing — which reproduces on unmodifiedorigin/main.)End-to-end on a purpose-built 5-surface skill package (
--no-llm):No behavior regression: scanning the same package on unmodified
origin/mainand on this branch produced field-identical issues (apart from thenew
surfaceand the per-runfinding_id) and an identicalrisk_assessment(score 83 / CRITICAL / DO_NOT_INSTALL, severity and rule distributions both
{'CRITICAL': 10}/{'P5': 10}), with identical suppression.Notes for Reviewer
down: 223 lines of production code (13 of which are the repo's required
license header, 17 a module docstring, ~40 the extension/filename constant
tables), 173 lines of parametrised tests, 15 lines of edits. Compressing to
300 would mean dropping one of the six surfaces or the comment detection —
both of which the issue explicitly asks for. Flagging rather than silently
cutting.
x = 1 # payload)stay
code, since no reliable column is retained; documented in theinfer_surfacedocstring.back to path-based inference. Adding a language is one entry in
_COMMENT_MARKERS.docsvsconfighas some judgement in it:MakefileandDockerfilearedeliberately treated as executable build scripts (
code), license files asdocs. Changing the call is a constant-table edit.text, so they use path-only inference and never land in
comments. That isintentional — not guessing at information we do not have.