How to ignore specific directorys during scan

[phschulth]

Hi all,

currently i'm playing around with scancode to get familiar with it. Since we have a few submodules for test purposes in our repositories id like to know if it is possible to say scancode to ignore specific directories. At the moment i only found the solution with the ignore/include pattern. But its quite exhausting to filter all file endings and most of the time the same endings are reused in a different subfolder which should be scanned.

Maybe I didn't find it in the documentation either. I do not know. But is there a way to ignore e.g. the googletest folder in the repository by parameter?

Thanks in advance!

[relikd]

Chiming in on this cause I wanted to open a bug report myself.

Normally you would be able to --ignore "googletest/*" however scancode does not look relative to the current working directory but relative to the parent directory. So you would need to --ignore "my-proj/googletest/*". Which is a bummer, because now I need to write code for a folder-name I have no control over. Imagine a public library: I have control over the subfolder "googletest" but not "my-proj" even though I named my repository so (e.g., user checks out as "my-proj-copy").

Scancode has an option --strip-root which removes this prefix from the output but is, sadly, ignored for the pre-scan option. So I am forced to write something like --ignore "${basename $PWD}/googletest/*" (imagine that for 5+ subfolders).

The other issue with this folder ignore is that it does seem to break the scanning process. When I start a scan with ignored folders, it display the last file indefinitely. I have a console open that is stuck at 50 small files since over an hour. I don't know what is broken but something is wrong with the ignore flag.

[relikd]

It gets weirder. Even though the console output does not show it, if I abort the scan, the output json will include sub-directories of ignored folders. To be clear: something seems to be ignored because the "Collect file inventory" step is much faster compared to scanning all 120k files without the ignore filter. But these files (or at least folders) are still processed somehow without showing any progress bar.

... Nevermind the indefinitely part. This was due to a 130mb json file. Without that file, the scan finishes normally (still outputs the ignored subfolder directories though). So the ignore params in my previous post work fine.

Just a small bug in context, the 130mb was not shown as the currently processed file. That is why I was not immediatelly alarmed. Instead it displayed a different (small) file.

[relikd]

It keeps getting better:

This works as expected:
scancode --include "proj/src/*" .

This, however, does not:
scancode --ignore 'src/temp/*' src

The asterisk (*) is expanded and puts the first file of temp into the ignore flag and the remaining files are treated as input. Which is not only utterly wrong but ignores the single quoted string which should prohibit any expansion.

This is even worse:
scancode --ignore 'src/temp/*' ./src

Notice the ./ before the input directory. This results in scancode ignoring the provided source dir and instead performing the scan on the entire project directory.

Only way to solve it is by escaping the asterisk:
scancode --ignore "src/temp/\*" src

EDIT: also notice, the ignore flag is relative to the target dir but always starts with the target dir name

[ehzoc]

The issue is still relevant in: scancode-toolkit 32.0.6

I am using scancode on Windows (not sure if this makes a difference on how the paths are expanded) and getting similar results.

Neither one of the following arguments passed to the --ignore flag seems to have an effect on the scan behavior:

• "*.txt"
• "\*.txt"
• '*.txt'
• '\*.txt'
• *.txt
• \*.txt

Using the --verbose flag shows that *.txt files are scanned regardless.

[richardzaat]

Having the exact same issue.
scancode -l -n 4 --ignore ".json" --ignore ".png"

still scans the json files, but does not scan the png files.

What does work is if I state --ignore "a.json"
This will ignore any file with an a in the json name.
But not if the name as brackets in it ().
So folder\folder\name(name).json gets found and not ignored

[a-z].json does not work either. **/*.json excludes all files

Really need this to help speed up the scancode process on our machines