Update documentation for v32

.github/PULL_REQUEST_TEMPLATE.md

@@ -17,7 +17,8 @@ Make sure these boxes are checked before your pull request (PR) is ready to be r
 * [ ] Tests pass -- look for a green checkbox ✔️ a few minutes after opening your PR
   Run [tests](https://scancode-toolkit.readthedocs.io/en/latest/contribute/contrib_dev.html#running-tests) locally to check for errors. 
 * [ ] Commits are in uniquely-named feature branch and has no merge conflicts 📁
-
+* [ ] Looked for possible updates in documentation and added updates if applicable
+* [ ] Updated CHANGELOG.rst
 <!--
 We're happy to help you get this ready -- don't be afraid to ask for help, and **don't be discouraged**
 if your tests fail at first!

README.rst

@@ -13,6 +13,8 @@ Check out the code at https://github.com/nexB/scancode-toolkit
 Discover also:
 
 - The ScanCode.io server project here: https://scancodeio.readthedocs.io
+- The ScanCode Workbench project for visualization of scancode results data:
+  https://github.com/nexB/scancode-workbench 
 - Other companion SCA projects for code origin, license and security analysis
   here: https://aboutcode.org
 
@@ -23,11 +25,11 @@ Build and tests status
 We run 30,000+ tests on each commit on multiple CIs to ensure a good platform
 compabitility with multiple versions of Windows, Linux and macOS.
 
-+--------------+--------------+--------------+
-| **Appveyor** | **Azure**    | **RTD Build**|
-+==============+==============+==============+
-| |appveyor|   |    |azure|   | |docs-rtd|   |
-+--------------+--------------+--------------+
++------------+--------------+-------------------------+----------------------------+
+| **Azure**  | **RTD Build**| **GitHub actions Docs** | **GitHub actions Release** |
++============+==============+=========================+============================+
+|  |azure|   | |docs-rtd|   |  |docs-github-actions|  |  |release-github-actions|  |
++------------+--------------+-------------------------+----------------------------+
 
 
 Why use ScanCode?
@@ -61,35 +63,39 @@ Why use ScanCode?
   new and improved scanners, data summarization, package manifest parsers, and
   new outputs.
 
-- You can save your scan results as **JSON, HTML, CSV or SPDX** or create your
-  own format with Jinja templates.
+- You can save your scan results as **JSON, YAML, HTML, CycloneDX or SPDX** or
+  even create your own format with Jinja templates.
 
 - You can also organize and run ScanCode server-side with the
   companion `ScanCode.io web app <https://github.com/nexB/scancode.io>`_
   to organize and store multiple scan projects including scripted scanning pipelines.
 
+- ScanCode output data can be easily visualized and analysed using the
+  `ScanCode Workbench <https://github.com/nexB/scancode-workbench>`_ desktop app.
+
 - ScanCode is **actively maintained**, has a **growing users and contributors
   community**.
 
 - ScanCode is heavily **tested** with an automated test suite of over **20,000 tests**.
 
 - ScanCode has an extensive and growing documentation.
 
-- ScanCode can process these packages, build manifest and lockfile formats to collect Package URLs and
-  extract metadata: Alpine packages, BUCK files, ABOUT files, Android apps, Autotools, Bazel, 
-  JavaScript Bower, Java Axis, MS Cab, Rust Cargo, Cocoapods, Chef Chrome apps, 
-  PHP Composer and composer.lock, Conda, CPAN, Debian, Apple dmg,
-  Java EAR, WAR, JAR, FreeBSD packages, Rubygems gemspec, Gemfile and Gemfile.lock, 
-  Go modules, Haxe packages, InstallShield installers, iOS apps, ISO images, 
-  Apache IVY, JBoss Sar, R CRAN, Apache Maven, Meteor, Mozilla extensions, 
-  MSI installers, JavaScript npm packages, package-lock.json, yarn.lock, 
-  NSIS Installers, NugGet, OPam, Cocoapods, Python PyPI setup.py, setup.cfg, and 
+- ScanCode can process packages, build manifest and lockfile formats to collect
+  Package URLs and extract metadata: Alpine packages, BUCK files, ABOUT files,
+  Android apps, Autotools, Bazel, JavaScript Bower, Java Axis, MS Cab,
+  Rust Cargo, Cocoapods, Chef Chrome apps, PHP Composer and composer.lock,
+  Conda, CPAN, Debian, Apple dmg, Java EAR, WAR, JAR, FreeBSD packages,
+  Rubygems gemspec, Gemfile and Gemfile.lock, Go modules, Haxe packages,
+  InstallShield installers, iOS apps, ISO images, Apache IVY, JBoss Sar,
+  R CRAN, Apache Maven, Meteor, Mozilla extensions, MSI installers,
+  JavaScript npm packages, package-lock.json, yarn.lock, NSIS Installers,
+  NugGet, OPam, Cocoapods, Python PyPI setup.py, setup.cfg, and 
   several related lockfile formats, semi structured README
   files such as README.android, README.chromium, README.facebook, README.google,
   README.thirdparty, RPMs, Shell Archives, Squashfs images, Java WAR, Windows
   executables and the Windows registry
-  and a few more.
-
+  and a few more. See `all available package parsers <https://scancode-toolkit.readthedocs.io/en/stable/reference/available_package_parsers.html>`_
+  for the exhaustive list.
 
 See our `roadmap <https://scancode-toolkit.readthedocs.io/en/latest/contribute/roadmap.html>`_
 for upcoming features.
@@ -101,11 +107,11 @@ Documentation
 The ScanCode documentation is hosted at
 `scancode-toolkit.readthedocs.io <https://scancode-toolkit.readthedocs.io/en/latest/>`_.
 
-If you are new to Scancode, start with our
+If you are new to visualization of scancode results data, start with our
 `newcomer <https://scancode-toolkit.readthedocs.io/en/latest/getting-started/newcomer.html>`_ page.
 
-If you want to compare output changes between different versions of Scancode, 
-or want to look at  scans generated by Scancode, review our
+If you want to compare output changes between different versions of ScanCode, 
+or want to look at  scans generated by ScanCode, review our
 `reference scans <https://github.com/nexB/scancode-toolkit-reference-scans>`_.
 
 Other Important Documentation Pages:
@@ -134,21 +140,23 @@ Installation
 
 Before installing ScanCode make sure that you have installed the prerequisites
 properly. This means installing Python 3.8 for x86/64 architectures.
-We support Python 3.8, 3.9 and 3.10.
+We support Python 3.8, 3.9, 3.10 and 3.11.
 
 See `prerequisites <https://scancode-toolkit.readthedocs.io/en/latest/getting-started/install.html#prerequisites>`_
 for detailed information on the support platforms and Python versions.
 
 There are a few common ways to `install ScanCode <https://scancode-toolkit.readthedocs.io/en/latest/getting-started/install.html>`_.
 
-- **Installation as an application: Install Python 3.8, download a release archive, extract and run**. `This is the recommended installation method.
+- `**Installation as an application: Install Python 3.8, download a release archive, extract and run**. 
   <https://scancode-toolkit.readthedocs.io/en/latest/getting-started/install.html#installation-as-an-application-downloading-releases>`_
+  This is the recommended installation method.
 
 - `Development installation from source code using a git clone
   <https://scancode-toolkit.readthedocs.io/en/latest/getting-started/install.html#installation-from-source-code-git-clone>`_
 
 - `Development installation as a library with "pip install scancode-toolkit"
   <https://scancode-toolkit.readthedocs.io/en/latest/getting-started/install.html#pip-install>`_
+  [Note that this is not supported on arm64 machines]
 
 - `Run in a Docker container with a git clone and "docker run"
   <https://scancode-toolkit.readthedocs.io/en/latest/getting-started/install.html#installation-via-docker>`_
@@ -157,20 +165,17 @@ There are a few common ways to `install ScanCode <https://scancode-toolkit.readt
 Quick Start
 ===========
 
-Note the `commands variation <https://scancode-toolkit.readthedocs.io/en/latest/getting-started/install.html#commands-variation>`_
-across installation methods and platforms.
+After ScanCode is installed successfully you can run an example scan printed on screen as JSON::
 
-You can run an example scan printed on screen as JSON::
-
-    ./scancode -clip --json-pp - samples
+    scancode -clip --json-pp - samples
 
 Follow the `How to Run a Scan <https://scancode-toolkit.readthedocs.io/en/latest/tutorials/how_to_run_a_scan.html>`_
 tutorial to perform a basic scan on the ``samples`` directory distributed by
-default with Scancode.
+default with ScanCode.
 
 See more command examples::
 
-    ./scancode --examples
+    scancode --examples
 
 See `How to select what will be detected in a scan
 <https://scancode-toolkit.readthedocs.io/en/latest/tutorials/how_to_set_what_will_be_detected_in_a_scan.html>`_
@@ -208,15 +213,13 @@ https://github.com/nexB/scancode-toolkit/issues
 For discussions and chats, we have:
 
 * an official Gitter channel for `web-based chats
-  <https://gitter.im/aboutcode-org/discuss>`_.
-  Gitter is also accessible via an `IRC bridge <https://irc.gitter.im/>`_.
+  <https://matrix.to/#/#aboutcode-org_discuss:gitter.im>`_.
+  Gitter is now accessible through `Element <https://element.io/download>`_
+  or an `IRC bridge <https://matrix-org.github.io/matrix-appservice-irc/latest/usage.html>`_.
   There are other AboutCode project-specific channels available there too.
 
-* an official `#aboutcode` IRC channel on liberachat (server web.libera.chat).
-  This channel receives build and commit notifications and can be noisy.
-  You can use your favorite IRC client or use the `web chat 
-  <https://web.libera.chat/?#aboutcode>`_.
-
+* The discussion channel for `scancode <https://matrix.to/#/#aboutcode-org_scancode:gitter.im>`_
+  specifically aimed at users and developers using scancode-toolkit.
 
 Source code and downloads
 =========================
@@ -225,7 +228,8 @@ Source code and downloads
 * https://github.com/nexB/scancode-toolkit.git
 * https://pypi.org/project/scancode-toolkit/
 * https://github.com/nexB/scancode-thirdparty-src.git
-
+* https://github.com/nexB/scancode-plugins.git
+* https://github.com/nexB/thirdparty-packages.git
 
 License
 =======
@@ -240,14 +244,18 @@ See the NOTICE file and the .ABOUT files that document the origin and license of
 the third-party code used in ScanCode for more details.
 
 
-.. |appveyor| image:: https://ci.appveyor.com/api/projects/status/4webymu0l2ip8utr?svg=true
-    :target: https://travis-ci.org/nexB/scancode-toolkit
-    :alt: Appveyor tests status (Windows)
-
 .. |azure| image:: https://dev.azure.com/nexB/scancode-toolkit/_apis/build/status/nexB.scancode-toolkit?branchName=develop
     :target: https://dev.azure.com/nexB/scancode-toolkit/_build/latest?definitionId=1&branchName=develop
     :alt: Azure tests status (Linux, macOS, Windows)
 
 .. |docs-rtd| image:: https://readthedocs.org/projects/scancode-toolkit/badge/?version=latest
     :target: https://scancode-toolkit.readthedocs.io/en/latest/?badge=latest
     :alt: Documentation Status
+
+.. |docs-github-actions| image:: https://github.com/nexB/scancode-toolkit/actions/workflows/docs-ci.yml/badge.svg?branch=develop
+    :target: https://github.com/nexB/scancode-toolkit/actions/workflows/docs-ci.yml
+    :alt: Documentation Tests
+
+.. |release-github-actions| image:: https://github.com/nexB/scancode-toolkit/actions/workflows/scancode-release.yml/badge.svg?event=push
+    :target: https://github.com/nexB/scancode-toolkit/actions/workflows/scancode-release.yml
+    :alt: Release tests

docs/Makefile

@@ -18,3 +18,4 @@ help:
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
 	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	./scripts/regen_package_docs.sh

docs/scripts/regen_package_docs.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+# halt script on error
+set -e
+
+# Path to the available packages doc
+DOC_PATH="source/reference/available_package_parsers.rst"
+
+# Regen docs for available package parsers
+regen-package-docs --path "$DOC_PATH"
+
+# Delete whitespace from last line to not fail doc8 tests
+sed -i '$ d' "$DOC_PATH"