PurlDB: purl2sym Data collection Batch3

[DennisClark]

Here we add purl2sym support for a list of PURLs listed below.

1. Add support in FetchCode and PurlDB
2. Test of end-to-end support in the PurlDB

• pkg:generic/busybox
• pkg:generic/uclibc
• pkg:github/u-boot/u-boot
• pkg:conan/lz4
• pkg:conan/xz_utils
• pkg:conan/lzma_sdk
• pkg:conan/zlib
• pkg:github/dosfstools/dosfstools
• pkg:generic/e2fsprogs
• pkg:generic/erofs-utils
• pkg:github/bestouff/genext2fs
• pkg:generic/mtd-utils
• pkg:github/plougher/squashfs-tools
• pkg:openssl/openssl
• pkg:github/avahi/avahi
• pkg:conan/libcurl
• pkg:github/inotify-tools/inotify-tools
• pkg:generic/opkg
  • There are too many versions floating around: Yocto has a one, forked from OpenWRT, OpenWRT has one and all are forked from Lede itself a fork of openwrt, that later rejoined. This ends up being too complex and we move this for later in a future Batch 6. See PurlDB: purl2sym Data collection Batch6 #355

[keshav-space]

Completed in:

• purl2sym: Add metadata support for packages hosted on GitHub fetchcode#112
• purl2sym: Add metadata support for linux, mtd-utils, barebox, e2fsprogs and erofs-utils fetchcode#113
• Add metadata support for openssl, udhcp fetchcode#114
• Use authenticated requests for GitHub REST API calls fetchcode#115
• purl2sym: Support indexing of Batch3, Batch4 and Batch5 packages #360
• Support conan package indexing #289

The simple way to test this is to follow the below steps after installing PurlDB (https://github.com/nexB/purldb?tab=readme-ov-file#installation):

1. Go to /api/collect/ add PURL for indexing. ex: /api/collect/?purl=pkg:github/u-boot/u-boot@2023.10&addon_pipelines=collect_symbols_ctags
2. Once the indexing has completed go to /api/resources/and filter out resource based on PURL to see the symbols for each resource in extra_data field. ex: /api/resources/?purl=pkg:github/u-boot/u-boot@2023.10

Tip

Follow this tutorial to index the symbols and strings literals for a PURL/Package using various different addon pipelines. https://aboutcode.readthedocs.io/projects/PURLdb/en/latest/how-to-guides/tutorial_symbol_and_string_collection.html