Add dependencies to rpm by alok1304 · Pull Request #4084 · aboutcode-org/scancode-toolkit

alok1304 · 2025-01-10T11:28:13Z

Added dependencies to rpm.

Reference: #649

Fixes #649

Tasks

Reviewed contribution guidelines
PR is descriptively titled 📑 and links the original issue above 🔗
Tests pass -- look for a green checkbox ✔️ a few minutes after opening your PR
Run tests locally to check for errors.
Commits are in uniquely-named feature branch and has no merge conflicts 📁
Updated documentation pages (if applicable)
Updated CHANGELOG.rst (if applicable)

Signed-off-by: Alok Kumar alokkumarjipura9973@gmail.com

alok1304 · 2025-01-10T11:32:45Z

@pombredanne please review this PR, also all test cases have been successfully executed and passed.

Added dependencies to rpm. Reference: aboutcode-org#649 Signed-off-by: Alok Kumar <alokkumarjipura9973@gmail.com> Signed-off-by: Alok Kumar <alokkumarjipura9973@gmail.com>

pombredanne

Thanks for this effort. Please see comments for feedback:

This code cannot and does not work as it is: .rpm archives and installed RPM dbs are not the same.
There is a lot of code duplication
A package cannot be its own dependency
Please start by crafting proper tests first so you can set expectations with carefully reviewed cases.

pombredanne · 2025-01-10T12:48:33Z

+        )
+
+        # Prepare the dependent package model
+        dependencies.append(


Since when there is a single dependency?

@pombredanne Can you suggest how I proceed to find multiple dependencies, I tried by Adding RPMTAG_REQUIRES and RPMTAG_REQUIREVERSION in RPMtags in pyrpm.py but in RPMTAG_REQUIRES I got this like eg: ['/bin/sh', '/bin/sh', '/bin/sh', '/bin/sh', '/bin/sh', 'rpmlib(PayloadFilesHavePrefix)', 'rpmlib(CompressedFileNames)', 'rpmlib(PayloadIsBzip2)']
corresponding I got require_version=[None, None, None, None, None, '4.0-1', '3.0.4-1', '3.0.5-1']
source: http://ftp.rpm.org/max-rpm/ch-queryformat-tags.html (For rpm tages)

can you tell how I proceed to find out their package name and their version, in rmp_requires, these represent capabilities or libraries, not actual packages

Since when there is a single dependency?

I do for single dependency only.

I do for single dependency only.

This cannot work. We need all of them.

You can use "get_rpm_tags" only for a .rpm file, but then you do not need to call it because this is already called

for installed RPM databases, you can use exclusively the data in parse_rpm_xmlish https://github.com/aboutcode-org/scancode-toolkit/pull/4084/files#diff-17fc898047e57c63a40599213e50fb03f8d853b60ea7790133ace1c6ab2b9709R170 and if it does not return what you need, parse_rpm_xmlish needs to be updated in https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/packagedcode/rpm_installed.py. Look at require*, provide* and conflict* for instance in https://github.com/aboutcode-org/scancode-toolkit/edit/develop/tests/packagedcode/data/rpm_installed/distro-xmlish/rhel-rpms.xmlish

pombredanne

@alok1304 you cannot use get_rpm_tags for installed packages. And its dope not make sense to collect "self" as a dependency and to further consider that there is only one dependency which is the package itself.
See also my comments inline

pombredanne · 2025-01-13T14:56:23Z

+        )
+
+        # Prepare the dependent package model
+        dependencies.append(


I do for single dependency only.

This cannot work. We need all of them.

You can use "get_rpm_tags" only for a .rpm file, but then you do not need to call it because this is already called

for installed RPM databases, you can use exclusively the data in parse_rpm_xmlish https://github.com/aboutcode-org/scancode-toolkit/pull/4084/files#diff-17fc898047e57c63a40599213e50fb03f8d853b60ea7790133ace1c6ab2b9709R170 and if it does not return what you need, parse_rpm_xmlish needs to be updated in https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/packagedcode/rpm_installed.py. Look at require*, provide* and conflict* for instance in https://github.com/aboutcode-org/scancode-toolkit/edit/develop/tests/packagedcode/data/rpm_installed/distro-xmlish/rhel-rpms.xmlish

alok1304 · 2025-01-13T15:00:18Z

@alok1304 you cannot use get_rpm_tags for installed packages. And its dope not make sense to collect "self" as a dependency and to further consider that there is only one dependency which is the package itself. See also my comments inline

Yes, I got it. I will sure see this and get some idea of this.
thanks : )

alok1304 · 2025-01-13T19:23:48Z

@pombredanne I checked .rpm(rpm_archive) for dependencies, there are four things in dependencies.

RPMTAG_PROVIDES, RPMTAG_PROVIDEVERSION
RPMTAG_REQUIRES, RPMTAG_REQUIREVERSION
RPMTAG_CONFLICTS, RPMTAG_CONFLICTVERSION
RPMTAG_OBSOLETES, RPMTAG_OBSOLETEVERSION

what I do, I add this in RPMTAGS in pyrmp.py
and getting its name and version
I scan for tests/packagedcode/data/rpm/header/zziplib-0.11.15-3sf.i586.rpm
I got this output in dependencies

"dependencies": [
    {
      "purl": "pkg:rpm/libzzip-0.so.11",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libzzip-0.so.11?uuid=f9d951c0-b971-48cb-8c63-1fbe62f7db35",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/zziplib@2:0.11.15-3sf",
      "extracted_requirement": "2:0.11.15-3sf",
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": true,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=cb1b8ea8-96d9-485b-8044-4ae59748fab6",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/libz",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libz?uuid=66e6c90d-27e8-4a1b-b56b-9e4515eb5043",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/sbin/ldconfig",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/sbin/ldconfig?uuid=0d7c0b46-4104-48ae-97bc-97eeaa6f5d14",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/sbin/ldconfig",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/sbin/ldconfig?uuid=9d824e07-f01d-4d95-ad37-6f4b19566d9e",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/rpmlib%28PayloadFilesHavePrefix%29@4.0-1",
      "extracted_requirement": "4.0-1",
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": true,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/rpmlib%28PayloadFilesHavePrefix%29@4.0-1?uuid=7b5e02b0-da80-4e56-8e7b-8fafae47b478",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/rpmlib%28CompressedFileNames%29@3.0.4-1",
      "extracted_requirement": "3.0.4-1",
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": true,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/rpmlib%28CompressedFileNames%29@3.0.4-1?uuid=a3667b77-e392-44b1-aa7f-d955ce604640",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/ld-linux.so.2",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/ld-linux.so.2?uuid=e2df7012-671c-43e7-8ae6-b608357006f4",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/libc.so.6",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libc.so.6?uuid=a2d6d367-5f5f-4dc4-9780-6b48ed78e186",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/libz.so.1",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libz.so.1?uuid=c3c3c20b-782c-424e-8ddd-ab79e5221b7b",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/libc.so.6%28GLIBC_2.0%29",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libc.so.6%28GLIBC_2.0%29?uuid=e17e3466-25a5-4044-a8c2-317b8a75cca9",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    }
  ]

is this right?

alok1304 force-pushed the 649-add-dependencies-to-rpm branch from f9c5a17 to 7b69733 Compare January 10, 2025 11:28

alok1304 force-pushed the 649-add-dependencies-to-rpm branch from 1c0e2b1 to 9bb48db Compare January 10, 2025 12:33

add dependencies to rpm

e16372a

Added dependencies to rpm. Reference: aboutcode-org#649 Signed-off-by: Alok Kumar <alokkumarjipura9973@gmail.com> Signed-off-by: Alok Kumar <alokkumarjipura9973@gmail.com>

alok1304 force-pushed the 649-add-dependencies-to-rpm branch from 9bb48db to e16372a Compare January 10, 2025 14:26

pombredanne requested changes Jan 10, 2025

View reviewed changes

pombredanne requested changes Jan 13, 2025

View reviewed changes

Uh oh!

Uh oh!

Conversation

alok1304 commented Jan 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tasks

Uh oh!

alok1304 commented Jan 10, 2025

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pombredanne Jan 10, 2025

Choose a reason for hiding this comment

Uh oh!

alok1304 Jan 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alok1304 Jan 11, 2025

Choose a reason for hiding this comment

Uh oh!

pombredanne Jan 13, 2025

Choose a reason for hiding this comment

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pombredanne Jan 13, 2025

Choose a reason for hiding this comment

Uh oh!

alok1304 commented Jan 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alok1304 commented Jan 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alok1304 commented Jan 10, 2025 •

edited

Loading

alok1304 Jan 11, 2025 •

edited

Loading

alok1304 commented Jan 13, 2025 •

edited

Loading

alok1304 commented Jan 13, 2025 •

edited

Loading