Added support for parsing citation.cff files #3625
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # | ||
| # Copyright (c) nexB Inc. and others. All rights reserved. | ||
| # ScanCode is a trademark of nexB Inc. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # See http://www.apache.org/licenses/LICENSE-2.0 for the license text. | ||
| # See https://github.com/nexB/scancode-toolkit for support or download. | ||
| # See https://aboutcode.org for more information about nexB OSS projects. | ||
| # | ||
|
|
||
| import io | ||
| import saneyaml | ||
| from packagedcode import models | ||
|
|
||
| """ | ||
| Handle CITATION.cff files, see https://citation-file-format.github.io/ | ||
| https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.md | ||
|
|
||
| See https://github.com/citation-file-format/citation-file-format/tree/main/examples/1.2.0/pass for examples. | ||
| """ | ||
|
|
||
| class CitationHandler(models.DatafileHandler): | ||
| datasource_id = 'citation_cff' | ||
| default_package_type = 'citation' | ||
| path_patterns = ('*/CITATION.cff',) | ||
|
Member
There was a problem hiding this comment. Based on https://github.com/search?q=path%3A*.cff&type=Code&ref=advsearch&l=&l=&p=1 these are often also in lowercase and others, and as we do a case-sensitive matching here (should we? 🤔) we would fail on lowercase and others, should specify more cases here. |
||
| description = 'Citation file format' | ||
| documentation_url = 'https://citation-file-format.github.io/' | ||
|
|
||
| @classmethod | ||
| def parse(cls, location): | ||
| metayaml = get_cff_data(location) | ||
| title = metayaml.get('title') | ||
| abstract = metayaml.get('abstract') | ||
| keywords = metayaml.get('keywords') | ||
| release_date = metayaml.get('date-released') | ||
| # This is the version of the software/item and not cff-version | ||
| version = metayaml.get('version') | ||
| parties = [] | ||
| authors = metayaml.get('authors') | ||
| if authors: | ||
| for author in authors: | ||
| name = author.get('family-names') or author.get('name') | ||
| email = author.get('email') | ||
| url = author.get('orcid') | ||
| parties.append(models.Party( | ||
| name=name, | ||
| email=email, | ||
| url=url | ||
| )) | ||
| repository_homepage_url = metayaml.get('repository') | ||
| repository_download_url = metayaml.get('repository-code') | ||
| extracted_license_statement = metayaml.get('license') | ||
|
Member
There was a problem hiding this comment. There are interesting details in https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.md#license |
||
| homepage_url = metayaml.get('url') | ||
|
|
||
| yield models.PackageData( | ||
| datasource_id=cls.datasource_id, | ||
| type=cls.default_package_type, | ||
| name=title, | ||
| description=abstract, | ||
| keywords=keywords, | ||
| release_date=release_date, | ||
| parties=parties, | ||
| version=version, | ||
| repository_homepage_url=repository_homepage_url, | ||
| repository_download_url=repository_download_url, | ||
| extracted_license_statement=extracted_license_statement, | ||
| homepage_url=homepage_url | ||
| ) | ||
|
|
||
|
|
||
| def get_cff_data(location): | ||
| """ | ||
| Return a mapping of yaml/cff data loaded from a CITATION.cff files. | ||
| """ | ||
|
|
||
| # TODO: Should this be Jinja-based? | ||
| yaml_lines = [] | ||
| with io.open(location, encoding='utf-8') as citationcff: | ||
| for line in citationcff: | ||
| if not line: | ||
| continue | ||
| yaml_lines.append(line) | ||
|
|
||
| return saneyaml.load('\n'.join(yaml_lines)) | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| # This CITATION.cff file was created by ruby-cff (v 1.1.0). | ||
| # Added few changes to help with testing. | ||
| # Gem: https://rubygems.org/gems/cff | ||
| # CFF: https://citation-file-format.github.io/ | ||
| # Link: https://github.com/citation-file-format/ruby-cff/blob/main/CITATION.cff | ||
|
|
||
| cff-version: 1.2.0 | ||
| message: If you use ruby-cff in your work, please cite it using the following metadata | ||
| title: Ruby CFF Library | ||
| type: software | ||
| abstract: This library provides a Ruby interface to manipulate Citation File Format files | ||
| authors: | ||
| - family-names: Haines | ||
| given-names: Robert | ||
| orcid: https://orcid.org/0000-0002-9538-7919 | ||
| affiliation: The University of Manchester, UK | ||
| - name: The Ruby Citation File Format Developers | ||
| email: abc@developer.com | ||
| keywords: | ||
| - ruby | ||
| - credit | ||
| - software citation | ||
| - research software | ||
| - software sustainability | ||
| - metadata | ||
| - citation file format | ||
| - CFF | ||
| version: 1.1.0 | ||
| doi: 10.5281/zenodo.1184077 | ||
|
Member
There was a problem hiding this comment. The |
||
| date-released: 2023-04-10 | ||
| license: Apache-2.0 | ||
| url: https://rubygems.org/ | ||
| repository: https://github.com/citation-file-format/ | ||
| repository-artifact: https://rubygems.org/gems/cff | ||
| repository-code: https://github.com/citation-file-format/ruby-cff | ||
| references: | ||
|
Member
There was a problem hiding this comment. Here we have to think if references can be listed as dependencies. See https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.md#referencing-other-work which does support this. Maybe only for the references with a |
||
| - type: software | ||
| title: Citation File Format | ||
| authors: | ||
| - family-names: Druskat | ||
| given-names: Stephan | ||
| orcid: https://orcid.org/0000-0003-4925-7248 | ||
| - family-names: Spaaks | ||
| given-names: Jurriaan H. | ||
| orcid: https://orcid.org/0000-0002-7064-4069 | ||
| - family-names: Chue Hong | ||
| given-names: Neil | ||
| orcid: https://orcid.org/0000-0002-8876-7606 | ||
| - family-names: Haines | ||
| given-names: Robert | ||
| orcid: https://orcid.org/0000-0002-9538-7919 | ||
| - family-names: Baker | ||
| given-names: James | ||
| orcid: https://orcid.org/0000-0002-2682-6922 | ||
| - family-names: Bliven | ||
| given-names: Spencer | ||
| orcid: https://orcid.org/0000-0002-1200-1698 | ||
| email: spencer.bliven@gmail.com | ||
| - family-names: Willighagen | ||
| given-names: Egon | ||
| orcid: https://orcid.org/0000-0001-7542-0286 | ||
| - family-names: Pérez-Suárez | ||
| given-names: David | ||
| orcid: https://orcid.org/0000-0003-0784-6909 | ||
| website: https://dpshelio.github.io | ||
| - family-names: Konovalov | ||
| given-names: Alexander | ||
| orcid: https://orcid.org/0000-0001-5299-3292 | ||
| identifiers: | ||
| - type: doi | ||
| value: 10.5281/zenodo.1003149 | ||
| description: The concept DOI for the collection containing all versions of the Citation File Format. | ||
| - type: doi | ||
| value: 10.5281/zenodo.5171937 | ||
| description: The versioned DOI for the version 1.2.0 of the Citation File Format. | ||
| keywords: | ||
| - citation file format | ||
| - CFF | ||
| - citation files | ||
| - software citation | ||
| - file format | ||
| - YAML | ||
| - software sustainability | ||
| - research software | ||
| - credit | ||
| abstract: CITATION.cff files are plain text files with human- and machine-readable citation information for software. Code developers can include them in their repositories to let others know how to correctly cite their software. This is the specification for the Citation File Format. | ||
| date-released: 2021-08-09 | ||
| license: CC-BY-4.0 | ||
| version: 1.2.0 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| [ | ||
| { | ||
| "type": "citation", | ||
| "namespace": null, | ||
| "name": "Ruby CFF Library", | ||
| "version": "1.1.0", | ||
| "qualifiers": {}, | ||
| "subpath": null, | ||
| "primary_language": null, | ||
| "description": "This library provides a Ruby interface to manipulate Citation File Format files", | ||
| "release_date": "2023-04-10", | ||
| "parties": [ | ||
| { | ||
| "type": null, | ||
| "role": null, | ||
| "name": "Haines", | ||
| "email": null, | ||
| "url": "https://orcid.org/0000-0002-9538-7919" | ||
| }, | ||
| { | ||
| "type": null, | ||
| "role": null, | ||
| "name": "The Ruby Citation File Format Developers", | ||
| "email": "abc@developer.com", | ||
| "url": null | ||
| } | ||
| ], | ||
| "keywords": [ | ||
| "ruby", | ||
| "credit", | ||
| "software citation", | ||
| "research software", | ||
| "software sustainability", | ||
| "metadata", | ||
| "citation file format", | ||
| "CFF" | ||
| ], | ||
| "homepage_url": "https://rubygems.org/", | ||
| "download_url": null, | ||
| "size": null, | ||
| "sha1": null, | ||
| "md5": null, | ||
| "sha256": null, | ||
| "sha512": null, | ||
| "bug_tracking_url": null, | ||
| "code_view_url": null, | ||
| "vcs_url": null, | ||
| "copyright": null, | ||
| "holder": null, | ||
| "declared_license_expression": "apache-2.0", | ||
| "declared_license_expression_spdx": "Apache-2.0", | ||
| "license_detections": [ | ||
| { | ||
| "license_expression": "apache-2.0", | ||
| "matches": [ | ||
| { | ||
| "score": 100.0, | ||
| "start_line": 1, | ||
| "end_line": 1, | ||
| "matched_length": 3, | ||
| "match_coverage": 100.0, | ||
| "matcher": "1-hash", | ||
| "license_expression": "apache-2.0", | ||
| "rule_identifier": "spdx_license_id_apache-2.0_for_apache-2.0.RULE", | ||
| "rule_relevance": 100, | ||
| "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/spdx_license_id_apache-2.0_for_apache-2.0.RULE", | ||
| "matched_text": "Apache-2.0" | ||
| } | ||
| ], | ||
| "identifier": "apache_2_0-d66ab77d-a5cc-7104-e702-dc7df61fe9e8" | ||
| } | ||
| ], | ||
| "other_license_expression": null, | ||
| "other_license_expression_spdx": null, | ||
| "other_license_detections": [], | ||
| "extracted_license_statement": "Apache-2.0", | ||
| "notice_text": null, | ||
| "source_packages": [], | ||
| "file_references": [], | ||
| "extra_data": {}, | ||
| "dependencies": [], | ||
| "repository_homepage_url": "https://github.com/citation-file-format/", | ||
| "repository_download_url": "https://github.com/citation-file-format/ruby-cff", | ||
| "api_data_url": null, | ||
| "datasource_id": "citation_cff", | ||
| "purl": "pkg:citation/Ruby%20CFF%20Library@1.1.0" | ||
| } | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| # | ||
| # Copyright (c) nexB Inc. and others. All rights reserved. | ||
| # ScanCode is a trademark of nexB Inc. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # See http://www.apache.org/licenses/LICENSE-2.0 for the license text. | ||
| # See https://github.com/nexB/scancode-toolkit for support or download. | ||
| # See https://aboutcode.org for more information about nexB OSS projects. | ||
| # | ||
|
|
||
| import os | ||
|
|
||
| from packages_test_utils import PackageTester | ||
| from packagedcode import citation | ||
| from scancode_config import REGEN_TEST_FIXTURES | ||
|
|
||
|
|
||
| class TestCitation(PackageTester): | ||
| #test_data_dir = os.path.join(os.path.dirname(__file__), 'data') | ||
|
|
||
| def test_get_cff_data(self): | ||
| test_file = self.get_test_loc('citation/CITATION.cff') | ||
| results = citation.get_cff_data(test_file) | ||
| assert list(results.items())[0] == ((u'cff-version', u'1.2.0')) \ | ||
| and (list(results.items())[8]) == (u'doi', | ||
| u'10.5281/zenodo.1184077') | ||
|
|
||
| def test_citationcff_is_package_data_file(self): | ||
| test_file = self.get_test_loc('citation/CITATION.cff') | ||
| assert citation.CitationHandler.is_datafile(test_file) | ||
|
|
||
| def test_parse(self): | ||
| test_file = self.get_test_loc('citation/CITATION.cff') | ||
| package = citation.CitationHandler.parse(test_file) | ||
| expected_loc = self.get_test_loc('citation/citation.cff.expected.json') | ||
| self.check_packages_data(package, expected_loc, regen=REGEN_TEST_FIXTURES) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this the best type? what about cff? and is this really a new PURL type?
There are likely details in the URLs that would make this another PURL type.