Hii @moranegg @zack I'm working on this task and I noticed that we are using pybadges for implementing swh-badges which isn't the case with shields.io. Based on what I understand, these badges have to be re-implemented as shields.io services will use swh apis to generate the dynamic parts of the badges.

Do you some more tests or this task can be declared as resolved?

Should I add the names of other contributors as well?

Should I add the names of other contributors as well?

Rebase before pushing

Add same test for regular kafka writer

Hey @hm, I don't see any typo in https://forge.softwareheritage.org/source/swh-lister/browse/master/docs/run_a_new_lister.rst$50-51. If you do, please make a revision to fix the same.

Updating D5419: Cassandra: Deduplicate lists passed to *_add endpoints

Hey @faux it's the same as T3145

Updating D5419: Cassandra: Deduplicate lists passed to *_add endpoints

Updating D5419: Cassandra: Deduplicate lists passed to *_add endpoints

we should build a single model object for the top-level dir, and either output its SWHID, or traverse it (without recomputing SWHIDs) to output all of it

Updating D5420: cli/identify: Use TerminalColor Enum and change recursive flag's description

What do you think should be done for releases?

Updating D5419: Cassandra: Fixed failing tests

These objects has an id attribute. You can use it for deduplication (eg. via a dict)

Not all objects had id so I used swhid. But some of the tests are failing.

Hi @hm, Your changes have been accepted but they aren't merged yet. Please do a git push to get your commit(s) merged :)

Updating D5419: Cassandra: Deduplicate lists passed to *_add endpoints

Updating D5419: Cassandra: Deduplicate lists passed to *_add endpoints

I just discovered that tests failed because the set's internally used hash function throws an error for if a dictionary is passed.
Do you know any other trick which can do the de-duplication in one line? Or should I just create a common function to loop over the list and find the unique ones?

I don't think you need to convert the sets back to lists

I did that and got type errors from mypy.
Imo, it's okay to leave it as list(set(..)) because anyhow it gets transformed back into a list within the next 2-3 lines.
What do you think?

Hi guys. Any pointers on where to start?

I might be wrong but, I think it has been completed. Check out these :

Hey @moranegg
I suggest the following modifications in adding-support-for-additional-metadata page:

• python3 -m swh.indexer.metadata_dictionary MyMapping path/to/input/file doesn't work. Replace with swh indexer mapping translate cff path/to/input/file
• Whenever adding new mappings, it has to be mentioned in the MAPPING_NAME variable of the swh/indexer/storage/__init__.py file which isn't mentioned in the documentation. (though it didn't throw error while testing or parsing) and in expected_output of test_cli_mapping_list
• Mentioning a few examples about fields that are not string_fields
• Elaborating how _traslate_dict function works. For example : it executes functions starting with 'normalize_'
• Mention command : swh indexer mapping list-terms to display Supported CodeMeta terms
• Add Youtube videos about JSONLD : JSON-LD Basics, JSON-LD: Core markup, Compaction and Expansion

I am here to just say: swh-loader-git doesn't have a CONTRIBUTORS file. You may ask the contributor to add it as well :)

Hey @vlorentz
How do I check https://forge.softwareheritage.org/source/swh-journal/browse/master/swh/journal/writer/inmemory.py$31. Do I have to pass dummy content, raw_extrinsic_metadata, origin_visit, et cetera as the object_ to write_addition function and before passing verify if they have unique_key function implemented ?

Hey @vlorentz, can you please give me some hints for this and an example URL for testing the code?

Hi @zack Is this task still valid? If so how do I reproduce the error. I tried running pytest in swh-fuse and swh-core but that doesn't throw any error.

You can check if there is currently some warnings issued during the documentation build by following this link. Every reported warnings should be fixed.

If you want to add .vscode to .gitignore, you should do it in its own diff, and for *all* repositories.

Updating D5273: swh-indexer : Fix issues related to datePublished and codeRepository fields.
Migrate code related to authors into normalize_authors function
Add .vscode/ in .gitignore to avoid tracking launch.json and other VScode config files

Hey @sdruskat, I have a few questions : (1) Any idea of when will be the new version available and (2) when will be the crosswalk file updated to at least 1.1.0? (3) The newer version will be backwards compatible, right?

Updating D5273: Remove 'schema:' and limit test to two authors

And the expected value in your test is not valid, because it uses the schema: prefix, without defining it.

Hey @sdruskat, I noticed that the current crosswalk.csv has CFF version 1.0.2. And I agree on keeping the crosswalk.csv file updated as much as possible. But to the best of my knowledge, updating the crosswalk.csv file later (when you are done with creating the new version) won't break anything here. Plus, I am new here and I am learning about metadata which is helping me write a better GSoC proposal.
Also, I can assure you that I will update the crosswalk.csv file myself if you want :)

Any valid Codemeta (or even schema.org) is fine

Updating D5273: swh-indexer : Fix failing tests

Any valid Codemeta (or even schema.org) is fine

So how does a parser decide which one to use for interpretation?

Updating D5273: swh-indexer : Add mapping for CITATION.cff files

I am writing tests for CITATION.cff metadata mapping and I have 2 doubts :
(1) Are CITATION and CITATION.cff the same? (If so, cff-converter isn't able to parse our swh/indexer/data/codemeta/CITATION file maybe because it doesn't look like a YAML file)
(2) cff-converter's output is :

{
    "@context": [
        "https://doi.org/10.5063/schema/codemeta-2.0",
        "http://schema.org"
    ],
    "@type": "SoftwareSourceCode",
    "author": [
        {
            "@id": "https://orcid.org/0000-0002-7064-4069",
            "@type": "Person",
            "affiliation": {
                "@type": "Organization",
                "legalName": "Netherlands eScience Center"
            },
            "familyName": "Spaaks",
            "givenName": "Jurriaan H."
        },
        {
            "@type": "Person",
            "affiliation": {
                "@type": "Organization",
                "legalName": "Netherlands eScience Center"
            },
            "familyName": "Klaver",
            "givenName": "Tom"
        }
    ],
    "codeRepository": "https://github.com/citation-file-format/cff-converter-python",
    "datePublished": "2019-11-12",
    "identifier": "https://doi.org/10.5281/zenodo.1162057",
    "keywords": [
        "citation",
        "bibliography",
        "cff",
        "CITATION.cff"
    ],
    "license": "http://www.apache.org/licenses/LICENSE-2.0",
    "name": "cffconvert",
    "version": "1.4.0-alpha0"
}

Updating D5273: Add mapping for CITATION.cff

Updating D5282: swh-indexer: sync data/codemeta with official codemeta repo

Updating D5282: swh-indexer: sync data/codemeta with official codemeta repo

Updating D5282: Fix SWHID in CITATION

Hey @vlorentz I noticed that creator has been removed from codemeta.jsonld which is also part of our repository (Source : Github Commit)

Updating D5282: Update data/codemeta

Updating D5282: Update SWHID in CITATION and remove creator from codemeta.jsonld (in sync with the changes in the official codemeta repo)

Please give some more details on what problems this diff solves. While adding the
reference task is good practice (thanks), it's not enough.

Please don't update crosswalk.csv directly here, it should only be imported from Codemeta. Make sure you read their CONTRIBUTING.md first.

Hi @zack, we can consider using Elasticsearch string query format to achieve this feature without having to design the syntax from scratch. It is really powerful and can cover most of the use cases.

In T1475#27210, @vlorentz wrote:

" AttributeError on @id with a colon but less than two slashes" https://github.com/digitalbazaar/pyld/issues/91

Hey @moranegg I had a discussion yesterday with @vlorentz on this. We came to the conclusion that the cff-converter should be used as it might cover some edge cases as well. So what should we do now? Please guide.

Updating D5213: swh/scanner : Add newline in CONTRIBUTORS file

Updating D5213: swh/scanner : Add Kumar Shivedu to CONTRIBUTORS