Page MenuHomeSoftware Heritage

CODEMETA.json files do not get indexed
Closed, MigratedEdits Locked

Description

When testing origin intrinsic metadata indexing in the docker environment, I noticed CodeMeta JSON files with uppercase characters do not get processed by swh-indexer.

Example repository with such a file: https://c4science.ch/source/costcalc.git

Event Timeline

anlambert triaged this task as Normal priority.Feb 18 2021, 12:20 PM
anlambert created this task.
vlorentz renamed this task from CODEMETA.json files do not get indexed to CODEMETA.json files do not get indexed in the docker environment.Feb 18 2021, 12:46 PM
vlorentz renamed this task from CODEMETA.json files do not get indexed in the docker environment to CODEMETA.json files do not get indexed.
vlorentz added a project: Docker environment.
vlorentz removed a project: Docker environment.
vlorentz added a project: Easy hack.

(I'm marking this as easy hack as it's suitable for newcomers)

Hint: codemeta.json is detected by CodemetaMapping, defined in swh-indexer/swh/indexer/metadata_dictionary/codemeta.py; and it relies on SingleFileMapping to implement the detection

Hello, I am new here and I completed this task and while committing I got the following error :

Trim Trailing Whitespace.................................................Passed
Check JSON...........................................(no files to check)Skipped
Check Yaml...........................................(no files to check)Skipped
flake8...................................................................Passed
codespell................................................................Passed
mypy.....................................................................Failed
- hook id: mypy
- exit code: 1

swh/indexer/rehash.py:11: error: Cannot find implementation or library stub for module named 'swh.core'
swh/indexer/rehash.py:12: error: Cannot find implementation or library stub for module named 'swh.core.config'
swh/indexer/rehash.py:13: error: Cannot find implementation or library stub for module named 'swh.model'
......
......
......
swh/indexer/tests/test_cli.py:23: error: Cannot find implementation or library stub for module named 'swh.model.hashutil'
swh/indexer/tests/test_cli.py:24: error: Cannot find implementation or library stub for module named 'swh.model.model'
swh/indexer/tests/test_origin_metadata.py:17: error: Cannot find implementation or library stub for module named 'swh.model.model'
swh/indexer/tests/test_origin_metadata.py:18: error: Cannot find implementation or library stub for module named 'swh.storage.interface'
Found 95 errors in 34 files (checked 60 source files)

isort....................................................................Passed
black....................................................................Passed

I tried searching on the internet and found that it should get fixed by using export MYPYPATH=$PWD when PWD was swh-environment/swh-indexer. But it didn't work for me. Can someone please help me through this?

Thank you

@KShivendu it sounds like you haven't installed the various swh-* modules with pip -e. Did you follow the developer setup tutorial? Steps in there should take care of that.

vlorentz assigned this task to KShivendu.
vlorentz moved this task from In progress to Done on the Easy hack board.