Extrinsic metadata indexers will not use a 'file' as input,
but will typically use RawExtrinsicMetadata containing formats
in JSON.
Details
Details
- Reviewers
ardumont - Group Reviewers
Reviewers - Maniphest Tasks
- T2073: Index extrinsic metadata from the journal in swh-search/Elasticsearch
- Commits
- rDCIDXe6b03db91c99: Remove SingleFileMapping from JsonMapping's base classes
Diff Detail
Diff Detail
- Repository
- rDCIDX Metadata indexer
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
Build is green
Patch application report for D8046 (id=29032)
Could not rebase; Attempt merge onto 1be4e184d4...
Updating 1be4e18..65edef3 Fast-forward swh/indexer/metadata.py | 41 ++++++++++++++++++--------------- swh/indexer/metadata_detector.py | 8 +++++-- swh/indexer/metadata_dictionary/base.py | 26 ++++++++++++--------- swh/indexer/metadata_dictionary/npm.py | 4 ++-- swh/indexer/metadata_dictionary/ruby.py | 7 ++++-- 5 files changed, 51 insertions(+), 35 deletions(-)
Changes applied before test
commit 65edef32831949de7b8e14846ecd4fa43bc619ee Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 29 12:08:49 2022 +0200 Remove SingleFileMapping from JsonMapping's base classes Extrinsic metadata indexers will not use a 'file' as input, but will typically use RawExtrinsicMetadata containing formats in JSON. commit f7a4bf4e04b3ac4c2fa89cf9b8a5c22e5f0c4d12 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 29 11:01:35 2022 +0200 Add typing to detect_metadata() and related functions
See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/287/ for more details.
Comment Actions
Build is green
Patch application report for D8046 (id=29114)
Could not rebase; Attempt merge onto 1be4e184d4...
Updating 1be4e18..6293230 Fast-forward swh/indexer/metadata.py | 44 ++++++++++++++++++--------------- swh/indexer/metadata_detector.py | 8 ++++-- swh/indexer/metadata_dictionary/base.py | 26 ++++++++++--------- swh/indexer/metadata_dictionary/npm.py | 4 +-- swh/indexer/metadata_dictionary/ruby.py | 7 ++++-- 5 files changed, 52 insertions(+), 37 deletions(-)
Changes applied before test
commit 62932303c8b8a2a551697f562b0406c03b457712 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 29 12:08:49 2022 +0200 Remove SingleFileMapping from JsonMapping's base classes Extrinsic metadata indexers will not use a 'file' as input, but will typically use RawExtrinsicMetadata containing formats in JSON. commit dc035a669329272713b0b977ba2f685501e80ba0 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 29 11:01:35 2022 +0200 Add typing to detect_metadata() and related functions This also moves the call to `detect_metadata()` to `translate_directory_intrinsic_metadata` so type annotations make more sense; and remove a dead/broken code branch in `DirectoryMetadataIndexer.index()` that was detected by mypy.
See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/309/ for more details.
Comment Actions
Build was aborted
Patch application report for D8046 (id=29117)
Could not rebase; Attempt merge onto 1be4e184d4...
Updating 1be4e18..e6b03db Fast-forward swh/indexer/metadata.py | 52 +++++++++++++++++---------------- swh/indexer/metadata_detector.py | 8 +++-- swh/indexer/metadata_dictionary/base.py | 26 ++++++++++------- swh/indexer/metadata_dictionary/npm.py | 4 +-- swh/indexer/metadata_dictionary/ruby.py | 7 +++-- 5 files changed, 55 insertions(+), 42 deletions(-)
Changes applied before test
commit e6b03db91c994fc1d680b76bf0798054a8120b79 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 29 12:08:49 2022 +0200 Remove SingleFileMapping from JsonMapping's base classes Extrinsic metadata indexers will not use a 'file' as input, but will typically use RawExtrinsicMetadata containing formats in JSON. commit 8c5a84459d4436af7b4b751e48091a99bce0a7d1 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Wed Jun 29 11:01:35 2022 +0200 Add typing to detect_metadata() and related functions This also moves the call to `detect_metadata()` to `translate_directory_intrinsic_metadata` so type annotations make more sense; and remove a dead/broken code branch in `DirectoryMetadataIndexer.index()` that was detected by mypy.
Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/312/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/312/console