Page MenuHomeSoftware Heritage

journal-client: Consume from origin_extrinsic_metadata topic
ClosedPublic

Authored by vlorentz on Jul 19 2022, 10:52 AM.

Details

Summary

In addition to origin_intrinsic_metadata.

If both sources exist for a given origin, they will be merged by ElasticSearch
as usual.

Diff Detail

Repository
rDSEA Archive search
Branch
extrinsic
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 30444
Build 47592: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 47591: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8131 (id=29370)

Could not rebase; Attempt merge onto 132b3b1238...

Updating 132b3b1..db0d35c
Fast-forward
 swh/search/elasticsearch.py             | 26 +++++-----
 swh/search/in_memory.py                 | 24 ++++-----
 swh/search/interface.py                 |  8 +--
 swh/search/journal_client.py            | 22 +++++++-
 swh/search/tests/test_cli.py            | 42 +++++++++++----
 swh/search/tests/test_journal_client.py |  9 ++--
 swh/search/tests/test_search.py         | 92 ++++++++++++++++-----------------
 swh/search/tests/test_translator.py     | 20 +++----
 swh/search/translator.py                | 16 +++---
 swh/search/utils.py                     | 14 ++---
 10 files changed, 158 insertions(+), 115 deletions(-)
Changes applied before test
commit db0d35c96f02cdb22f3b7a033cb35da9322ed28e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jul 19 10:52:33 2022 +0200

    journal-client: Consume from origin_extrinsic_metadata topic
    
    In addition to origin_intrinsic_metadata.
    
    If both sources exist for a given origin, they will be merged by ElasticSearch
    as usual.

commit 19a825889cfeac4f9019dcffe121508cf7a41c51
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jul 19 10:24:57 2022 +0200

    Rename "intrinsic_metadata" field to "jsonld"
    
    It will soon be used to store and query extrinsic metadata as well

See https://jenkins.softwareheritage.org/job/DSEA/job/tests-on-diff/337/ for more details.

This revision is now accepted and ready to land.Jul 19 2022, 12:44 PM