Page MenuHomeSoftware Heritage

journal_client: Add origins processing
ClosedPublic

Authored by anlambert on Jun 22 2021, 4:50 PM.

Details

Summary

Process origin messages in order to extract an count
network locations in origin URLs.

Related to T3127

Depends on D5909

Diff Detail

Repository
rDCNT Archive counters
Branch
swh-counters-origins
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 22194
Build 34539: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 34538: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D5910 (id=21204)

Could not rebase; Attempt merge onto f4a98c8095...

Updating f4a98c8..a6ffa7e
Fast-forward
 .gitignore                                |   1 +
 requirements-test.txt                     |   3 +-
 swh/counters/journal_client.py            |  22 ++++++
 swh/counters/tests/test_journal_client.py | 115 ++++++++++++++++++------------
 4 files changed, 95 insertions(+), 46 deletions(-)
Changes applied before test
commit a6ffa7eea1993a8e8109888a4e13705f957287fd
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Jun 22 15:01:55 2021 +0200

    journal_client: Add origins processing
    
    Process origin messages in order to extract an count
    network locations in origin URLs.
    
    Related to T3127

commit e9fd9ff7900df7388c7c9c96997cdb8c021a560b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Jun 22 15:23:17 2021 +0200

    tests/journal_client: Use pytest-redis fixture instead of mocks
    
    It makes the tests much simpler to read, use of mocks should be avoided
    when it is possible.

See https://jenkins.softwareheritage.org/job/DCNT/job/tests-on-diff/55/ for more details.

This revision is now accepted and ready to land.Jun 22 2021, 5:06 PM

Build is green

Patch application report for D5910 (id=21207)

Could not rebase; Attempt merge onto d4ce0c497f...

Updating d4ce0c4..374f82b
Fast-forward
 .gitignore                                |   1 +
 requirements-test.txt                     |   3 +-
 swh/counters/journal_client.py            |  22 ++++++
 swh/counters/tests/test_journal_client.py | 115 ++++++++++++++++++------------
 4 files changed, 95 insertions(+), 46 deletions(-)
Changes applied before test
commit 374f82b1e26234c52fdd3a9ab4f6f4afb8e7020a
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Jun 22 15:01:55 2021 +0200

    journal_client: Add origins processing
    
    Process origin messages in order to extract an count
    network locations in origin URLs.
    
    Related to T3127

commit 2be3cf1bc226ac2b5e9051b0786926b2e0e48fa7
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Jun 22 15:23:17 2021 +0200

    tests/journal_client: Use pytest-redis fixture instead of mocks
    
    It makes the tests much simpler to read, use of mocks should be avoided
    when it is possible.

See https://jenkins.softwareheritage.org/job/DCNT/job/tests-on-diff/57/ for more details.

Build is green

Patch application report for D5910 (id=21210)

Could not rebase; Attempt merge onto d4ce0c497f...

Updating d4ce0c4..cd595e7
Fast-forward
 .gitignore                                |   1 +
 requirements-test.txt                     |   3 +-
 swh/counters/journal_client.py            |  22 ++++++
 swh/counters/tests/test_journal_client.py | 120 +++++++++++++++++-------------
 4 files changed, 95 insertions(+), 51 deletions(-)
Changes applied before test
commit cd595e71aef44f93972013fb13bf1f77312283ea
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Jun 22 15:01:55 2021 +0200

    journal_client: Add origins processing
    
    Process origin messages in order to extract an count
    network locations in origin URLs.
    
    Related to T3127

commit a3ea67faa9b995990d58fbba45e5342002304294
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Jun 22 15:23:17 2021 +0200

    tests/journal_client: Use pytest-redis fixture instead of mocks
    
    It makes the tests much simpler to read, use of mocks should be avoided
    when it is possible.

See https://jenkins.softwareheritage.org/job/DCNT/job/tests-on-diff/59/ for more details.

This revision was automatically updated to reflect the committed changes.