Page MenuHomeSoftware Heritage
Differential D8971

origin_contributors: Write table mapping origin ID to origin URL (base64-encoded)
ClosedPublic
Actions

Authored by vlorentz on Dec 19 2022, 5:55 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T4695: Provide a collaboration graph / dataset
T4729: collaboration graph: drop pseudo-SWHIDs and add mapping ori<->url
Commits
rDGRPHb7a991441696: origin_contributors: Write table mapping origin ID to origin URL (base64…
Summary

Depends on D8970.

Part 2/2 to resolve T4729

Diff Detail

Unit TestsFailed
View All

TimeTest
12 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.graph.tests.test_origin_contributors::test_list_origin_contributors
tmpdir = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_list_origin_contributors0') def test_list_origin_contributors(tmpdir):
4 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.graph.http_naive_client::swh.graph.http_naive_client.NaiveClient
8 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.graph.tests.test_cli::test_luigi[0]
4 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.graph.tests.test_cli::test_luigi[1]
44,953 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.graph.tests.test_cli::test_pipeline
View Full Test Results (1 Failed · 101 Passed · 9 Skipped)

Event Timeline

vlorentz created this revision.Dec 19 2022, 5:55 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptDec 19 2022, 5:55 PM
vlorentz mentioned this in P1543 origin contributors format.Dec 19 2022, 5:58 PM
swh-public-ci added a comment.Dec 19 2022, 6:00 PM

Build has FAILED

Patch application report for D8971 (id=32314)

Could not rebase; Attempt merge onto eccd978d45...

Merge made by the 'recursive' strategy.
 .../java/org/softwareheritage/graph/SwhGraph.java  |  5 +++
 .../softwareheritage/graph/SwhGraphProperties.java |  7 +++-
 .../graph/utils/ListOriginContributors.java        | 25 +++++++++---
 swh/graph/luigi/origin_contributors.py             | 31 +++++++++------
 swh/graph/tests/test_origin_contributors.py        | 46 ++++++++++++++++------
 5 files changed, 84 insertions(+), 30 deletions(-)
Changes applied before test
commit b529365825585d66d545b48a3b29d336a43be76e
Merge: eccd978 393aee1
Author: Jenkins user <jenkins@localhost>
Date:   Mon Dec 19 16:55:36 2022 +0000

    Merge branch 'diff-target' into HEAD

commit 393aee102c9df7c29d40405c35f14c1dacd08a1a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 19 17:54:52 2022 +0100

    origin_contributors: Write table mapping origin ID to origin URL (base64-encoded)

commit 73c36d4a5abb335acacfdba1d5b3ebca3a4e6b49
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 19 17:45:09 2022 +0100

    origin_contributors: Use origin IDs instead of SWHIDs
    
    Origin pseudo-SWHIDs are kind of useless on their own, as there is no public API
    to resolve them yet; so we might as well produce integers instead.
    
    A future commit will add a mapping table to URLs, which are actually
    useful.

Link to build: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/341/
See console output for more information: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/341/console

Harbormaster returned this revision to the author for changes because remote builds failed.Dec 19 2022, 6:00 PM
Harbormaster failed remote builds in B33305: Diff 32314!
vlorentz updated this revision to Diff 32315.Dec 19 2022, 6:09 PM

bleh

vlorentz added a child revision: D8972: origin_contributors: Rename 'person' to 'contributor' in outputs.Dec 19 2022, 6:10 PM
swh-public-ci added a comment.Dec 19 2022, 6:17 PM

Build is green

Patch application report for D8971 (id=32315)

Could not rebase; Attempt merge onto eccd978d45...

Merge made by the 'recursive' strategy.
 .../java/org/softwareheritage/graph/SwhGraph.java  |  5 +++
 .../softwareheritage/graph/SwhGraphProperties.java |  7 +++-
 .../graph/utils/ListOriginContributors.java        | 25 +++++++++---
 swh/graph/luigi/origin_contributors.py             | 32 +++++++++------
 swh/graph/tests/test_origin_contributors.py        | 46 ++++++++++++++++------
 5 files changed, 85 insertions(+), 30 deletions(-)
Changes applied before test
commit 200fd14b6231bd6f51a3fcfde76d226217c72f0c
Merge: eccd978 c5211ea
Author: Jenkins user <jenkins@localhost>
Date:   Mon Dec 19 17:09:40 2022 +0000

    Merge branch 'diff-target' into HEAD

commit c5211ea253b72c88d293daccf503ce2125e10db9
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 19 17:54:52 2022 +0100

    origin_contributors: Write table mapping origin ID to origin URL (base64-encoded)

commit 73c36d4a5abb335acacfdba1d5b3ebca3a4e6b49
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 19 17:45:09 2022 +0100

    origin_contributors: Use origin IDs instead of SWHIDs
    
    Origin pseudo-SWHIDs are kind of useless on their own, as there is no public API
    to resolve them yet; so we might as well produce integers instead.
    
    A future commit will add a mapping table to URLs, which are actually
    useful.

See https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/342/ for more details.

Harbormaster completed remote builds in B33306: Diff 32315.Dec 19 2022, 6:17 PM
vlorentz requested review of this revision.Dec 19 2022, 6:17 PM
vlorentz added a task: T4695: Provide a collaboration graph / dataset.Dec 22 2022, 1:57 PM
ardumont accepted this revision.Jan 2 2023, 2:12 PM
This revision is now accepted and ready to land.Jan 2 2023, 2:12 PM
vlorentz updated this revision to Diff 32395.Jan 3 2023, 11:10 AM

rebase

swh-public-ci added a comment.Jan 3 2023, 11:18 AM

Build is green

Patch application report for D8971 (id=32395)

Could not rebase; Attempt merge onto eccd978d45...

Updating eccd978..b7a9914
Fast-forward
 .../java/org/softwareheritage/graph/SwhGraph.java  |  5 +++
 .../softwareheritage/graph/SwhGraphProperties.java |  7 +++-
 .../graph/utils/ListOriginContributors.java        | 25 +++++++++---
 swh/graph/luigi/origin_contributors.py             | 32 +++++++++------
 swh/graph/tests/test_origin_contributors.py        | 46 ++++++++++++++++------
 5 files changed, 85 insertions(+), 30 deletions(-)
Changes applied before test
commit b7a991441696fdec8a59e7edc4d9f1e7c886dba4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 19 17:54:52 2022 +0100

    origin_contributors: Write table mapping origin ID to origin URL (base64-encoded)

commit 09365b488aa1381923708f1232d7b1ccf43e9555
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 19 17:45:09 2022 +0100

    origin_contributors: Use origin IDs instead of SWHIDs
    
    Origin pseudo-SWHIDs are kind of useless on their own, as there is no public API
    to resolve them yet; so we might as well produce integers instead.
    
    A future commit will add a mapping table to URLs, which are actually
    useful.

See https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/345/ for more details.

Harbormaster completed remote builds in B33381: Diff 32395.Jan 3 2023, 11:18 AM
Closed by commit rDGRPHb7a991441696: origin_contributors: Write table mapping origin ID to origin URL (base64… (authored by vlorentz). · Explain WhyJan 3 2023, 11:39 AM
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDGRPHb7a991441696: origin_contributors: Write table mapping origin ID to origin URL (base64….

Revision Contents
Changeset List

PathSize
java/
src/
main/
java/
org/
softwareheritage/
graph/
5 lines
7 lines
utils/
19 lines
swh/
graph/
luigi/
23 lines
tests/
22 lines

Diff 32314

View Options

java/src/main/java/org/softwareheritage/graph/SwhGraph.java

Loading...
View Options

java/src/main/java/org/softwareheritage/graph/SwhGraphProperties.java

Loading...
View Options

java/src/main/java/org/softwareheritage/graph/utils/ListOriginContributors.java

Loading...
View Options

swh/graph/luigi/origin_contributors.py

Loading...
View Options

swh/graph/tests/test_origin_contributors.py

Loading...
About · Developer information · Legal