lauchpad: review adaptations
ClosedPublic
Actions

Authored by ardumont on Feb 17 2022, 12:02 PM.

Details

Reviewers

Group Reviewers

Commits

rDLS4265e5dd77f0: launchpad: Drop extra filtering step which is no longer necessary
rDLSc86e4b43f489: launchpad: Use tuple instead of list

Summary

Diff to adapt according to base commit:

Use tuple instead of list
drop no longer needed duplicate filtering

Related to D7193

Test Plan

tox

Diff Detail

Repository

rDLS Listers

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 26947
Build 42140: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 42139: arc lint + arc unit

Event Timeline

ardumont created this revision.Feb 17 2022, 12:02 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptFeb 17 2022, 12:02 PM

ardumont mentioned this in D7193: launchpad: Allow bzr origins listing.Feb 17 2022, 12:02 PM

Build is green

Patch application report for D7196 (id=26077)

Could not rebase; Attempt merge onto 31b4429ced...

Updating 31b4429..973110d
Fast-forward
 swh/lister/launchpad/lister.py                     | 145 ++++++++++++++++-----
 .../tests/data/launchpad_bzr_response.json         | 126 ++++++++++++++++++
 swh/lister/launchpad/tests/test_lister.py          | 135 +++++++++++++++----
 3 files changed, 346 insertions(+), 60 deletions(-)
 create mode 100644 swh/lister/launchpad/tests/data/launchpad_bzr_response.json

Changes applied before test

commit 973110d2e38093738c7a4fa255f8756f3a0681db
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Feb 17 12:01:14 2022 +0100

    lauchpad: Use tuple instead of list

commit 1ce3ad1c6801b65ea55a9753fb4263cade349c32
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Feb 17 09:52:19 2022 +0100

    lauchpad: Manage unhandled exceptions when listing
    
    Prior to this commit, the listing could fail when either reading a page or the page of
    results (lauchpad api raises RestfulError). This now retries when those kind of
    exceptions happen. If the error persists (after multiple tryouts and exponential
    backoff), the listing continues nonetheless (with warning logs).
    
    Note that if the page ends up being empty, it's no longer accounted for.
    
    This actually allows the listing to finish in case of issues.
    
    Related to T3945

commit 262f9369c837e293f8389dd9f7a6a965c09f621e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Feb 16 17:56:13 2022 +0100

    launchpad: Allow bzr origins listing
    
    Related to T3945

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/459/ for more details.

Harbormaster completed remote builds in B26945: Diff 26077.Feb 17 2022, 12:04 PM

ardumont requested review of this revision.Feb 17 2022, 12:04 PM

launchpad: Use tuple instead of list
launchpad: Drop extra filtering step which is no longer necessary

ardumont retitled this revision from lauchpad: Use tuple instead of list to lauchpad: review adaptations.Feb 17 2022, 12:14 PM

ardumont edited the summary of this revision. (Show Details)

ardumont added a reviewer: anlambert.

Build is green

Patch application report for D7196 (id=26079)

Could not rebase; Attempt merge onto 31b4429ced...

Updating 31b4429..4265e5d
Fast-forward
 swh/lister/launchpad/lister.py                     | 144 +++++++++++++++------
 .../tests/data/launchpad_bzr_response.json         | 126 ++++++++++++++++++
 swh/lister/launchpad/tests/test_lister.py          | 135 +++++++++++++++----
 3 files changed, 341 insertions(+), 64 deletions(-)
 create mode 100644 swh/lister/launchpad/tests/data/launchpad_bzr_response.json

Changes applied before test

commit 4265e5dd77f0877e79658e16a4ee97e097bf5207
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Feb 17 12:03:43 2022 +0100

    launchpad: Drop extra filtering step which is no longer necessary
    
    as the scheduler is now able to deduplicate it when recording listed origins.
    
    Related to T3945

commit c86e4b43f489345c72f37b28e02050fd087891c8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Feb 17 12:01:14 2022 +0100

    launchpad: Use tuple instead of list
    
    Related to T3945

commit fc2edd24aa4b71376e2bfba6dbcab1fa68af6f72
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Feb 17 09:52:19 2022 +0100

    launchpad: Manage unhandled exceptions when listing
    
    Prior to this commit, the listing could fail when either reading a page or the page of
    results (lauchpad api raises RestfulError). This now retries when those kind of
    exceptions happen. If the error persists (after multiple tryouts and exponential
    backoff), the listing continues nonetheless (with warning logs).
    
    Note that if the page ends up being empty, it's no longer accounted for.
    
    This actually allows the listing to finish in case of issues.
    
    Related to T3945

commit 262f9369c837e293f8389dd9f7a6a965c09f621e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Feb 16 17:56:13 2022 +0100

    launchpad: Allow bzr origins listing
    
    Related to T3945

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/461/ for more details.

Harbormaster completed remote builds in B26947: Diff 26079.Feb 17 2022, 12:16 PM

anlambert accepted this revision.Feb 17 2022, 1:12 PM

This revision is now accepted and ready to land.Feb 17 2022, 1:12 PM

Closed by commit rDLSc86e4b43f489: launchpad: Use tuple instead of list (authored by ardumont). · Explain WhyFeb 17 2022, 1:14 PM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDLSc86e4b43f489: launchpad: Use tuple instead of list.

ardumont added a commit: rDLS4265e5dd77f0: launchpad: Drop extra filtering step which is no longer necessary.