Page MenuHomeSoftware Heritage

Use all snapshots to compute base_snapshot_reverse_branches
ClosedPublic

Authored by vlorentz on May 16 2022, 2:30 PM.

Details

Reviewers
olasd
Group Reviewers
Reviewers
Maniphest Tasks
Restricted Maniphest Task
Commits
rDLDGcf674f7c795e: Use all snapshots to compute base_snapshot_reverse_branches
Summary

base_snapshot_reverse_branches needs to contain all objects that
may be a snapshot target that the remote did not send to us.

Because we now use all snapshots to build the "have" list, such
targets include all targets of snapshots of "parent" origins,
not just the previous snapshot of the current origin.

This typically happens when a forge-fork pull branches from its
parent.

Resolves T4248 (Sentry issue SWH-LOADER-GIT-102)

Diff Detail

Repository
rDLDG Git loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build has FAILED

Patch application report for D7834 (id=28299)

Rebasing onto 9b47b24b98...

Current branch diff-target is up to date.
Changes applied before test
commit 6f66c7f2ab1edbbdbf49dfe5ee27733c41446a09
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon May 16 14:26:16 2022 +0200

    Use all snapshots to compute base_snapshot_reverse_branches
    
    base_snapshot_reverse_branches needs to contain all objects that
    may be a snapshot target that the remote did not send to us.
    
    Because we now use all snapshots to build the "have" list, such
    targets include all targets of snapshots of "parent" origins,
    not just the previous snapshot of the current origin.
    
    This typically happens when a forge-fork pull branches from its
    parent.
    
    Resolves Sentry issue [[ https://sentry.softwareheritage.org/share/issue/1c08f5d764e7494e83ba254dc47f17af/ | SWH-LOADER-GIT-102 ]]

Link to build: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/212/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/212/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 16 2022, 2:32 PM
Harbormaster failed remote builds in B29401: Diff 28299!

failure is due to addition of type annotations in the latest Dulwich version. I'll open a different diff to solve the issue.

Build is green

Patch application report for D7834 (id=28301)

Could not rebase; Attempt merge onto 9b47b24b98...

Updating 9b47b24..cf674f7
Fast-forward
 swh/loader/git/loader.py            |  11 +++-
 swh/loader/git/tests/test_loader.py | 123 +++++++++++++++++++++++++++++++++++-
 2 files changed, 131 insertions(+), 3 deletions(-)
Changes applied before test
commit cf674f7c795e2c727177df8f702b75ede0cf65b9
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon May 16 14:26:16 2022 +0200

    Use all snapshots to compute base_snapshot_reverse_branches
    
    base_snapshot_reverse_branches needs to contain all objects that
    may be a snapshot target that the remote did not send to us.
    
    Because we now use all snapshots to build the "have" list, such
    targets include all targets of snapshots of "parent" origins,
    not just the previous snapshot of the current origin.
    
    This typically happens when a forge-fork pull branches from its
    parent.
    
    Resolves Sentry issue [[ https://sentry.softwareheritage.org/share/issue/1c08f5d764e7494e83ba254dc47f17af/ | SWH-LOADER-GIT-102 ]]

commit af41ebde0942257bf11b864856da03e2c252c8e9
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon May 16 14:41:53 2022 +0200

    Don't assume all GitClient instances have a 'dumb' attribute
    
    Only instances of (subclasses of) AbstractHttpGitClient have this attribute.
    
    For other instances, we can consider it to be False, because the dumb
    protocol only exists over HTTP(S).
    
    This issue was found by mypy, thanks to the addition of type annotations
    to changes in dulwich 0.20.36 affecting mypy's type inference.

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/214/ for more details.

olasd added a subscriber: olasd.

Oops, thanks!

This revision is now accepted and ready to land.May 16 2022, 3:07 PM