Page MenuHomeSoftware Heritage
Differential D5046

loader.svn: Mark visit status as not_found when relevant
ClosedPublic
Actions

Authored by ardumont on Feb 9 2021, 11:58 AM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Maniphest Tasks
T3030: Improve loaders to deal with new visit status events
Commits
rDLDSVN71e3b4478432: svn.loader: Mark visit status as not_found when relevant
Summary

When failing to communicate with the svn server at initialization time, a visit status
ends up in "not_found" status.

Other failures to communicate ends up in a visit status marked as "failed".

Related to T3030

Test Plan

tox
(failing until swh.loader.core > 0.16 it released)

Diff Detail

Event Timeline

ardumont created this revision.Feb 9 2021, 11:58 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptFeb 9 2021, 11:58 AM
swh-public-ci added a comment.Feb 9 2021, 11:59 AM

Build has FAILED

Patch application report for D5046 (id=17993)

Could not rebase; Attempt merge onto 824880e7d7...

Updating 824880e..7dccd60
Fast-forward
 requirements-swh.txt                   |  2 +-
 swh/loader/git/loader.py               | 10 ++++++---
 swh/loader/git/tests/test_from_disk.py | 41 +++++++++++++++++++++++++++++++++-
 swh/loader/git/tests/test_loader.py    | 40 +++++++++++++++++++++++++++++----
 4 files changed, 84 insertions(+), 9 deletions(-)
Changes applied before test
commit 7dccd6062cf717ca8c76642d215968873c7fed70
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Feb 9 11:53:42 2021 +0100

    loader.git: Mark visit status as not_found when relevant
    
    Related to T3030

commit f65fde77a942476de8173bb70c7eef9717bbffb0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Feb 8 19:07:04 2021 +0100

    loader.git: Explicit the failure test cases
    
    Related to T3030

Link to build: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/76/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/76/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 9 2021, 11:59 AM
Harbormaster failed remote builds in B19114: Diff 17993!
ardumont updated this revision to Diff 18056.Feb 11 2021, 12:07 PM

Rework commit message

swh-public-ci added a comment.Feb 11 2021, 12:09 PM

Build is green

Patch application report for D5046 (id=18056)

Could not rebase; Attempt merge onto 220678c753...

Updating 220678c..3fd2dc5
Fast-forward
 requirements-swh.txt                |  2 +-
 swh/loader/svn/loader.py            | 15 +++++++++++----
 swh/loader/svn/tests/test_loader.py | 16 ++++++++++++++--
 3 files changed, 26 insertions(+), 7 deletions(-)
Changes applied before test
commit 3fd2dc5203ed9cfa4378f1c03ca089528c66bc03
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Feb 8 17:14:26 2021 +0100

    svn.loader: Mark visit status as not_found when relevant
    
    When failing to communicate with the svn server, a visit status ends up in "not_found"
    status.
    
    Related to T3030

commit ddc18d08514725fe7fb59857c8181e2cdaa47e3e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Feb 8 17:36:57 2021 +0100

    svn.loader: Mark visit status as failed
    
    With the latest swh.loader.core 0.17, a partial without without snapshot is failed.
    
    Related to T3030

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/108/ for more details.

Harbormaster completed remote builds in B19168: Diff 18056.Feb 11 2021, 12:09 PM
ardumont requested review of this revision.Feb 11 2021, 12:09 PM
ardumont edited the summary of this revision. (Show Details)Feb 11 2021, 12:10 PM
vlorentz requested changes to this revision.Feb 11 2021, 1:14 PM
vlorentz retitled this revision from loader.git: Mark visit status as not_found when relevant to loader.svn: Mark visit status as not_found when relevant.
vlorentz added a subscriber: vlorentz.

dupe of D5041

This revision now requires changes to proceed.Feb 11 2021, 1:15 PM
ardumont added a comment.Feb 11 2021, 2:15 PM

dupe of D5041

indeed, i mixed the git and svn loader diffs...

ardumont added a comment.Feb 11 2021, 2:22 PM
In D5041#127642, @vlorentz wrote:

That seems too generic. SubversionException has some attributes that can be used to check what caused the error.

Thanks, I'll check.

ardumont added a comment.Feb 11 2021, 2:28 PM
In D5046#127681, @ardumont wrote:
In D5041#127642, @vlorentz wrote:

That seems too generic. SubversionException has some attributes that can be used to check what caused the error.

Thanks, I'll check.

I don't see anything specific though [1]

What did i miss?

[1] https://github.com/jelmer/subvertpy/blob/bc652e106bd2a1c571f464b5fd7042c2282e8101/subvertpy/__init__.py#L102-L108

ardumont updated this revision to Diff 18085.Feb 12 2021, 11:14 AM

Adapt according to review

ardumont edited the summary of this revision. (Show Details)Feb 12 2021, 11:15 AM
swh-public-ci added a comment.Feb 12 2021, 11:16 AM

Build is green

Patch application report for D5046 (id=18085)

Rebasing onto ddc18d0851...

Current branch diff-target is up to date.
Changes applied before test
commit 1da0b342e3437c983a3f8e778b45906c8710c24e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Feb 8 17:14:26 2021 +0100

    svn.loader: Mark visit status as not_found when relevant
    
    When failing to communicate with the svn server at initialization time, a visit status
    ends up in "not_found" status.
    
    Other failures to communicate ends up in a visit status marked as "failed".
    
    Related to T3030

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/109/ for more details.

Harbormaster completed remote builds in B19193: Diff 18085.Feb 12 2021, 11:16 AM
vlorentz accepted this revision.Feb 12 2021, 11:24 AM
vlorentz added inline comments.
swh/loader/svn/tests/test_loader.py
77

"somehow"

This revision is now accepted and ready to land.Feb 12 2021, 11:24 AM
ardumont updated this revision to Diff 18086.Feb 12 2021, 11:28 AM

Drop somehow

swh-public-ci added a comment.Feb 12 2021, 11:29 AM

Build is green

Patch application report for D5046 (id=18086)

Rebasing onto ddc18d0851...

Current branch diff-target is up to date.
Changes applied before test
commit 71e3b4478432429a730cb80687602a12a2030cc1
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Feb 8 17:14:26 2021 +0100

    svn.loader: Mark visit status as not_found when relevant
    
    When failing to communicate with the svn server at initialization time, a visit status
    ends up in "not_found" status.
    
    Other failures to communicate ends up in a visit status marked as "failed".
    
    Related to T3030

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/110/ for more details.

Harbormaster completed remote builds in B19194: Diff 18086.Feb 12 2021, 11:30 AM
Closed by commit rDLDSVN71e3b4478432: svn.loader: Mark visit status as not_found when relevant (authored by ardumont). · Explain WhyFeb 12 2021, 11:31 AM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDLDSVN71e3b4478432: svn.loader: Mark visit status as not_found when relevant.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
loader/
svn/
25 lines83%
Loading...
tests/
56 lines100%
Loading...

Diff 18085

View Options

swh/loader/svn/loader.py

Loading...
View Options

swh/loader/svn/tests/test_loader.py

Loading...
About · Developer information · Legal