Page MenuHomeSoftware Heritage
Differential D3308

loader: Read snapshot out of the last visit status
ClosedPublic
Actions

Authored by ardumont on Jun 17 2020, 5:30 PM.

Details

Reviewers
douardda
Group Reviewers
Reviewers
Maniphest Tasks
T2310: Make origin visits immutable
Commits
rDLDG55ff84b1a6fe: loader: Read snapshot out of the last origin visit status
Summary

Preparatory work to drop unused fields from OriginVisit

Related to T2310

Diff Detail

Repository
rDLDG Git loader
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 13026
Build 19856: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 19855: arc lint + arc unit

Event Timeline

ardumont created this revision.Jun 17 2020, 5:30 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptJun 17 2020, 5:30 PM
ardumont mentioned this in T2310: Make origin visits immutable.Jun 17 2020, 5:31 PM
swh-public-ci added a comment.Jun 17 2020, 5:31 PM

Build is green

Patch application report for D3308 (id=11714)

Rebasing onto 10376d5da4...

Current branch diff-target is up to date.
Changes applied before test
commit 223d53b96c9a65c78463b1168c2fcde6a98a086c
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 17 17:29:53 2020 +0200

    loader: Read snapshot out of the last visit status
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/21/ for more details.

Harbormaster completed remote builds in B12923: Diff 11714.Jun 17 2020, 5:31 PM
ardumont planned changes to this revision.Jun 18 2020, 7:49 AM
ardumont mentioned this in D3313: algos.origin: Open origin-get-latest-visit-status function.Jun 18 2020, 1:45 PM
ardumont mentioned this in D3305: package.loader: Retrieve latest snapshot out of the latest visit status.Jun 19 2020, 3:40 PM
ardumont updated this revision to Diff 11770.Jun 19 2020, 7:11 PM

Drop duplication
(Tests will fail as a new release is needed for the loader-core)

swh-public-ci added a comment.Jun 19 2020, 7:12 PM

Build has FAILED

Patch application report for D3308 (id=11770)

Rebasing onto 10376d5da4...

Current branch diff-target is up to date.
Changes applied before test
commit 046e1d5681c2acf16d99f8ae1e85740425a915d9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 17 17:29:53 2020 +0200

    loader: Read snapshot out of the last visit status
    
    Related to T2310

Link to build: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/22/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/22/console

Harbormaster failed remote builds in B12980: Diff 11770!Jun 19 2020, 7:12 PM
ardumont updated this revision to Diff 11810.Jun 22 2020, 4:30 PM

Simplify diff to use swh.storage.algos.snapshot.snapshot-get-latest
function

ardumont updated this revision to Diff 11811.Jun 22 2020, 4:30 PM

Add missing requirements change

swh-public-ci added a comment.Jun 22 2020, 4:31 PM

Build is green

Patch application report for D3308 (id=11810)

Rebasing onto 473a615a96...

Current branch diff-target is up to date.
Changes applied before test
commit 169acd0802405a5979bd55b69ae17576231c0f1e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 22 16:30:08 2020 +0200

    loader: Retrieve latest snapshot with snapshot-get-latest function
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/23/ for more details.

Harbormaster completed remote builds in B13025: Diff 11810.Jun 22 2020, 4:31 PM
swh-public-ci added a comment.Jun 22 2020, 4:33 PM

Build is green

Patch application report for D3308 (id=11811)

Rebasing onto 473a615a96...

Current branch diff-target is up to date.
Changes applied before test
commit 132217e2eb24236e3bc25a9fd2e1e124782fa188
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 22 16:30:08 2020 +0200

    loader: Retrieve latest snapshot with snapshot-get-latest function
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/24/ for more details.

Harbormaster completed remote builds in B13026: Diff 11811.Jun 22 2020, 4:33 PM
ardumont added inline comments.Jun 22 2020, 4:34 PM
swh/loader/git/loader.py
213

hmmm, wondering if that should not go directly in swh.loader.core.loader then...
That way, remaining dvcs loaders (svn, git, hg) have access to it directly...
get-last-snapshot or something

ardumont added subscribers: douardda, olasd.Jun 23 2020, 10:45 AM
ardumont added inline comments.
swh/loader/git/from_disk.py
144

From discussion with @olasd and @douardda, that could be reworked this to origin_get_latest_visit_status instead.

ardumont updated this revision to Diff 11826.Jun 23 2020, 11:23 AM

Use origin-get-latest-visit-status in from_disk loader

We don't need to retrieve the snapshot object so directly retrieve its
reference.

swh-public-ci added a comment.Jun 23 2020, 11:24 AM

Build is green

Patch application report for D3308 (id=11826)

Rebasing onto 473a615a96...

Current branch diff-target is up to date.
Changes applied before test
commit c901e60b024632aaa6750cd7df6f93bb9a9aa770
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 22 16:30:08 2020 +0200

    loader: Retrieve latest snapshot information
    
    using latest snapshot-get-latest and origin-get-latest-visit-status functions
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/25/ for more details.

Harbormaster completed remote builds in B13040: Diff 11826.Jun 23 2020, 11:24 AM
ardumont updated this revision to Diff 11828.Jun 23 2020, 11:39 AM

Drop unnecessary moved instruction

swh/loader/git/from_disk.py
144

done

swh-public-ci added a comment.Jun 23 2020, 11:40 AM

Build is green

Patch application report for D3308 (id=11828)

Rebasing onto 473a615a96...

Current branch diff-target is up to date.
Changes applied before test
commit 55ff84b1a6fe7e83844ad0f8b1375ea1c2a983ea
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 22 16:30:08 2020 +0200

    loader: Read snapshot out of the last origin visit status
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/26/ for more details.

Harbormaster completed remote builds in B13042: Diff 11828.Jun 23 2020, 11:40 AM
douardda accepted this revision.Jun 23 2020, 2:30 PM
This revision is now accepted and ready to land.Jun 23 2020, 2:30 PM
Closed by commit rDLDG55ff84b1a6fe: loader: Read snapshot out of the last origin visit status (authored by ardumont). · Explain WhyJun 23 2020, 2:32 PM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDLDG55ff84b1a6fe: loader: Read snapshot out of the last origin visit status.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
2 lines--
swh/
loader/
git/
10 lines74%
Loading...
11 lines80%
Loading...

Diff 11811

View Options

requirements-swh.txt

Loading...
View Options

swh/loader/git/from_disk.py

Loading...
View Options

swh/loader/git/loader.py

Loading...
About · Developer information · Legal