Differential D5364

package.loader: Write to the ExtID storage
ClosedPublic
Actions

Authored by vlorentz on Mar 26 2021, 5:39 PM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Maniphest Tasks

T3142: Make loaders write to the ExtId storage

Commits

rDLDBASEe48ced028c50: package.loader: Write to the ExtID storage

Summary

This allows future runs of a loader to know a package was already
loaded, without querying each of the revisions individually and
parsing their metadata.

Eventually, this will allow us to get rid of the 'metadata' column
on the 'revision' table entirely.

Test Plan

They shall not pass, because they depend on D5363.

Diff Detail

Repository

rDLDBASE Generic VCS/Package Loader

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Mar 26 2021, 5:39 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptMar 26 2021, 5:39 PM

Build has FAILED

Patch application report for D5364 (id=19217)

Rebasing onto a32f687151...

Current branch diff-target is up to date.

Changes applied before test

commit 0230b713eb74a2c7d2d0d073d4f9b3461cd13bf5
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Mar 26 17:38:27 2021 +0100

    package.loader: Write to the ExtID storage
    
    This allows future runs of a loader to know a package was already
    loaded, without querying each of the revisions individually and
    parsing their metadata.
    
    Eventually, this will allow us to get rid of the 'metadata' column
    on the 'revision' table entirely.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/450/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/450/console

Harbormaster returned this revision to the author for changes because remote builds failed.Mar 26 2021, 5:40 PM

Harbormaster failed remote builds in B20299: Diff 19217!

vlorentz requested review of this revision.Mar 26 2021, 5:40 PM

vlorentz added a parent revision: D5363: extid: remove unicity on (extid_type, extid) and (target_type, target).

vlorentz added a task: T3142: Make loaders write to the ExtId storage.Mar 29 2021, 10:55 AM

Build has FAILED

Patch application report for D5364 (id=19217)

Rebasing onto a32f687151...

Current branch diff-target is up to date.

Changes applied before test

commit 0230b713eb74a2c7d2d0d073d4f9b3461cd13bf5
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Mar 26 17:38:27 2021 +0100

    package.loader: Write to the ExtID storage
    
    This allows future runs of a loader to know a package was already
    loaded, without querying each of the revisions individually and
    parsing their metadata.
    
    Eventually, this will allow us to get rid of the 'metadata' column
    on the 'revision' table entirely.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/451/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/451/console

fix test

Build is green

Patch application report for D5364 (id=19253)

Rebasing onto a32f687151...

Current branch diff-target is up to date.

Changes applied before test

commit e48ced028c5032ee7ddd9f6dda419ebc90699c2e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Mar 26 17:38:27 2021 +0100

    package.loader: Write to the ExtID storage
    
    This allows future runs of a loader to know a package was already
    loaded, without querying each of the revisions individually and
    parsing their metadata.
    
    Eventually, this will allow us to get rid of the 'metadata' column
    on the 'revision' table entirely.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/452/ for more details.

Harbormaster completed remote builds in B20323: Diff 19253.Mar 29 2021, 3:47 PM