Differential D8135

rehash: Call objstorage.content_get() with a HashDict instead of single hash
ClosedPublic
Actions

Authored by vlorentz on Jul 19 2022, 2:10 PM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Maniphest Tasks

T4402: Pass dict of hashes instead of single sha1 to objstorage.get()

Commits

rDCIDX42cb37769714: rehash: Call objstorage.content_get() with a HashDict instead of single hash

Summary

Hash dicts are now prefered by swh-objstorage, in order to support
individual hash collisions.

Test Plan

Depends on D8122

Diff Detail

Repository

rDCIDX Metadata indexer

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Jul 19 2022, 2:10 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJul 19 2022, 2:10 PM

Build has FAILED

Patch application report for D8135 (id=29378)

Rebasing onto fa67b73d6a...

First, rewinding head to replay your work on top of it...
Applying: rehash: Call objstorage.content_get() with a HashDict instead of single hash

Changes applied before test

commit b319f2a4f54bd9ac676abf805b39cc47aff1787b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jul 19 14:09:46 2022 +0200

    rehash: Call objstorage.content_get() with a HashDict instead of single hash
    
    Hash dicts are now prefered by swh-objstorage, in order to support
    individual hash collisions.

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/361/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/361/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 19 2022, 2:11 PM

Harbormaster failed remote builds in B30452: Diff 29378!

vlorentz requested review of this revision.Jul 19 2022, 2:39 PM

vlorentz edited the test plan for this revision. (Show Details)

vlorentz added a parent revision: D8122: Fix crash when calling __contains__/get/check/delete with composite obj ids.

vlorentz added a task: T4402: Pass dict of hashes instead of single sha1 to objstorage.get().Jul 19 2022, 3:16 PM

Build is green

Patch application report for D8135 (id=29378)

Rebasing onto 466108c166...

First, rewinding head to replay your work on top of it...
Applying: rehash: Call objstorage.content_get() with a HashDict instead of single hash

Changes applied before test

commit 16cb9749f0015f49345ac9be59d0c68023aa941a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jul 19 14:09:46 2022 +0200

    rehash: Call objstorage.content_get() with a HashDict instead of single hash
    
    Hash dicts are now prefered by swh-objstorage, in order to support
    individual hash collisions.

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/452/ for more details.

Harbormaster completed remote builds in B30452: Diff 29378.Aug 23 2022, 8:16 AM

lgtm

Note: That's actually a tool (and not an indexer). It was used once some (long) time ago to add the blake
columns to the content model.

We never used it after that. I wonder whether we want to keep it or simply drop it to reduce
the maintenance volume to a minimum in that part of the code. And if we want to keep it, we might
want to move it somewhere else, make it a cli probably using the journal as well...

This revision is now accepted and ready to land.Aug 23 2022, 10:47 AM

rebase

This revision was landed with ongoing or failed builds.Aug 30 2022, 11:58 AM

Closed by commit rDCIDX42cb37769714: rehash: Call objstorage.content_get() with a HashDict instead of single hash (authored by vlorentz). · Explain Why

This revision was automatically updated to reflect the committed changes.

vlorentz added a commit: rDCIDX42cb37769714: rehash: Call objstorage.content_get() with a HashDict instead of single hash.

Build is green

Patch application report for D8135 (id=30121)

Rebasing onto 85b675fd19...

Current branch diff-target is up to date.

Changes applied before test

commit 42cb37769714e245803023ccf02105006dd0e474
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jul 19 14:09:46 2022 +0200

    rehash: Call objstorage.content_get() with a HashDict instead of single hash
    
    Hash dicts are now prefered by swh-objstorage, in order to support
    individual hash collisions.

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/466/ for more details.

Harbormaster completed remote builds in B31164: Diff 30121.Aug 30 2022, 12:06 PM