Page MenuHomeSoftware Heritage

Make buffer and validate proxy storage also handle other object types
AcceptedPublic

Authored by douardda on Apr 29 2021, 3:28 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Summary

but OriginVisit (which requires _add() method normalization).

Also remove unused 'FilteringProxyStorage.object_types' class attribute.

Diff Detail

Event Timeline

forgot origin for the buffer proxy

Build is green

Patch application report for D5646 (id=20159)

Rebasing onto 92d551a4a5...

Current branch diff-target is up to date.
Changes applied before test
commit 75fd304bb204274b027c4b9bee98f33cf38ad910
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Apr 29 15:25:08 2021 +0200

    Make buffer and validate proxy storage also handle other object types
    
    but OriginVisit (which requires _add() method normalization).
    
    Also remove unused 'FilteringProxyStorage.object_types' class attribute.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1289/ for more details.

also forgot the proper origin_add() method in the buffer proxy

Build is green

Patch application report for D5646 (id=20161)

Rebasing onto 92d551a4a5...

Current branch diff-target is up to date.
Changes applied before test
commit 1d575fc8ebeef8ee38e819af6aa1909e93fce76d
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Apr 29 15:25:08 2021 +0200

    Make buffer and validate proxy storage also handle other object types
    
    but OriginVisit (which requires _add() method normalization).
    
    Also remove unused 'FilteringProxyStorage.object_types' class attribute.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1290/ for more details.

Build is green

Patch application report for D5646 (id=20162)

Rebasing onto 92d551a4a5...

Current branch diff-target is up to date.
Changes applied before test
commit c8e5bcdf9b8c5575cd4779e759a69b001cfe1c85
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Apr 29 15:25:08 2021 +0200

    Make buffer and validate proxy storage also handle other object types
    
    but OriginVisit (which requires _add() method normalization).
    
    Also remove unused 'FilteringProxyStorage.object_types' class attribute.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1291/ for more details.

and the origin_visit_status_add() specific implementation as well...

Build was aborted

Patch application report for D5646 (id=20164)

Rebasing onto 92d551a4a5...

Current branch diff-target is up to date.
Changes applied before test
commit 280386107925c29404c4a8caec48d222ebfb0d01
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Apr 29 15:25:08 2021 +0200

    Make buffer and validate proxy storage also handle other object types
    
    but OriginVisit (which requires _add() method normalization).
    
    Also remove unused 'FilteringProxyStorage.object_types' class attribute.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1293/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1293/console

Build is green

Patch application report for D5646 (id=20166)

Rebasing onto 92d551a4a5...

Current branch diff-target is up to date.
Changes applied before test
commit 0161570ec16117df782cded80f6a3704e9aaa39f
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Apr 29 15:25:08 2021 +0200

    Make buffer and validate proxy storage also handle other object types
    
    but OriginVisit (which requires _add() method normalization).
    
    Also remove unused 'FilteringProxyStorage.object_types' class attribute.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1295/ for more details.

vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/storage/buffer.py
64

hmm, they can be pretty big. You may want to divide this by 10

This revision is now accepted and ready to land.Apr 29 2021, 4:39 PM
swh/storage/buffer.py
64

I wasn't sure, so ok

The buffer bits look reasonable to me. I'm not so sure about the validator bits, as we're not actually storing the id fields for these objects, afaik?