Page MenuHomeSoftware Heritage

Instantiate a noop objstorage for testing purposes
ClosedPublic

Authored by ardumont on Oct 25 2021, 5:25 PM.

Details

Summary

This also prevents the main server from running the noop objstorage in production.

Related to T3627

Test Plan

tox (happy)

Use docker-compose with override to allow using such objstorage without eating all disk
when running big tests on repositories.

Diff Detail

Repository
rDOBJS Object storage
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24721
Build 38588: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 38587: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D6548 (id=23799)

Rebasing onto 8ed5f4ebc9...

First, rewinding head to replay your work on top of it...
Applying: Instantiate a noop objstorage for testing purposes
Using index info to reconstruct a base tree...
M	swh/objstorage/factory.py
Falling back to patching base and 3-way merge...
Auto-merging swh/objstorage/factory.py
CONFLICT (content): Merge conflict in swh/objstorage/factory.py
Patch failed at 0001 Instantiate a noop objstorage for testing purposes

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

Rebase failed (ret=1)!

Could not rebase; Attempt merge onto 8ed5f4ebc9...

Already up to date.
Changes applied before test
commit ac0b08b12d15f29e3811091f39f3c508f795bfce
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Oct 25 17:23:41 2021 +0200

    Instantiate a noop objstorage for testing purposes
    
    Related to T3627

Link to build: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/65/
See console output for more information: https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/65/console

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 25 2021, 5:26 PM
Harbormaster failed remote builds in B24683: Diff 23799!
ardumont edited the summary of this revision. (Show Details)

Focus only on preventing noop from running in production.

Build is green

Patch application report for D6548 (id=23800)

Rebasing onto 8ed5f4ebc9...

First, rewinding head to replay your work on top of it...
Applying: Instantiate a noop objstorage for testing purposes
Using index info to reconstruct a base tree...
M	swh/objstorage/factory.py
Falling back to patching base and 3-way merge...
Auto-merging swh/objstorage/factory.py
CONFLICT (content): Merge conflict in swh/objstorage/factory.py
Patch failed at 0001 Instantiate a noop objstorage for testing purposes

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

Rebase failed (ret=1)!

Could not rebase; Attempt merge onto 8ed5f4ebc9...

Already up to date.
Changes applied before test
commit ad1f2de3e9b26dd1b0ecf5486cbe4549ffb772a6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Oct 25 17:23:41 2021 +0200

    Instantiate a noop objstorage for testing purposes
    
    Related to T3627

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/66/ for more details.

swh/objstorage/backends/noop.py
14

i have a better docstring locally:

Noop objstorage. Basic implementation which does no operations at all.

    Only intended for test purposes to avoid either memory or i/o operations. This
    allows swh clients to use the swh stack without having to deal with objstorage
    configuration. So users can concentrate on testing the remaining part of the stack
    without the objstorage.
vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/objstorage/api/server.py
258
This revision is now accepted and ready to land.Oct 27 2021, 12:44 PM

Adapt according to suggestion

Build is green

Patch application report for D6548 (id=23837)

Rebasing onto 8ed5f4ebc9...

First, rewinding head to replay your work on top of it...
Applying: Instantiate a noop objstorage for testing purposes
Using index info to reconstruct a base tree...
M	swh/objstorage/factory.py
Falling back to patching base and 3-way merge...
Auto-merging swh/objstorage/factory.py
CONFLICT (content): Merge conflict in swh/objstorage/factory.py
Patch failed at 0001 Instantiate a noop objstorage for testing purposes

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

Rebase failed (ret=1)!

Could not rebase; Attempt merge onto 8ed5f4ebc9...

Already up to date.
Changes applied before test
commit 35afa820403f73beb809efce0aee0a4d7cc3dabc
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Oct 25 17:23:41 2021 +0200

    Instantiate a noop objstorage for testing purposes
    
    Related to T3627

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/67/ for more details.

Build is green

Patch application report for D6548 (id=23838)

Rebasing onto 8ed5f4ebc9...

Current branch diff-target is up to date.
Changes applied before test
commit 64acd020aa7c4d430b9f6b9c0b6c07438a6ebfb9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Oct 25 17:23:41 2021 +0200

    Instantiate a noop objstorage for testing purposes
    
    Related to T3627

See https://jenkins.softwareheritage.org/job/DOBJS/job/tests-on-diff/68/ for more details.