Page MenuHomeSoftware Heritage
Differential D1690

Add an env var to the in-mem storage to disable origin ids.
ClosedPublic
Actions

Authored by vlorentz on Jul 4 2019, 5:00 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Commits
rDSTOC4a16a103a8b5: Add an env var to the in-mem storage to disable origin ids.
rDSTO4a16a103a8b5: Add an env var to the in-mem storage to disable origin ids.
Summary

If it is true (the default), there is no change from the current behavior,
but setting it to False and running another package's tests allows
to make sure that package does not rely on origin ids at all. (eg. see D1691)

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Jul 4 2019, 5:00 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptJul 4 2019, 5:00 PM
swh-public-ci added a comment.Jul 4 2019, 5:03 PM

Build is green
See https://jenkins.softwareheritage.org/job/DSTO/job/tox/542/ for more details.

Harbormaster completed remote builds in B6694: Diff 5671.Jul 4 2019, 5:03 PM
vlorentz mentioned this in D1691: Stop relying on origin ids unless necessary..Jul 4 2019, 5:06 PM
vlorentz edited the summary of this revision. (Show Details)
ardumont accepted this revision.Jul 5 2019, 2:35 PM
ardumont added a subscriber: ardumont.

sounds fine.

a couple of suggestion to improve readability.

swh/storage/in_memory.py
1146

why not use the same instruction as below?

return origin_id if ENABLE_ORIGIN_IDS else origin['url']
1160

Just a note outside the scope of this diff.
If we aren't using those (fetch_history_{start,end}) anymore (you removed the call from the loader-core IIRC), this should go away as well.

1334

simplify this a bit:

key = 'id' if ENABLE_ORIGIN_IDS else 'url'
visit['origin'] = self._origins[visit['origin']['url']][key]
return visit
This revision is now accepted and ready to land.Jul 5 2019, 2:35 PM
vlorentz updated this revision to Diff 5689.Jul 8 2019, 10:56 AM
  • make it an env var
  • simplify _convert_visit
vlorentz retitled this revision from Add a global boolean to the in-mem storage to disable origin ids. to Add an env var to the in-mem storage to disable origin ids..Jul 8 2019, 10:56 AM
vlorentz edited the summary of this revision. (Show Details)
vlorentz added inline comments.
swh/storage/in_memory.py
1146

I used the ternary to make tests shorter, but in the main code I prefer readability.
I don't have a strong opinion on this, though.

swh-public-ci added a comment.Jul 8 2019, 11:00 AM

Build is green
See https://jenkins.softwareheritage.org/job/DSTO/job/tox/544/ for more details.

Harbormaster completed remote builds in B6710: Diff 5689.Jul 8 2019, 11:00 AM
vlorentz updated this revision to Diff 5694.Jul 8 2019, 1:53 PM

rebase

This revision was landed with ongoing or failed builds.Jul 8 2019, 1:53 PM
Closed by commit rDSTO4a16a103a8b5: Add an env var to the in-mem storage to disable origin ids. (authored by vlorentz). · Explain Why
This revision was automatically updated to reflect the committed changes.
Harbormaster failed remote builds in B6716: Diff 5694!Jul 8 2019, 1:53 PM
swh-public-ci added a comment.Jul 8 2019, 1:53 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tox/545/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tox/545/console

vlorentz mentioned this in D1701: Add tests for the in-mem storage with origin ids disabled..Jul 9 2019, 3:02 PM
vlorentz mentioned this in D1717: Add tests for the in-mem storage with origin ids disabled..Jul 10 2019, 4:14 PM

Revision Contents
Changeset List

PathSize
swh/
storage/
87 lines

Diff 5695

View Options

swh/storage/in_memory.py

Loading...
About · Developer information · Legal