Details

Reviewers

Group Reviewers

Commits

rDCIDXc9838220b395: indexer: Change from policy_update to a boolean named skip_existing

Summary

policy_update is a variable to determine whether to update or ignore
duplicates. Prior to this commit, this was conflated with the
incremental nature of the indexer. This declares a meaningful name
instead 'incremental' and force the db policy update writing to always
update conflicting data (if any).

This policy_update variable should probably go away.

Test Plan

tox

Diff Detail

Repository

rDCIDX Metadata indexer

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Nov 16 2018, 4:01 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptNov 16 2018, 4:01 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/44/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/44/console

Harbormaster failed remote builds in B2428: Diff 2102!Nov 16 2018, 4:02 PM

vlorentz added a subscriber: vlorentz.Nov 16 2018, 4:04 PM

vlorentz added inline comments.

swh/indexer/indexer.py
439	Parse error: "Either the indexer is incremental [...] drop or not"

Build has FAILED

because issue to retrieve the new storage's v0.0.110 in pypi.

Collecting swh.storage>=0.0.110 (from swh.indexer==0.0.55.post51)
  Could not find a version that satisfies the requirement swh.storage>=0.0.110 (from swh.indexer==0.0.55.post51) (from versions: 0.0.107, 0.0.108, 0.0.109)
No matching distribution found for swh.storage>=0.0.110 (from swh.indexer==0.0.55.post51)

Fix docstring typo

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/45/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/45/console

Harbormaster failed remote builds in B2430: Diff 2104!Nov 16 2018, 4:11 PM

Change from incremental to skip_existing

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/47/ for more details.

Harbormaster completed remote builds in B2434: Diff 2108.Nov 16 2018, 6:06 PM

ardumont retitled this revision from indexer: Change confusing name policy_update to incremental boolean to indexer: Change confusing name policy_update to skip_existing boolean.Nov 16 2018, 6:51 PM

ardumont marked an inline comment as done.

vlorentz accepted this revision.Nov 16 2018, 8:43 PM

This revision is now accepted and ready to land.Nov 16 2018, 8:43 PM

Change branch then diff update (before merging)

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/48/ for more details.

Harbormaster completed remote builds in B2435: Diff 2109.Nov 17 2018, 9:34 AM

Closed by commit rDCIDXc9838220b395: indexer: Change from policy_update to a boolean named skip_existing (authored by ardumont). · Explain WhyNov 17 2018, 9:35 AM

This revision was automatically updated to reflect the committed changes.