Details

Reviewers

Group Reviewers

Commits

rDCIDXc9838220b395: indexer: Change from policy_update to a boolean named skip_existing

Summary

policy_update is a variable to determine whether to update or ignore
duplicates. Prior to this commit, this was conflated with the
incremental nature of the indexer. This declares a meaningful name
instead 'incremental' and force the db policy update writing to always
update conflicting data (if any).

This policy_update variable should probably go away.

Test Plan

tox

Diff Detail

Repository

rDCIDX Metadata indexer

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 2428
Build 2979: tox-on-jenkins	Jenkins
Build 2978: arc lint + arc unit

Event Timeline

ardumont created this revision.Nov 16 2018, 4:01 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptNov 16 2018, 4:01 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/44/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/44/console

Harbormaster failed remote builds in B2428: Diff 2102!Nov 16 2018, 4:02 PM

vlorentz added a subscriber: vlorentz.Nov 16 2018, 4:04 PM

vlorentz added inline comments.

swh/indexer/indexer.py
440	Parse error: "Either the indexer is incremental [...] drop or not"

Build has FAILED

because issue to retrieve the new storage's v0.0.110 in pypi.

Collecting swh.storage>=0.0.110 (from swh.indexer==0.0.55.post51)
  Could not find a version that satisfies the requirement swh.storage>=0.0.110 (from swh.indexer==0.0.55.post51) (from versions: 0.0.107, 0.0.108, 0.0.109)
No matching distribution found for swh.storage>=0.0.110 (from swh.indexer==0.0.55.post51)

Fix docstring typo

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/45/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/45/console

Harbormaster failed remote builds in B2430: Diff 2104!Nov 16 2018, 4:11 PM

Change from incremental to skip_existing

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/47/ for more details.

Harbormaster completed remote builds in B2434: Diff 2108.Nov 16 2018, 6:06 PM

ardumont retitled this revision from indexer: Change confusing name policy_update to incremental boolean to indexer: Change confusing name policy_update to skip_existing boolean.Nov 16 2018, 6:51 PM

ardumont marked an inline comment as done.

vlorentz accepted this revision.Nov 16 2018, 8:43 PM

This revision is now accepted and ready to land.Nov 16 2018, 8:43 PM

Change branch then diff update (before merging)

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/48/ for more details.

Harbormaster completed remote builds in B2435: Diff 2109.Nov 17 2018, 9:34 AM

Closed by commit rDCIDXc9838220b395: indexer: Change from policy_update to a boolean named skip_existing (authored by ardumont). · Explain WhyNov 17 2018, 9:35 AM

This revision was automatically updated to reflect the committed changes.