Differential D1754

Send smaller batches to origin_get.
ClosedPublic
Actions

Authored by vlorentz on Jul 19 2019, 3:52 PM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Commits

rDCIDX869e486f8d77: Send smaller batches to origin_get.

Summary

Avoids getting a timeout.

Diff Detail

Repository

rDCIDX Metadata indexer

Branch

origin-get-batch

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 6940
Build 9750: tox-on-jenkins	Jenkins
Build 9749: arc lint + arc unit

Event Timeline

vlorentz created this revision.Jul 19 2019, 3:52 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJul 19 2019, 3:52 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/578/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tox/578/console

Harbormaster failed remote builds in B6939: Diff 5903!Jul 19 2019, 3:54 PM

consume the generator

Build is green
See https://jenkins.softwareheritage.org/job/DCIDX/job/tox/581/ for more details.

Harbormaster completed remote builds in B6940: Diff 5904.Jul 19 2019, 4:00 PM

It worked for revisions indeed.

I feel like this must be hidden as implementation detail in swh-storage though.

@douardda ping for when you come back ;)

I'll accept in the mean time.

This revision is now accepted and ready to land.Jul 19 2019, 4:05 PM

Closed by commit rDCIDX869e486f8d77: Send smaller batches to origin_get. (authored by vlorentz). · Explain WhyJul 19 2019, 4:20 PM

This revision was automatically updated to reflect the committed changes.

ardumont mentioned this in T1788: indexer-license: Investigate timeouts.Sep 24 2019, 4:25 PM

Revision Contents
Changeset List

Path

Size

swh/

indexer/

metadata.py

21 lines

Diff 5904

View Options

Send smaller batches to origin_get.ClosedPublicActions