Differential D7914

Wrap query in transaction
ClosedPublic
Actions

Authored by ardumont on May 31 2022, 10:36 AM.

Details

Reviewers

douardda

Group Reviewers

Reviewers

Maniphest Tasks

T4228: scrubber: Investigate the apparent lock (staging)

Commits

rDSCRUB31bcec6000a8: Wrap queries in transaction

Summary

This alleviates current locking issues.

Related to T4228

Test Plan

T4228#85847

Corrupt objects are popping up now (and growing fast):

Tue 31 May 2022 11:01:14 AM CEST (every 2s)

+-------------------------------+-------+
|              now              | count |
+-------------------------------+-------+
| 2022-05-31 09:01:14.233113+00 |  1620 |
+-------------------------------+-------+
(1 row)
+-------------------------------+-------+
|              now              | count |
+-------------------------------+-------+
| 2022-05-31 09:01:54.394741+00 |  3233 |
+-------------------------------+-------+
(1 row)

Time: 6.463 ms

Diff Detail

Repository

rDSCRUB Datastore Scrubber

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 29617
Build 46285: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 46284: arc lint + arc unit

Event Timeline

ardumont created this revision.May 31 2022, 10:36 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptMay 31 2022, 10:36 AM

Build is green

Patch application report for D7914 (id=28526)

Rebasing onto f24850469c...

Current branch diff-target is up to date.

Changes applied before test

commit 377f5e084ab71ec6fb4f9c69b89ff2364d93c3eb
Author: Antoine R. Dumont (@ardumont) <antoine.romain.dumont@gmail.com>
Date:   Tue May 31 10:35:45 2022 +0200

    Wrap query in transaction
    
    This might help alleviate current locking issues.
    
    Related to T4228

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/46/ for more details.

Harbormaster completed remote builds in B29617: Diff 28526.May 31 2022, 10:39 AM

ardumont requested review of this revision.May 31 2022, 10:39 AM

ardumont mentioned this in T4228: scrubber: Investigate the apparent lock (staging).May 31 2022, 10:58 AM

ardumont retitled this revision from wip: Wrap query in transaction to Wrap query in transaction.

ardumont edited the summary of this revision. (Show Details)

ardumont edited the test plan for this revision. (Show Details)

ardumont edited the test plan for this revision. (Show Details)May 31 2022, 11:01 AM

ardumont edited the test plan for this revision. (Show Details)

makes sense to me

This revision is now accepted and ready to land.May 31 2022, 11:06 AM

Reword commit message

Build is green

Patch application report for D7914 (id=28527)

Rebasing onto f24850469c...

Current branch diff-target is up to date.

Changes applied before test

commit 31bcec6000a8e7130f67a1d0f3b3d9c5f4fd6aba
Author: Antoine R. Dumont (@ardumont) <antoine.romain.dumont@gmail.com>
Date:   Tue May 31 10:35:45 2022 +0200

    Wrap queries in transaction
    
    This alleviates current locking issues where nothing gets written in production.
    
    Related to T4228

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/47/ for more details.

Harbormaster completed remote builds in B29618: Diff 28527.May 31 2022, 11:13 AM

Closed by commit rDSCRUB31bcec6000a8: Wrap queries in transaction (authored by ardumont). · Explain WhyMay 31 2022, 11:15 AM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDSCRUB31bcec6000a8: Wrap queries in transaction.

ardumont mentioned this in T4305: Aligh swh backends for migration tools to work.Jun 3 2022, 11:33 AM