Page MenuHomeSoftware Heritage
Differential D6969

Update mirror docker docs following a walkthrough
ClosedPublic
Actions

Authored by olasd on Jan 18 2022, 6:07 PM.

Details

Reviewers
ardumont
vlorentz
Group Reviewers
Reviewers
Maniphest Tasks
T3829: Document mirror - how to create and deploy a mirror from scratch
Commits
rDDOCb2a68a9d2605: Add hints on how to deploy the storage db externally
rDDOC2b8d86d713fb: Update mirror docker docs following a walkthrough
Summary

Needs some more discussion

Depends on D6965

Related to T3829

Test Plan

We actually used the docs to set up a mirror stack

Diff Detail

Repository
rDDOC Development documentation
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

olasd created this revision.Jan 18 2022, 6:07 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptJan 18 2022, 6:07 PM
swh-public-ci added a comment.Jan 18 2022, 6:12 PM

Build has FAILED

Patch application report for D6969 (id=25273)

Could not rebase; Attempt merge onto 578f414aca...

Updating 578f414..5251903
Fast-forward
 sysadm/mirror-operations/docker.rst | 145 +++++++++++++++++++-----------------
 1 file changed, 76 insertions(+), 69 deletions(-)
Changes applied before test
commit 5251903961567b84192a7e957a0f05ff939567be
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Jan 18 16:02:08 2022 +0100

    Update mirror docker docs following a walkthrough

commit 2e1ddedcb374a52183eba70d969cb32c08827eea
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Jan 18 15:47:34 2022 +0100

    mirror-operations: Convert markup to restructuredtext

Link to build: https://jenkins.softwareheritage.org/job/DDOC/job/build-on-diff/64/
See console output for more information: https://jenkins.softwareheritage.org/job/DDOC/job/build-on-diff/64/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 18 2022, 6:12 PM
Harbormaster failed remote builds in B26132: Diff 25273!
douardda added a subscriber: douardda.Jan 19 2022, 11:09 AM
douardda added inline comments.
sysadm/mirror-operations/docker.rst
28

should be a link

201

This part needs to be updated.

A better approach to update a config swh_xxx if to do:

swh config create swh_xxx-2 updated.cfg
swh service update --config-rm swh_xxx --config-add source=swh_xxx,target=/path/to/dest swh_service

with all the bells and whistles swh service update provides.
I'll send a diff for this

olasd updated this revision to Diff 25296.Jan 19 2022, 12:27 PM
  • Update placement options to use explicit node labels instead of putting all storage on the coordinator
  • Apply @douardda's comment on linking the upstream docker swarm docs
  • Move the SWH_IMAGE_TAG environment variable information forward to remove one backtracking step
  • Update for the default base_service.yml spawning a mirror database instead of a default database, which removes another backtracking step
  • Rename all the stacks to swh so they can be built incrementally
  • Update the command outputs for recent image versions
swh-public-ci added a comment.Jan 19 2022, 12:36 PM

Build is green

Patch application report for D6969 (id=25296)

Rebasing onto f75b1db1d6...

Current branch diff-target is up to date.
Changes applied before test
commit 5b38b1de21db35fd539501930270c4f846556d0f
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Jan 18 16:02:08 2022 +0100

    Update mirror docker docs following a walkthrough

See https://jenkins.softwareheritage.org/job/DDOC/job/build-on-diff/78/ for more details.

Harbormaster completed remote builds in B26154: Diff 25296.Jan 19 2022, 12:36 PM
olasd requested review of this revision.Jan 19 2022, 12:36 PM
ardumont accepted this revision as: ardumont.Jan 19 2022, 12:44 PM
ardumont added 1 blocking reviewer(s): Reviewers.
vlorentz added a subscriber: vlorentz.Jan 19 2022, 2:08 PM
vlorentz added inline comments.
sysadm/mirror-operations/docker.rst
108

Why -a post? Doesn't it make passwords needlessly weaker?

ardumont edited the summary of this revision. (Show Details)Jan 19 2022, 2:30 PM
ardumont added a task: T3829: Document mirror - how to create and deploy a mirror from scratch.
olasd updated this revision to Diff 25307.Jan 19 2022, 2:44 PM

*shrug*

olasd marked an inline comment as done.Jan 19 2022, 2:44 PM
olasd marked an inline comment as done.
swh-public-ci added a comment.Jan 19 2022, 2:54 PM

Build is green

Patch application report for D6969 (id=25307)

Rebasing onto 09ef5e2c6a...

First, rewinding head to replay your work on top of it...
Applying: Update mirror docker docs following a walkthrough
Changes applied before test
commit f70a8a9d9f6481e02321474f5dc44d2e06500c43
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Jan 18 16:02:08 2022 +0100

    Update mirror docker docs following a walkthrough

See https://jenkins.softwareheritage.org/job/DDOC/job/build-on-diff/83/ for more details.

Harbormaster completed remote builds in B26163: Diff 25307.Jan 19 2022, 2:54 PM
olasd mentioned this in D6978: Update docker stack configuration to match documentation changes.Jan 19 2022, 2:56 PM
olasd updated this revision to Diff 25311.Jan 19 2022, 3:30 PM

Add hints on how to deploy the storage db externally

swh-public-ci added a comment.Jan 19 2022, 3:40 PM

Build is green

Patch application report for D6969 (id=25311)

Rebasing onto 09ef5e2c6a...

First, rewinding head to replay your work on top of it...
Applying: Update mirror docker docs following a walkthrough
Applying: Add hints on how to deploy the storage db externally
Changes applied before test
commit bf008d3561a4dd644345fdbd678fb00e2412869c
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Wed Jan 19 15:29:29 2022 +0100

    Add hints on how to deploy the storage db externally

commit e1565943c9b41edae0e61160d6111f33e19c00b4
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Jan 18 16:02:08 2022 +0100

    Update mirror docker docs following a walkthrough

See https://jenkins.softwareheritage.org/job/DDOC/job/build-on-diff/85/ for more details.

Harbormaster completed remote builds in B26166: Diff 25311.Jan 19 2022, 3:40 PM
vlorentz accepted this revision.Jan 19 2022, 3:41 PM
This revision is now accepted and ready to land.Jan 19 2022, 3:41 PM
olasd updated this revision to Diff 25316.Jan 19 2022, 3:47 PM

Gotta love rebasing

swh-public-ci added a comment.Jan 19 2022, 3:57 PM

Build is green

Patch application report for D6969 (id=25316)

Rebasing onto 09ef5e2c6a...

Current branch diff-target is up to date.
Changes applied before test
commit b2a68a9d2605fbd1653f39c820c2056d14abae7f
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Wed Jan 19 15:29:29 2022 +0100

    Add hints on how to deploy the storage db externally

commit 2b8d86d713fbd0a0322c63797e0dbfb90f2ba23b
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Jan 18 16:02:08 2022 +0100

    Update mirror docker docs following a walkthrough

See https://jenkins.softwareheritage.org/job/DDOC/job/build-on-diff/87/ for more details.

Harbormaster completed remote builds in B26168: Diff 25316.Jan 19 2022, 3:57 PM
Closed by commit rDDOC2b8d86d713fb: Update mirror docker docs following a walkthrough (authored by olasd). · Explain WhyJan 19 2022, 4:04 PM
This revision was automatically updated to reflect the committed changes.
olasd added a commit: rDDOC2b8d86d713fb: Update mirror docker docs following a walkthrough.
olasd added a commit: rDDOCb2a68a9d2605: Add hints on how to deploy the storage db externally.

Revision Contents
Changeset List

PathSize
sysadm/
mirror-operations/
262 lines

Diff 25317

View Options

sysadm/mirror-operations/docker.rst

Loading...
About · Developer information · Legal