Page MenuHomeSoftware Heritage

Build documentation on every diff
Closed, MigratedEdits Locked

Description

Currently, the documentation is only built once a day with no notification, so it's easy to let typos slip through.

It should be part of the test suite. Options (ordered from easier to most efficient):

  1. naively clone all of swh-environment + the current diff and build swh-docs with tox -e sphinx-dev. this takes 5-10 minutes, which is more than the build time of any single repo, because it needs to install all the deps and sphinx has no cache
  2. have some base state with all the dependencies installed (and possibly sphinx's cache warmed up), eg. generated by the daily cron; then just apply the current Diff, re-run its setup, and make html.
  3. only build the local documentation of the repo, and find a way to ignore references to other repositories. (This will also hide typos in internal references...)
  4. only build the local documentation of the repo, but migrate all repos to use inter-sphinx, so building the local repo can check all references are valid. But have about 20 sphinx instances interconnected like this requires migrating all references and keeping them in sync

Personally I have a preference for option 2.

Revisions and Commits

rDTPL Python module template
D5610
D5610
D5610
rDJNL Journal infrastructure
D5599
rDOBJSRPL Content replayer
D5609
rDLDG Git loader
D5602
rDSCH Scheduling utilities
D5611
rDGRPH Compressed graph representation
D5597
rDSEA Archive search
D5612
rDWAPPS Web applications
D5620
D5566
rDLDHG Mercurial loader
D5603
rDWCLI Web client
D5621
rDDEP Push deposit
D5596
D5596
rDLDBASE Generic VCS/Package Loader
D5601
rDMFCD ClearlyDefined metadata fetcher
D5604
rDSTO Storage manager
D5613
rCJSWH Jenkins jobs
D5619
D5617
rDLS Listers
D5600
rDMOD Data model
D5606
rDOBJS Object storage
D5607
rDDOC Development documentation
D5571
D5567
D5565
rDCORE Foundations and core functionalities
D5550
rDAUTH Common authentication libraries
D5593
D5593
rDICP Icinga plugins
D5598
rDCNT Archive counters
D5595
rDFUSE FUSE virtual file system
D5608
rDLDSVN Subversion (SVN) loader
D5605

Event Timeline

vlorentz triaged this task as Normal priority.Apr 16 2021, 5:33 PM
vlorentz created this task.

Solution 3 has been implemented through new tox environments in each swh package.

A new step has been added to our Jenkins pipeline to build and check the sphinx documentation on each diff and commit.

Maybe as an ultimate step to check for reference errors, we could trigger the build of swh-docs in Jenkins each time a commit is pushed ?

We should also build the whole documentation when submitting a diff to swh-docs, some sphinx errors not related to packages documentation could be caught this way (see https://jenkins.softwareheritage.org/job/DDOC/job/dev/5700/artifact/swh-docs/docs/errors.log)