Page MenuHomeSoftware Heritage

Fix documentation dev build (again)
Closed, MigratedEdits Locked

Description

This prevents having diff (which builds green).

Event Timeline

Tentatively trying to reproduce the ci issue:

$ cd $SWH_CI_ENVIRONMENT_HOME/swh-jenkins-dockerfile
# build the image
$ make swh-jenkins/sphinx
$ cd $SWH_ENVIRONMENT_HOME/swh-docs
$ docker run -ti -v $PWD:/src/swh-docs/ --entrypoint bash swh-jenkins/sphinx
-> $ git clone https://forge.softwareheritage.org/source/swh-environment.git
-> $ cd swh-environment
# copy out the local diffs (or git checkout from staging as you wish)
-> $ crudini --del .mrconfig snippets
-> $ crudini --del .mrconfig swh-py-template
-> $ mr -j 8 -t update
-> $ cd swh-docs
# install the locally mount the swh-docs "changes" to debug
-> $ rsync -avz --exclude .tox /src/swh-docs/ ./
-> $ git clean -dfx
-> $ SPHINXOPTS='-W -q --keep-going -w errors.log' SPHINXOPTCOLOR='--no-color' tox -e sphinx-dev
...

To no avail ^
Not reproduced exactly the same issue.
And it's taking forever like the ci build...

Legend:

  • '->': In docker container

Step to tentatively reproduce the issue:

  • create the docker image jenkins run locally
  • connect to it
  • setup the container to run as jenkins does

as follows:

-> $ SPHINXOPTS='-W -q --keep-going -w errors.log' SPHINXOPTCOLOR='--no-color' tox -e sphinx-dev
...
touch rec-build-stamp
sphinx-build -b html "." "_build/html" -W -q --keep-going -w errors.log --no-color
/home/jenkins/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/scanner/dashboard/dashboard.py:11: UserWarning:
The dash_core_components package is deprecated. Please replace
`import dash_core_components as dcc` with `from dash import dcc`
  import dash_core_components as dcc
/home/jenkins/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/scanner/dashboard/dashboard.py:12: UserWarning:
The dash_html_components package is deprecated. Please replace
`import dash_html_components as html` with `from dash import html`
  import dash_html_components as html
WARNING: autodoc: failed to import module 'checks' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'collection' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'common' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'content' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'edit' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'edit_media' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'deposit_check' from module 'swh.deposit.api.private'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'deposit_read' from module 'swh.deposit.api.private'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'deposit_update_status' from module 'swh.deposit.api.private'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'urls' from module 'swh.deposit.api.private'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'service_document' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'state' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'sword_edit' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'urls' from module 'swh.deposit.api'; the following exception was raised:
No module named 'xmlschema'
WARNING: autodoc: failed to import module 'urls' from module 'swh.deposit'; the following exception was raised:
No module named 'xmlschema'
make: *** [Makefile:72: sphinx/html] Error 1
make: Leaving directory '/home/jenkins/swh-environment/swh-docs/docs'
WARNING [pifpaf.util] `psutil.Popen(pid=3706, status='terminated', exitcode=0, started='15:30:26')` is already gone, sending SIGKILL to its process group
WARNING [pifpaf.util] `psutil.Popen(pid=3689, status='terminated', exitcode=0, started='15:30:25')` is already gone, sending SIGKILL to its process group
WARNING [pifpaf.util] `psutil.Popen(pid=3684, status='terminated', exitcode=0, started='15:30:25')` is already gone, sending SIGKILL to its process group
WARNING [pifpaf.util] `psutil.Popen(pid=3717, status='terminated', exitcode=2, started='15:30:26')` is already gone, sending SIGKILL to its process group
ERROR: InvocationError for command '/home/jenkins/swh-environment/swh-docs/.tox/sphinx-dev/bin/python -m pifpaf run postgresql -- make -C docs html' (exited with code 2)
___________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
ERROR:   sphinx-dev: commands failed

# the last 2 commits correspond to the diffs i opened to "fix" that build
-> $ git log
commit 39d9a83813743ca265d65cd70105bc0bbb59d8ea (HEAD -> master)
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Mar 30 12:49:11 2022 +0200

    add-new-package: Update with necessary documentation instructions

    Without this, the actual documentation build will fail once a new package is
    incompletely installed.

commit f6cbf42ad1d961b35fa13d33ee07ef3c346dcf1b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Mar 29 09:33:46 2022 +0200

    Reference the new swh.scrubber module
->

The error message in there ^ is not actually what i expected.
I wanted the same error message as the build failures [1] [2].

Whose gist of it is "solely" a problem in rendering the cli docstring [3]
(swh.model.cli:identify).

13:48:09  /var/lib/jenkins/workspace/DDOC/build-on-diff@2/swh-docs/docs/swh-model/cli.rst:14: WARNING: Literal block expected; none found

As @vlorentz suggested, it might be an upstream bug in sphinx-click (used for rendering
sphinx out of a click docstring). As I failed to reproduce for now, i'll put this on the
side for now.

FWIW, I've looked into their issue tracker and found no related issues [4]

[1] https://jenkins.softwareheritage.org/job/DDOC/job/build-on-diff/141/console

[2] https://jenkins.softwareheritage.org/job/DDOC/job/build-on-diff/140/console

[3] https://forge.softwareheritage.org/source/swh-model/browse/master/swh/model/cli.py$224-250

[4] https://github.com/click-contrib/sphinx-click/issues

It seems to have fixed it!
Great. Thanks.

Now it's the (new and) same error as in docker ^, the new xmlschema introduced in swh.deposit[server] deps.

D7480 should fix it?

nope...
well, yeah... the warnings about xmlschema disappeared...
but the initial problem supposedly fixed by [1] came back...

[1] rDMODba7af7447e5aa7fcf05b327fe1a35419446849a9

That last diff fixed it in the end...
My docker tryout agreed [1] (contrary to jenkins diff for some reason).
So i pushed and jenkins dev build got green again [2] (finally).

I can finally close that one.

[1]

___________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
  sphinx-dev: commands succeeded
  congratulations :)

[2] https://jenkins.softwareheritage.org/job/DDOC/job/dev/1136/console

ardumont claimed this task.