apidoc generation is not performed on swh modules during dev build
Closed, MigratedEdits Locked
Actions

Assigned To

Authored By

	vsellier
	May 5 2021, 10:25 AM

Description

The swh dependencies are not installed during the dev build[1] which leads to a build failure.

/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-auth/index.rst:8: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.auth'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-core/index.rst:8: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.core'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-deposit/index.rst:17: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.deposit'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-fuse/index.rst:6: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.fuse'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-graph/index.rst:6: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.graph'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-indexer/index.rst:22: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.indexer'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-journal/index.rst:12: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.journal'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-lister/index.rst:27: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.lister'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-loader-core/index.rst:16: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.loader.core'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-loader-core/index.rst:16: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.loader.package'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-loader-git/index.rst:12: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.loader.git'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-loader-mercurial/index.rst:12: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.loader.mercurial'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-loader-svn/index.rst:12: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.loader.svn'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-model/index.rst:8: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.model'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-objstorage-replayer/index.rst:17: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.objstorage.replayer'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-objstorage/index.rst:12: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.objstorage'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-scanner/index.rst:12: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.scanner'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-scheduler/index.rst:165: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.scheduler'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-search/index.rst:33: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.search'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-storage/index.rst:90: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.storage'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-vault/index.rst:22: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.vault'
/home/jenkins/workspace/DDOC/dev/swh-docs/docs/swh-web-client/index.rst:90: WARNING: toctree contains reference to nonexisting document 'apidoc/swh.web.client'
make: *** [Makefile:70: sphinx/html] Error 1

It seems it's related to the work done during D5525

[1] https://jenkins.softwareheritage.org/job/DDOC/job/dev

Revisions and Commits

rDDOC Development documentation
	D5681	rDDOC5e6f5d854803 Don't install the modules in editable mode

Related Objects

Mentioned Here: D5681: Add support to not install the dev modules in editable mode
D5525: tox/sphinx-dev: Workaround really long dependencies installation

Event Timeline

vsellier changed the task status from Open to Work in Progress.May 5 2021, 10:25 AM

vsellier triaged this task as Normal priority.

vsellier created this task.

This is not a dependencies installation issue. When the swh packages are installed in dev mode (pip -e) and the global doc is built, the apidoc files are generated in swh-docs/docs/apidoc, but as those files are linked in the index.rst file of each package, sphinx expects them to be in swh-docs/docs/<swh_package>/apidoc so the warnings.

I think the simplest fix is to add a symbolic link to swh-docs/docs/apidoc in each swh-docs/docs/<swh_package>/ directory when building the global doc and remove it afterwards.

Aparrently, the modules are well installed but the sphinx-apidoc is not detecting them due to the local installation

jenkins@4e5220b923d8:~/workspace/swh-environment/swh-docs$ .tox/sphinx-dev/bin/python3 -m pip list | grep swh
swh.auth                      0.5.4                           /home/jenkins/workspace/swh-environment/swh-auth
swh.core                      0.13.2.dev1+g7d42035            /home/jenkins/workspace/swh-environment/swh-core
swh.counters                  0.7.1.dev1+g6a44a84             /home/jenkins/workspace/swh-environment/swh-counters
swh.deposit                   0.13.6                          /home/jenkins/workspace/swh-environment/swh-deposit
swh.docs                      0.0.1.dev334+g044cb9b.d20210505
swh.fuse                      1.0.3                           /home/jenkins/workspace/swh-environment/swh-fuse
swh.graph                     0.3.2.dev3+g62c2fd3             /home/jenkins/workspace/swh-environment/swh-graph
swh.icinga-plugins            0.3.1.dev1+g8878925             /home/jenkins/workspace/swh-environment/swh-icinga-plugins
swh.indexer                   0.7.1.dev4+g8f1fb0f             /home/jenkins/workspace/swh-environment/swh-indexer
swh.journal                   0.7.2.dev8+g2972c7a             /home/jenkins/workspace/swh-environment/swh-journal
swh.lister                    1.1.0                           /home/jenkins/workspace/swh-environment/swh-lister
swh.loader.core               0.22.1.dev2+g0e4bb4b            /home/jenkins/workspace/swh-environment/swh-loader-core
swh.loader.git                0.9.2.dev1+g15e12fa             /home/jenkins/workspace/swh-environment/swh-loader-git
swh.loader.mercurial          0.5.1.dev4+g8884714             /home/jenkins/workspace/swh-environment/swh-loader-mercurial
swh.loader.svn                0.7.1                           /home/jenkins/workspace/swh-environment/swh-loader-svn
swh.model                     2.4.2.dev1+gdf036ef             /home/jenkins/workspace/swh-environment/swh-model
swh.objstorage                0.2.3                           /home/jenkins/workspace/swh-environment/swh-objstorage
swh.objstorage.replayer       0.2.2                           /home/jenkins/workspace/swh-environment/swh-objstorage-replayer
swh.scanner                   0.4.2.dev1+g30b40cc             /home/jenkins/workspace/swh-environment/swh-scanner
swh.scheduler                 0.13.1.dev5+gbab557e            /home/jenkins/workspace/swh-environment/swh-scheduler
swh.search                    0.8.1                           /home/jenkins/workspace/swh-environment/swh-search
swh.storage                   0.27.5.dev4+g051b7715           /home/jenkins/workspace/swh-environment/swh-storage
swh.vault                     0.5.2.dev2+gf87dd54             /home/jenkins/workspace/swh-environment/swh-vault
swh.web                       0.0.307.dev3+gf8c750b6          /home/jenkins/workspace/swh-environment/swh-web
swh.web.client                0.3.1.dev1+g4b610ad             /home/jenkins/workspace/swh-environment/swh-web-client

sphinx-apidoc \
	--implicit-namespaces \
	--templatedir=../swh/docs/templates/ \
	--maxdepth=3 \
	--ext-viewcode --separate \
	-o apidoc \
	/home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh \
	/home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/tests /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/tests/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/tests/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/*/tests/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/migrations /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/migrations/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/migrations/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/*/migrations/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/wsgi.py /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/wsgi.py /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/*/wsgi.py /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/*/wsgi.py /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/deposit/settings/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/settings/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/dataset/*
Creating file apidoc/swh.rst.
Creating file apidoc/swh.docs.rst.
Creating file apidoc/swh.docs.django_settings.rst.
Creating file apidoc/swh.docs.sphinx.rst.
Creating file apidoc/swh.docs.sphinx.conf.rst.
Creating file apidoc/swh.docs.sphinx.view_in_phabricator.rst.
Creating file apidoc/modules.rst.

@vsellier, you should be able to create the required symbolic links by adding an extra step in the apidoc-stamp make target.

You could add new functions in the ln-sphinx-subprojects file to do so.

thanks a lot @anlambert, I will look in that direction

vsellier renamed this task from swh dependencies not installed during dev build to apidoc generation is not performed on swh modules during dev build.May 5 2021, 11:36 AM

vsellier added a revision: D5681: Add support to not install the dev modules in editable mode.May 5 2021, 1:06 PM

@anlambert I didn't succeed to work with link to the modules, but I found another way with a standard installation (i.e without the -e flag) we can force with a flag on the ci (cf D5681).
I really not sure if it's the good approach or not.

vsellier added a commit: rDDOC5e6f5d854803: Don't install the modules in editable mode.May 5 2021, 2:32 PM

vsellier closed this task as Resolved.May 5 2021, 2:52 PM

This task has been migrated to GitLab.

apidoc generation is not performed on swh modules during dev buildClosed, MigratedEdits LockedActions

Description

Revisions and Commits

Related Objects

Event Timeline

apidoc generation is not performed on swh modules during dev build
Closed, MigratedEdits Locked
Actions