Page MenuHomeSoftware Heritage

Continuous IntegrationFolder
ActivePublic

Details

Description

Continuous Integration toolchain, powering https://jenkins.softwareheritage.org/

Recent Activity

Wed, Jul 7

anlambert added a comment to T3423: Execute cypress tests in parallel on Jenkins.

After some hacks on the current Jenkins pipeline for swh-web to test if it was feasible, I managed to setup the use of sorry-cypress but could not run multiple cypress in parallel as Jenkins pipeline forbid nested parallel stages.
So this would require setting up a dedicated pipeline build job for cypress tests as currently Python and cypress tests are executed in parallel in a single job.

Wed, Jul 7, 2:40 PM · Continuous Integration, Web app

Tue, Jul 6

anlambert updated the task description for T3423: Execute cypress tests in parallel on Jenkins.
Tue, Jul 6, 2:09 PM · Continuous Integration, Web app
anlambert triaged T3423: Execute cypress tests in parallel on Jenkins as Normal priority.
Tue, Jul 6, 2:03 PM · Continuous Integration, Web app

May 19 2021

anlambert closed T3326: docker tests on Jenkins: error while removing network as Resolved.

Issue is now solved, closing this.

May 19 2021, 11:19 AM · Continuous Integration, Docker environment
ardumont added a comment to T3326: docker tests on Jenkins: error while removing network.

awesome, thanks :)

May 19 2021, 9:22 AM · Continuous Integration, Docker environment

May 18 2021

vsellier added a comment to T3326: docker tests on Jenkins: error while removing network.

and the build is green ;)

May 18 2021, 7:07 PM · Continuous Integration, Docker environment
anlambert added a comment to T3326: docker tests on Jenkins: error while removing network.
16:50:52 ============================= test session starts ==============================
16:50:52 platform linux -- Python 3.7.3, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
16:50:52 cachedir: .tox/py3/.pytest_cache
16:50:52 rootdir: /var/lib/jenkins/workspace/swh-docker-dev/docker
16:50:52 plugins: testinfra-6.3.0, testinfra-6.0.0
16:50:52 collected 7 items
16:50:52 
16:50:52 tests/test_deposit.py .....                                              [ 71%]
16:53:13 tests/test_git_loader.py .                                               [ 85%]
May 18 2021, 6:57 PM · Continuous Integration, Docker environment
vsellier added a comment to T3326: docker tests on Jenkins: error while removing network.

thanks for having investigated that

May 18 2021, 6:49 PM · Continuous Integration, Docker environment
anlambert added a comment to T3326: docker tests on Jenkins: error while removing network.

So the docker_default network did not get removed since April 20, 2021, see docker network inspect docker_default output below:

15:07:16 [
15:07:16     {
15:07:16         "Name": "docker_default",
15:07:16         "Id": "a0145eb2f5b3f055bfafce7a88289346c4668f3a19f2544122267272cd96f7b3",
15:07:16         "Created": "2021-04-20T02:57:51.141593622Z",
15:07:16         "Scope": "local",
15:07:16         "Driver": "bridge",
15:07:16         "EnableIPv6": false,
15:07:16         "IPAM": {
15:07:16             "Driver": "default",
15:07:16             "Options": null,
15:07:16             "Config": [
15:07:16                 {
15:07:16                     "Subnet": "172.28.0.0/16",
15:07:16                     "Gateway": "172.28.0.1"
15:07:16                 }
15:07:16             ]
15:07:16         },
15:07:16         "Internal": false,
15:07:16         "Attachable": true,
15:07:16         "Ingress": false,
15:07:16         "ConfigFrom": {
15:07:16             "Network": ""
15:07:16         },
15:07:16         "ConfigOnly": false,
15:07:16         "Containers": {},
15:07:16         "Options": {},
15:07:16         "Labels": {
15:07:16             "com.docker.compose.network": "default",
15:07:16             "com.docker.compose.project": "docker",
15:07:16             "com.docker.compose.version": "1.29.1"
15:07:16         }
15:07:16     }
15:07:16 ]
May 18 2021, 5:18 PM · Continuous Integration, Docker environment

May 12 2021

anlambert triaged T3326: docker tests on Jenkins: error while removing network as Normal priority.
May 12 2021, 1:46 PM · Continuous Integration, Docker environment
anlambert created T3326: docker tests on Jenkins: error while removing network.
May 12 2021, 1:46 PM · Continuous Integration, Docker environment

May 10 2021

anlambert added a comment to T3258: Build documentation on every diff.

We should also build the whole documentation when submitting a diff to swh-docs, some sphinx errors not related to packages documentation could be caught this way (see https://jenkins.softwareheritage.org/job/DDOC/job/dev/5700/artifact/swh-docs/docs/errors.log)

May 10 2021, 12:08 PM · Continuous Integration, Development environment, Documentation

May 5 2021

vsellier closed T3307: apidoc generation is not performed on swh modules during dev build as Resolved.
May 5 2021, 2:52 PM · Continuous Integration, Documentation
vsellier added a comment to T3307: apidoc generation is not performed on swh modules during dev build.

@anlambert I didn't succeed to work with link to the modules, but I found another way with a standard installation (i.e without the -e flag) we can force with a flag on the ci (cf D5681).
I really not sure if it's the good approach or not.

May 5 2021, 1:09 PM · Continuous Integration, Documentation
vsellier added a revision to T3307: apidoc generation is not performed on swh modules during dev build: D5681: Add support to not install the dev modules in editable mode.
May 5 2021, 1:06 PM · Continuous Integration, Documentation
vsellier renamed T3307: apidoc generation is not performed on swh modules during dev build from swh dependencies not installed during dev build to apidoc generation is not performed on swh modules during dev build.
May 5 2021, 11:36 AM · Continuous Integration, Documentation
vsellier added a comment to T3307: apidoc generation is not performed on swh modules during dev build.

thanks a lot @anlambert, I will look in that direction

May 5 2021, 11:27 AM · Continuous Integration, Documentation
anlambert added a comment to T3307: apidoc generation is not performed on swh modules during dev build.

@vsellier, you should be able to create the required symbolic links by adding an extra step in the apidoc-stamp make target.

May 5 2021, 11:25 AM · Continuous Integration, Documentation
vsellier added a comment to T3307: apidoc generation is not performed on swh modules during dev build.

Aparrently, the modules are well installed but the sphinx-apidoc is not detecting them due to the local installation

jenkins@4e5220b923d8:~/workspace/swh-environment/swh-docs$ .tox/sphinx-dev/bin/python3 -m pip list | grep swh
swh.auth                      0.5.4                           /home/jenkins/workspace/swh-environment/swh-auth
swh.core                      0.13.2.dev1+g7d42035            /home/jenkins/workspace/swh-environment/swh-core
swh.counters                  0.7.1.dev1+g6a44a84             /home/jenkins/workspace/swh-environment/swh-counters
swh.deposit                   0.13.6                          /home/jenkins/workspace/swh-environment/swh-deposit
swh.docs                      0.0.1.dev334+g044cb9b.d20210505
swh.fuse                      1.0.3                           /home/jenkins/workspace/swh-environment/swh-fuse
swh.graph                     0.3.2.dev3+g62c2fd3             /home/jenkins/workspace/swh-environment/swh-graph
swh.icinga-plugins            0.3.1.dev1+g8878925             /home/jenkins/workspace/swh-environment/swh-icinga-plugins
swh.indexer                   0.7.1.dev4+g8f1fb0f             /home/jenkins/workspace/swh-environment/swh-indexer
swh.journal                   0.7.2.dev8+g2972c7a             /home/jenkins/workspace/swh-environment/swh-journal
swh.lister                    1.1.0                           /home/jenkins/workspace/swh-environment/swh-lister
swh.loader.core               0.22.1.dev2+g0e4bb4b            /home/jenkins/workspace/swh-environment/swh-loader-core
swh.loader.git                0.9.2.dev1+g15e12fa             /home/jenkins/workspace/swh-environment/swh-loader-git
swh.loader.mercurial          0.5.1.dev4+g8884714             /home/jenkins/workspace/swh-environment/swh-loader-mercurial
swh.loader.svn                0.7.1                           /home/jenkins/workspace/swh-environment/swh-loader-svn
swh.model                     2.4.2.dev1+gdf036ef             /home/jenkins/workspace/swh-environment/swh-model
swh.objstorage                0.2.3                           /home/jenkins/workspace/swh-environment/swh-objstorage
swh.objstorage.replayer       0.2.2                           /home/jenkins/workspace/swh-environment/swh-objstorage-replayer
swh.scanner                   0.4.2.dev1+g30b40cc             /home/jenkins/workspace/swh-environment/swh-scanner
swh.scheduler                 0.13.1.dev5+gbab557e            /home/jenkins/workspace/swh-environment/swh-scheduler
swh.search                    0.8.1                           /home/jenkins/workspace/swh-environment/swh-search
swh.storage                   0.27.5.dev4+g051b7715           /home/jenkins/workspace/swh-environment/swh-storage
swh.vault                     0.5.2.dev2+gf87dd54             /home/jenkins/workspace/swh-environment/swh-vault
swh.web                       0.0.307.dev3+gf8c750b6          /home/jenkins/workspace/swh-environment/swh-web
swh.web.client                0.3.1.dev1+g4b610ad             /home/jenkins/workspace/swh-environment/swh-web-client
sphinx-apidoc \
	--implicit-namespaces \
	--templatedir=../swh/docs/templates/ \
	--maxdepth=3 \
	--ext-viewcode --separate \
	-o apidoc \
	/home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh \
	/home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/tests /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/tests/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/tests/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/*/tests/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/migrations /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/migrations/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/migrations/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/*/migrations/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/wsgi.py /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/wsgi.py /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/*/wsgi.py /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/*/*/*/wsgi.py /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/deposit/settings/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/web/settings/* /home/jenkins/workspace/swh-environment/swh-docs/.tox/sphinx-dev/lib/python3.7/site-packages/swh/dataset/*
Creating file apidoc/swh.rst.
Creating file apidoc/swh.docs.rst.
Creating file apidoc/swh.docs.django_settings.rst.
Creating file apidoc/swh.docs.sphinx.rst.
Creating file apidoc/swh.docs.sphinx.conf.rst.
Creating file apidoc/swh.docs.sphinx.view_in_phabricator.rst.
Creating file apidoc/modules.rst.
May 5 2021, 11:14 AM · Continuous Integration, Documentation
anlambert added a comment to T3307: apidoc generation is not performed on swh modules during dev build.

This is not a dependencies installation issue. When the swh packages are installed in dev mode (pip -e) and the global doc is built, the apidoc files are generated in swh-docs/docs/apidoc, but as those files are linked in the index.rst file of each package, sphinx expects them to be in swh-docs/docs/<swh_package>/apidoc so the warnings.

May 5 2021, 11:02 AM · Continuous Integration, Documentation
vsellier changed the status of T3307: apidoc generation is not performed on swh modules during dev build from Open to Work in Progress.
May 5 2021, 10:25 AM · Continuous Integration, Documentation
vsellier closed D5674: sphinx: Fix the creation of the pre-commit file during the mr update.
May 5 2021, 9:37 AM · Continuous Integration, Documentation
ardumont accepted D5674: sphinx: Fix the creation of the pre-commit file during the mr update.

Less warnings to distract us when something goes wrong in the build.

May 5 2021, 9:10 AM · Continuous Integration, Documentation
vsellier added a reviewer for D5674: sphinx: Fix the creation of the pre-commit file during the mr update: Reviewers.
May 5 2021, 8:49 AM · Continuous Integration, Documentation

Apr 28 2021

anlambert added a comment to T3258: Build documentation on every diff.

Solution 3 has been implemented through new tox environments in each swh package.

Apr 28 2021, 5:29 PM · Continuous Integration, Development environment, Documentation

Apr 27 2021

moranegg moved T3258: Build documentation on every diff from Backlog to building docs on the Documentation board.
Apr 27 2021, 3:49 PM · Continuous Integration, Development environment, Documentation
moranegg moved T3148: Make Jenkins publish the built documentation on swh-docs diffs from sponsors/clients to building docs on the Documentation board.
Apr 27 2021, 3:46 PM · System administration, Continuous Integration, Documentation
moranegg moved T3148: Make Jenkins publish the built documentation on swh-docs diffs from Backlog to sponsors/clients on the Documentation board.
Apr 27 2021, 3:42 PM · System administration, Continuous Integration, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5621: tox: Add sphinx environments to check sane doc build.
Apr 27 2021, 2:54 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5620: tox: Add sphinx environments to check sane doc build.
Apr 27 2021, 2:48 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5619: templates/swh-pipeline: Prevent BrokenPipe when checking tox sphinx env.
Apr 27 2021, 12:08 PM · Continuous Integration, Development environment, Documentation

Apr 26 2021

anlambert added a revision to T3258: Build documentation on every diff: D5617: templates/swh-pipeline: Add stage to build sphinx documentation.
Apr 26 2021, 6:43 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5613: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 6:09 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5612: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 6:06 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5611: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 6:02 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5610: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 5:50 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5609: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 5:26 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5608: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 5:21 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5607: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 5:19 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5606: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 5:15 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5605: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 5:03 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5604: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 4:50 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5603: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 4:46 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5602: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 4:41 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5601: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 4:38 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5600: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 4:35 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5599: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 4:32 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5598: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 4:29 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5597: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 4:22 PM · Continuous Integration, Development environment, Documentation
anlambert added a revision to T3258: Build documentation on every diff: D5596: tox: Add sphinx environments to check sane doc build.
Apr 26 2021, 4:20 PM · Continuous Integration, Development environment, Documentation