Page MenuHomeSoftware Heritage

Package fossology-nomossa for buster-swh suite
Closed, MigratedEdits Locked

Description

New staging worker got created from scratch to try and run the same indexers as the production does.
It's failing to install for the fossology indexer since that dependency is only available for jessie/stretch.

This needs to be rebuilt for buster (and uploaded in our debian repository).

root@pergamon:/srv/softwareheritage/repository# reprepro ls fossology-nomossa
fossology-nomossa | 3.1-1~bpo8~swh+1 |      jessie | amd64
fossology-nomossa | 3.1-1~bpo9~swh+1 | stretch-swh | amd64

root@pergamon:/srv/softwareheritage/repository# debdiff pool/main/f/fossology/fossology-nomossa_3.1-1~bpo*
[The following lists of changes regard files as different if they have
different names, permissions or owners.]

Files in second .deb but not in first
-------------------------------------
-rw-r--r--  root/root   /usr/share/doc/fossology-nomossa/changelog.gz

Control files: lines which differ (wdiff format)
------------------------------------------------
Installed-Size: [-548-] {+644+}
Version: [-3.1-1~bpo8~swh+1-] {+3.1-1~bpo9~swh+1+}

Event Timeline

Note:

10:40 <+olasd> ardumont: btw, I've been looking for the fossology-nomossa package for stretch, and I don't have it on this laptop; I suspect I just pulled the previous source and just bumped the version and rebuilt the package with no changes
10:40 <+olasd> ardumont: you could debdiff the jessie and stretch versions to see what happened...
10:42 <+ardumont> i'm fuzzy on the command to actually rebuild it (i guess our gbp/sbuild stanza will work)
10:43 <+ardumont> til on debdiff, thx for the hints

Finally unstuck this.

Found back the repository, pushed in our forge [1].

Created the debian/buster-swh branch and adapted accordingly so the build can happen
again.

Added the pristine-tar branch holds the origin tarball required for build (retrieved
from our debian repository). Otherwise, building the tarball from source triggers a
mistmatch checksums down the line (during upload).

Finally, trigger the build and upload to our repository:

$ git checkout debian/buster-swh
$ gbp buildpackage --git-builder='sbuild -As'
...
Source-Version: 3.1-1~bpo10~swh+1
Space: 48308
Status: successful
Version: 3.1-1~bpo10~swh+1
--------------------------------------------------------------------------------
Finished at 2021-02-22T13:11:56Z
Build needed 00:01:14, 48308k disk space
$ swh-debian-auto-upload ~/debian/build-area/

In our debian repository, the package for buster exists:

$ root@pergamon:/srv/softwareheritage/repository# reprepro ls "fossology"; reprepro ls "fossology-nomossa"
fossology |  3.1-1~bpo8~swh+1 |     jessie | source
fossology | 3.1-1~bpo10~swh+1 | buster-swh | source
fossology-nomossa |  3.1-1~bpo8~swh+1 |      jessie | amd64
fossology-nomossa |  3.1-1~bpo9~swh+1 | stretch-swh | amd64
fossology-nomossa | 3.1-1~bpo10~swh+1 |  buster-swh | amd64
ardumont renamed this task from Update fossology-nomossa version for buster to Package fossology-nomossa for buster-swh suite.Feb 22 2021, 2:26 PM
ardumont changed the task status from Open to Work in Progress.
ardumont moved this task from Backlog to in-progress on the System administration board.

At last, trigger back the puppet on worker3.internal.staging.swh.network:

root@worker3:~# puppet agent --test
Info: Using configured environment 'staging'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for worker3.internal.staging.swh.network
Info: Applying configuration version '1613999976'
Notice: /Stage[main]/Profile::Swh::Deploy::Worker::Indexer_fossology_license/Package[fossology-nomossa]/ensure: created
Notice: /Stage[main]/Profile::Swh::Deploy::Worker::Indexer_fossology_license/Profile::Swh::Deploy::Worker::Instance[indexer_fossology_license]/File[/etc/softwareheritage/indexer_fossology_license.yml]/ensure: defined content as '{md5}71db8b37a3e1b963a2ec9f6e4a6a00c0'
Notice: /Stage[main]/Profile::Swh::Deploy::Worker::Indexer_fossology_license/Profile::Swh::Deploy::Worker::Instance[indexer_fossology_license]/Service[swh-worker@indexer_fossology_license]/enable: enable changed 'false' to 'true'
Notice: /Stage[main]/Profile::Swh::Deploy::Worker::Indexer_fossology_license/Profile::Swh::Deploy::Worker::Instance[indexer_fossology_license]/Systemd::Dropin_file[swh-worker@indexer_fossology_license/parameters.conf]/File[/etc/systemd/system/swh-worker@indexer_fossology_license.service.d]/ensure: created
Notice: /Stage[main]/Profile::Swh::Deploy::Worker::Indexer_fossology_license/Profile::Swh::Deploy::Worker::Instance[indexer_fossology_license]/Systemd::Dropin_file[swh-worker@indexer_fossology_license/parameters.conf]/File[/etc/systemd/system/swh-worker@indexer_fossology_license.service.d/parameters.conf]/ensure: defined content as '{md5}bab06958a00a4880e94c5f20e82d8b50'
Info: /Stage[main]/Profile::Swh::Deploy::Worker::Indexer_fossology_license/Profile::Swh::Deploy::Worker::Instance[indexer_fossology_license]/Systemd::Dropin_file[swh-worker@indexer_fossology_license/parameters.conf]/File[/etc/systemd/system/swh-worker@indexer_fossology_license.service.d/parameters.conf]: Scheduling refresh of Class[Systemd::Systemctl::Daemon_reload]
Info: Class[Systemd::Systemctl::Daemon_reload]: Scheduling refresh of Exec[systemctl-daemon-reload]
Notice: /Stage[main]/Systemd::Systemctl::Daemon_reload/Exec[systemctl-daemon-reload]: Triggered 'refresh' from 1 event
Notice: Applied catalog in 8.41 seconds

And it's happening, worker3 (buster node) is now computing back fossology license.

Feb 22 13:21:45 worker3 python3[30440]: [2021-02-22 13:21:45,571: INFO/MainProcess] indexer_fossology_license@worker3.internal.staging.swh.network ready.
Feb 22 13:21:45 worker3 python3[30440]: [2021-02-22 13:21:45,572: INFO/MainProcess] Received task: swh.indexer.tasks.ContentFossologyLicensePartition[96f4c340-281f-4fd9-a688-e39fc15f0303]
Feb 22 13:21:45 worker3 python3[30440]: [2021-02-22 13:21:45,680: INFO/MainProcess] Received task: swh.indexer.tasks.ContentFossologyLicensePartition[67438480-57be-46c8-966e-333aeed2d6da]
Feb 22 13:24:07 worker3 python3[30446]: [2021-02-22 13:24:07,043: INFO/ForkPoolWorker-1] Task swh.indexer.tasks.ContentFossologyLicensePartition[96f4c340-281f-4fd9-a688-e39fc15f0303] succeeded in 141.33637197001372s: {'status': 'eventful', 'content_fossology_license:add': 1163}
Feb 22 13:24:07 worker3 python3[30440]: [2021-02-22 13:24:07,053: INFO/MainProcess] Received task: swh.indexer.tasks.ContentFossologyLicensePartition[0f04bf89-fd63-44ea-9a84-8c5091a587cc]
Feb 22 13:26:45 worker3 python3[30446]: [2021-02-22 13:26:45,608: INFO/ForkPoolWorker-1] Task swh.indexer.tasks.ContentFossologyLicensePartition[67438480-57be-46c8-966e-333aeed2d6da] succeeded in 158.55806935403962s: {'status': 'eventful', 'content_fossology_license:add': 1171}
Feb 22 13:26:45 worker3 python3[30440]: [2021-02-22 13:26:45,619: INFO/MainProcess] Received task: swh.indexer.tasks.ContentFossologyLicensePartition[43ac6361-e2a2-42a8-b024-ce7d8cb40d28]

And now, we start having licenses computed on staging as well:

swh-indexer=> \conninfo
You are connected to database "swh-indexer" as user "guest" on host "db1.internal.staging.swh.network" (address "192.168.130.11") at port "5432".
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
swh-indexer=> select now(), count(*) from fossology_license; select now(), count(*) from content_fossology_license;
              now              | count
-------------------------------+-------
 2021-02-22 13:24:52.615727+00 |    75
(1 row)

             now              | count
------------------------------+-------
 2021-02-22 13:24:52.62186+00 |  1163
(1 row)
ardumont claimed this task.
ardumont moved this task from deployed/landed/monitoring to done on the System administration board.