♟ ardumont

wip: Start swh-apps/update-dependencies job

README.md: Adapt docker instructions

fix typo for fun and profit! ;)

couple of questions inline.

Fixes included in the diff landed [1].

Install `swh provenance origin from-journal` cli and tests

14:07:39 swh-scrubber@belvedere:5432=> select * from corrupt_object where id='swh:1:rel:a03d79c816c7ffebe3e8d90d7d395482075b086f';
+-[ RECORD 1 ]-----+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id               | swh:1:rel:a03d79c816c7ffebe3e8d90d7d395482075b086f                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| datastore        | 1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| object           | \x88a46e616d65c41864656269616e2f302e32362e312d307562756e7475322e32a76d657373616765c44574616767696e67207061636b61676520707974686f6e2d676c616e63652d73746f72652076657273696f6e2064656269616e2f302e32362e312d307562756e7475322e320aa6746172676574c414e4c28d0c01ea1274e8e9b28f113572c36211d538ab7461726765745f74797065a87265766973696f6ea973796e746865746963c2a6617574686f7283a866756c6c6e616d65c429436f72657920427279616e74203c636f7265792e627279616e744063616e6f6e6963616c2e636f6d3ea46e616d65c40c436f72657920427279616e74a5656d61696cc41a636f7265792e627279616e744063616e6f6e6963616c2e636f6da46461746582a974696d657374616d7082a77365636f6e6473ce5cfe6f7bac6d6963726f7365636f6e647300ac6f66667365745f6279746573c4052d30343030a26964c414a03d79c816c7ffebe3e8d90d7d395482075b086f |
| first_occurrence | 2022-06-27 15:43:47.263775+00                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Looks good enough to start having a local jenkins, that will ease tampering with whatever we need to develop on it.
This without having to tamper directly with the main swh ci instance.
So thanks.

thx (and i guess that fixes the master build as well).

Drop self.conn shenanigans (tests no longer stuck but journal client tests are not happy)

I'm of a mind of just push it and check whether that's not some discrepancy with the master build somehow.
Master build is fine and local build as well.

Revert to original commit

Tentatively work around the rabbitmq tests build failures 5/n

Tentatively work around the rabbitmq tests build failures 4/n

Tentatively work around the rabbitmq initialization 3/n

Tentatively work around the rabbitmq initialization.

What the next steps here?

well, i'll trust the copyright date are ok according to the existing code ;)

scrubber: Set objstorage key to noop unused by scrubber services

including the right amount of commits

In T4361#87419, @ardumont wrote:

wip tryout [1] without those tests to ensure whether it's the rabbitmq fixture which hangs or not.

[1] D8023#209122

wip tryout [1] without those tests to ensure whether it's the rabbitmq fixture which hangs or not.

wip tryout (drop rabbitmq from fixture) to check if that still hangs or not

That also seem to incur a heavy load on the jenkins node thyssen:

11:28 <+swhbot> icinga PROBLEM: service load on thyssen.internal.softwareheritage.org is WARNING: WARNING - load average: 30.18, 22.54, 11.58

It's been done for a while now.
And there is a mechanism in place in swh.core.tarball to extend it further if need be.
Closing.

So taking a bit more look into this possible new lister, we'd end up with the following
possible outputs:

artifacts url which are mostly tarballs [1] and sometimes file [2]
dvcs repositories delegated to dedicated loader to ingestion: svn [3], hg [4], git [5] (out of guix manifest)
Other stuff can be ignored [6]

provenance: Increase build timeout

You should set that in the scrubber config in common.yaml rather than using a per-host override

yes, totally, i'll attend to that later in the day.

You should set that in the scrubber config in common.yaml rather than using a per-host override

Seems to have done the job:

07:59:35 swh-scrubber@belvedere:5432=> select now(), count(*) from corrupt_object ;
+-------------------------------+-------+
|              now              | count |
+-------------------------------+-------+
| 2022-06-28 05:59:39.490737+00 | 76511 |
+-------------------------------+-------+
(1 row)

scrubber: Deactivate the unneeded objstorage configuration part

Amend commit message

Use proper repository to update diff

After discussing with olasd, turns out that the part declaring the objstorage is not currently needed so i'll try to do better.

ardumont (Antoine R. Dumont)
UserAdministrator

Projects (11)
View All

User Details

Recent Activity
View All

Yesterday

Wed, Jun 29

Tue, Jun 28

Mon, Jun 27

ardumont (Antoine R. Dumont)UserAdministrator

Projects (11)View All