Rebase

Rebase and simplify

In D4568#114255, @ardumont wrote:

I got questions... for those functions in general.

Any particular reason for having those encoders/decoders hard-coded in the main encoding/decoding functions?
(i know you are here only extending the current behavior to solve the problem at hand but i mean long term)

Or can we envision refactor those within the realm of the extra_encoders/decoders at some point?
And then encode_types becomes the simple for look around calling the encoders.
(ideally something equivalent could happen in the decoding part).

Move comment back to the right place

In D4568#114274, @vlorentz wrote:

Why don't you pass signed=True to int.to_bytes/int.from_bytes, and add "one eighth of a byte" to the length (ie. replace if rem: length += 1 by length += 1) when encoding?

Storage has been extended. I've restored the function to its original version (without policy = 'oneshot').

After a few hit-and-miss reboots, the multipath configuration was set to only enable explicitly listed wwids, but the list of wwids was empty.

@vsellier I believe that we're good to push this now.

Even though that's probably not a problem in that case, in general, please avoid f-strings or manual interpolation in logging statements, but rather use the built-in expansion which uses the extra arguments passed to logger.info. This avoids some (potentially costly) string interpolation when the logger is not enabled.

D4516 by @vsellier moves all storage/objstorage/indexer storage clients from uffizi to saam

We use the puppet java module in a bunch of other places, maybe it makes sense to directly import that (which would mean using include ::java)?

D4528 carries the local storage/objstorage configuration on uffizi to saam.
D4531/D4532 adds all the local mountpoints needed for this local config to work.

There was an issue with the indexer storage package missing the swh.indexer setuptools metadata. I've moved the metadata to the swh.indexer.storage package in rDCIDX3809bb03

In D4523#112919, @vsellier wrote:

Shouldn't the multipath configuration itself also be commited?

On the saam software setup:

The downtime, scheduled for the whole day of 18 November, was posted to status.io on 16 November (https://status.softwareheritage.org/pages/maintenance/578e5eddcdc0cc7951000520/5fb2ae1fbf590a04c7fdffb0)

Looks fine to me except for the vagrant bit.

The new cluster in rocquencourt is using the built-in Kafka ACLs now (9993a81ffc7a1c8bd519b33ae63ac1145105f624).

Bumped kafka[1-4] too now. The upgrade is complete on all hosts.

Restarted esnode[1-3] again after pushing 87909f184325.

Did a rolling restart of the cluster on kafka[1-4] (enabling puppet, and running the puppet agent manually on each host in turn).

Did a rolling restart of the cluster on esnode[1-3] (enabling puppet, and running the puppet agent manually on each host in turn).

Disabled puppet on kafka[1-4] esnode[1-3].

Rebase

Rebase

All done now that orsay is being decommissionned.

Messed up my arc diff --create.