In D7039#183023, @olasd wrote:

Thanks.

--build-dep-resolver=aptitude should only be used when building with an extra-repository which has a non-default priority, that is only when using a -backports suite (so only for the bullseye and buster instructions). It should probably be documented in the list of "useful options" rather than as the default.

It might need a dedicated bot user to be created on hedgedoc also.

Things to fix in the script:

forgot one...

Use roles as suggested by vlorentz

yat (yet-another-typo)

more typos

and fix the spurious () (thx ardumont)

fix rst syntax (thx D6995 review)

thx

Is there a reason not to close this task?

update using new statsd.status_gauge() context manager (in swh.core 1.1)

In D6944#181139, @olasd wrote:

The code for the gauges feels like something that would be usefully handled with a context manager.

Something like (untested)

class StatsdStatusGauges:
    def __init__(self, metric_name: str, statuses: Collection[str], common_tags: Optional[Dict[str, str]] = None):
        self.metric_name = metric_name
        self.statuses = set(statuses)
        self.common_metrics = common_tags or {}
        self.current_status: Optional[str] = None

    def reset_gauges(self):
        self.current_status = None
        for status in self.statuses:
            statsd.gauge(self.metric_name, 0, {**self.common_tags, "status": status})

    def send_current_gauge(self, value: int):
        if self.current_status is not None:
            statsd.gauge(self.metric_name, value, {**self.common_tags, "status": self.current_status})

    def set(self, new_status: str):
        if new_status not in self.statuses:
            raise ValueError(f'{new_status} not in {self.statuses}')

        # May not be needed; May even be counter-productive if we want to send the gauges to keep them around in the statsd exporter
        if new_status == self.current_status:
            return

        self.send_current_gauge(0)
        self.current_status = new_status
        self.send_current_gauge(1)

    def __enter__(self):
        self.reset_gauges()
        return self

    def __exit__(self, *exc):
        self.reset_gauges()
        return False

Which would be used like:

with StatsdStatusGauges(JOURNAL_STATUS_METRIC, {"processing", "waiting"}) as status_gauge:
    [...]
    status_gauge.set("waiting")
    [...]
    status_gauge.set("processing")

In D6884#181115, @olasd wrote:

I'm kinda wondering if this import stuff should move to a common module - I think we do kind of the same thing with entrypoints?

Rebase and update according suggestions

improve comment as suggested by ardumont

Add the cli option to configure this concurrency value

fine for me (but plz give a bit more insight)

I guess this is then related to T3653 somehow

move the code in get_journal_client

make the warning an exception, as suggested by vlorentz

In D6875#178755, @olasd wrote:

statsd.timing/statsd.timed do full histograms. Do we really want to keep bucketed counts for all of these values, or just a running total?

Gawd this is horrible! (not your fault!)

I'd feel more comfortable also if we have a good (aka documented and understood) reason for doing this.

vlorentz' comment + update copyright timestamps

closed by 259bf6fe1e3bacbcd2e91f8f3d55d49f5219892c

In T3134#76032, @vlorentz wrote:

wishlist: it would be nice ot be able to check the whole hash of a revision/release even when the author name/email are replaced by a hash. (eg. by making SWHIDv2 a tree hash)

I guess https://grafana.softwareheritage.org/d/d3l2oqXWz/s3-object-copy?orgId=1 is almost an answer to this task

LGTM (but see my comment)

Same comment as in D6712

It would really nice to have a better explanation of what this flag is added for and why. Why do we add complexity in the code for this? I know there are good reasons for that, but I cannot see them just reading the code or the commit message.

There is a small typo in the commit message (adn instead of and)