Query: Advanced Search

	Include stories about projects I am a member of.

Thanks for making mypy happy ;)

LGTM, it's not really ideal but it avoid to break the current subnet spliting we are using

Unfornunately I didn't find any tool supporting natively the backup of postgresql through zfs snapshots.
We have a couple of alternatives:

Use borg to to implement a clean backup of the postgresql data directory based on hooks [1]
Use a tool like pgbackrest[2], the backup type and storage target need to be choosen (S3, storage server, ...)
Implement a backup based on zfs snapshots, which should not be too complicated but we need to manage all the plumbing to deal with full/incremental backups and the history cleaning

Document the mirror credentials management

rebase
update according the feedback

thanks, just a small inlined remark regarding the httpie package installation

netbox: migrate the vm to the admin vlan

Configure the kafka clusters environment for the consumer lag exporter

vagrant: declare the getty ip

kafka: add a script to create the kafka credentials

Prepare the bojiman migration to the admin vlan

vagrant: update the facts at each run

fix the typo on the getty hostname

Rebase
Update the kafka credentials part to use the new user management script
Add a section to describe the username pattern
Add a section to describe how to create the read-only storage credentials

fix the admin layout

add the puppet header on the script

Install the scripts for all the environments in getty, the journal orchestrator;
as the cluster configurations are now global, it impacts the consumer group exporter. It make sense to move it from storage1 to getty to also centralize this part (FW rules will need to be adapted accordingly);

vagrant: Declare the getty node

vagrant: Declare the default vm type to debian

staging: remove duplicate properties

staging: Use the right elasticsearch cluster to display the scn task logs

remove the fact of the decommissioned webapp0 production server

Yes it could, It will be cleaner, I will update in this way.
I took a shortcut estimating we should know which broker is up or not, to avoid to have to build the connection string :)

The ack late propery is deployed in staging and production.
The number of scn in the 'scheduled' status should decrease.

Reactivate the ack late configuration for the high priority tasks

Move mirror documentation from devel to sysadm

fix according the review

Remove deprecated documentation

Add the related task

It looks it's the normal behavior of celery with the current configuration:

After analyzing further the logs, it seems the celery process is accepting the task eb038b42-713e-4c23-a655-17d04f23870a whereas a previous loading is still running
A more contextual logs:

Previous task is received and started:

Jan 17 16:19:31 worker14 python3[2180500]: [2022-01-17 16:19:31,868: INFO/MainProcess] Received task: swh.loader.git.tasks.UpdateGitRepository[70c13ea8-19f4-4ac7-ae4f-ea71f80fc25a]  
Jan 17 16:19:32 worker14 python3[2721731]: [2022-01-17 16:19:32,467: INFO/ForkPoolWorker-133] Load origin 'https://git.libreoffice.org/translations' with type 'git'

The new task is accepted whereas the previous one is not yet finished:

Jan 17 16:26:10 worker14 python3[2180500]: [2022-01-17 16:26:10,683: INFO/MainProcess] Received task: swh.loader.git.tasks.UpdateGitRepository[eb038b42-713e-4c23-a655-17d04f23870a]

the first task ends:

Jan 17 19:48:20 worker14 python3[2721731]: [51B blob data]
Jan 17 19:48:20 worker14 python3[2721731]: [3.6K blob data]
Jan 17 19:49:57 worker14 python3[2721731]: [2022-01-17 19:49:57,085: ERROR/ForkPoolWorker-133] Loading failure, updating to `failed` status
                                           Traceback (most recent call last):
                                             File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 338, in load
                                               more_data_to_fetch = self.fetch_data()
                                             File "/usr/lib/python3/dist-packages/swh/loader/git/loader.py", line 277, in fetch_data
                                               self.origin.url, base_repo, do_progress
                                             File "/usr/lib/python3/dist-packages/swh/loader/git/loader.py", line 213, in fetch_pack_from_origin
                                               progress=do_activity,
                                             File "/usr/lib/python3/dist-packages/dulwich/client.py", line 2001, in fetch_pack
                                               progress,
                                             File "/usr/lib/python3/dist-packages/dulwich/client.py", line 845, in _handle_upload_pack_tail
                                               SIDE_BAND_CHANNEL_PROGRESS: progress,
                                             File "/usr/lib/python3/dist-packages/dulwich/client.py", line 604, in _read_side_band64k_data
                                               cb(pkt)
                                             File "/usr/lib/python3/dist-packages/swh/loader/git/loader.py", line 201, in do_pack
                                               f"Pack file too big for repository {origin_url}, "
                                           OSError: Pack file too big for repository https://git.libreoffice.org/translations, limit is 4294967296 bytes, current size is 4294966867, would write 65515
Jan 17 19:49:59 worker14 python3[2721731]: [2022-01-17 19:49:59,468: INFO/ForkPoolWorker-133] Task swh.loader.git.tasks.UpdateGitRepository[70c13ea8-19f4-4ac7-ae4f-ea71f80fc25a] succeeded in 12627.452697345056s: {'status': 'failed'}

the second task loading starts after 3 hours:

Jan 17 19:50:02 worker14 python3[2736039]: [2022-01-17 19:50:02,768: INFO/ForkPoolWorker-134] Load origin 'https://github.com/webtorrent/webtorrent-desktop' with type 'git'

create the admin vm grafana0

Update after the apply

pergamon cleanup done with D6952.

grafana: clean pergamon resources

remove unecessary spaces

restore useful removed comment

grafana: disable grafana0 backups

grafana0 server created
configuration applied on the reverse proxy
network configuration checks:
- grafana0 -> pergamon:9090 (prometheus): Allowed for all in a floating rule
- grafana0 -> esnode[1-3]:9200 (elasticsearch queries):
public DNS updated to use rp1 (swh-rproxy3.inria.fr) as the public entry point
pergamon apache configuration temporary updated to point to the new grafana0 backend server ( and add a temporary rule on the firewall to allow it)

grafana: fix the missing database name properties

webapp: Temporary use the main database

Move grafana on a dedicated server behind the admin RP

Advanced Search
Use Results
Edit Query
Hide Query

Jan 28 2022

Jan 27 2022

Jan 25 2022

Jan 24 2022

Jan 21 2022

Jan 20 2022

Jan 19 2022

Jan 18 2022

Jan 14 2022

Advanced SearchUse ResultsEdit QueryHide Query