- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed Advanced Search
Advanced Search
Advanced Search
Jan 8 2023
Jan 8 2023
gitlab-migration changed the status of T508: prototype: git archive from SWH, a subtask of T67: prototype: git clone from SWH, from Resolved to Migrated.
gitlab-migration changed the status of T508: prototype: git archive from SWH from Resolved to Migrated.
gitlab-migration changed the status of T78: Unable to perfectly round-trip releases pointed at by occurrences, a subtask of T67: prototype: git clone from SWH, from Resolved to Migrated.
gitlab-migration changed the status of T67: prototype: git clone from SWH from Resolved to Migrated.
Oct 19 2022
Oct 19 2022
gitlab-migration changed the status of T3920: Errors from vault workers are not logged to Sentry from Resolved to Migrated.
gitlab-migration changed the status of T3918: Deploy swh-vault workers v1.4.2 on prod and staging from Resolved to Migrated.
gitlab-migration changed the status of T3734: Deploy latest swh.vault v1.3.0 from Resolved to Migrated.
gitlab-migration changed the status of T3543: Debian package python3-swh.graph.client from Resolved to Migrated.
gitlab-migration changed the status of T3543: Debian package python3-swh.graph.client, a subtask of T3518: Enable vault cookers to access swh-graph, from Resolved to Migrated.
gitlab-migration changed the status of T3518: Enable vault cookers to access swh-graph, a subtask of T3505: Make the git-bare cooker available to the staff and beta-testers in the production webapp, from Resolved to Migrated.
gitlab-migration changed the status of T3518: Enable vault cookers to access swh-graph from Resolved to Migrated.
gitlab-migration changed the status of T3507: prod: vault: Deploy v1.0.0, a subtask of T3506: Get rid of the concept of vault "object_type", from Resolved to Migrated.
gitlab-migration changed the status of T3503: staging: vault: Deploy v1.0.0, a subtask of T3507: prod: vault: Deploy v1.0.0, from Resolved to Migrated.
gitlab-migration changed the status of T3503: staging: vault: Deploy v1.0.0 from Resolved to Migrated.
gitlab-migration changed the status of T3412: Deploy vault "git bare" tarball cooker (swh-vault v0.6) from Resolved to Migrated.
gitlab-migration changed the status of T3412: Deploy vault "git bare" tarball cooker (swh-vault v0.6), a subtask of T843: Vault: Add a "git bare" tarball cooker, from Resolved to Migrated.
gitlab-migration changed the status of T936: Deploy a storage backend on azure, a subtask of T888: Deploy the Vault and a DB replica on Azure, from Resolved to Migrated.
gitlab-migration changed the status of T888: Deploy the Vault and a DB replica on Azure from Resolved to Migrated.
gitlab-migration changed the status of T883: set up a replica of the main DB on azure, a subtask of T888: Deploy the Vault and a DB replica on Azure, from Resolved to Migrated.
gitlab-migration changed the status of T706: establish openvpn link with unibo for the vault from Resolved to Migrated.
Jun 27 2022
Jun 27 2022
Jun 23 2022
Jun 23 2022
Jun 21 2022
Jun 21 2022
vlorentz removed a project from T4345: get_filtered_files_content fails with "unexpected status None for content": Mirror.
vlorentz triaged T4345: get_filtered_files_content fails with "unexpected status None for content" as Normal priority.
Jun 1 2022
Jun 1 2022
May 31 2022
May 31 2022
douardda triaged T4289: Replace usage of swh.core's postgresql_fact by stock pytest_postgresql's factory function as Normal priority.
Mar 25 2022
Mar 25 2022
vlorentz closed T3552: Fix corrupted releases, revisions, and directories in the storage, a subtask of T3551: Fix git-fsck errors in the git-bare cooker, as Resolved.
vlorentz closed T3552: Fix corrupted releases, revisions, and directories in the storage, a subtask of T887: Vault: "snapshot" cooker, as Resolved.
Mar 7 2022
Mar 7 2022
vlorentz updated the task description for T3924: Write mailmaps after cooking git-bare archives with display names?.
Mar 4 2022
Mar 4 2022
Already done in D7264...
Feb 14 2022
Feb 14 2022
Feb 10 2022
Feb 10 2022
vlorentz updated the task description for T3924: Write mailmaps after cooking git-bare archives with display names?.
vlorentz triaged T3924: Write mailmaps after cooking git-bare archives with display names? as Normal priority.
Feb 9 2022
Feb 9 2022
ardumont changed the status of T3920: Errors from vault workers are not logged to Sentry from Invalid to Resolved.
It seems everything is already ok (another cooking task issue reported [1]) in the end so closing this.
I don't know how to trigger an error in the vault, currently; you'd have to change the code to manually do that :|
Also that's a cooker worker issue from sentry from 18h ago or so (as of the moment of this comment).
I'll let you trigger some cooking and reports here your finding.
Something is bothering me, isn't there some catchall exceptions happening in the vault source code somewhere already?
Feb 8 2022
Feb 8 2022
I wonder if replacing @worker_init.connect with @worker_process_init.connect at https://forge.softwareheritage.org/source/swh-scheduler/browse/master/swh/scheduler/celery_backend/config.py$157 would work.
so I'm guessing Celery is eating logs somehow, so Sentry doesn't see them
So i don't currently know what's wrong (if anything is).
So i was wrong, it is correctly set [1].
And there are sentry issues about workers [2].
ardumont changed the status of T3920: Errors from vault workers are not logged to Sentry from Open to Work in Progress.
ardumont moved T3920: Errors from vault workers are not logged to Sentry from Backlog to Weekly backlog on the System administration board.
ardumont moved T3918: Deploy swh-vault workers v1.4.2 on prod and staging from in-progress to deployed/landed/monitoring on the System administration board.
Yes, i confirm (from #swh-sysadm discussion)
Looking at puppet configuration, my guess is that the sentry_dsn is not set for the vault cookers.
ongoing upgrade for production workers.
ardumont updated the task description for T3918: Deploy swh-vault workers v1.4.2 on prod and staging.
In the end, the plan is more simple
ardumont updated the task description for T3918: Deploy swh-vault workers v1.4.2 on prod and staging.
ardumont renamed T3918: Deploy swh-vault workers v1.4.2 on prod and staging from Deploy swh-vault v1.4.2 on prod and staging to Deploy swh-vault workers v1.4.2 on prod and staging.
ardumont updated the task description for T3918: Deploy swh-vault workers v1.4.2 on prod and staging.
that one is mostly fixed, the current blocker to cook snapshots (with reasonable performance) is T2220
- Check storage server api version:
ardumont@vault:~% hostname -f
vault.internal.staging.swh.network
ardumont@vault:~% sudo grep 5003 /etc/softwareheritage/vault/server.yml
url: http://storage1.internal.staging.swh.network:5003/
ssh storage1.internal.staging.swh.network dpkg -l python3-swh.storage | grep ii
ii python3-swh.storage 0.43.0-1~swh1~bpo10+1 all Software Heritage storage utilitiesardumont changed the status of T3918: Deploy swh-vault workers v1.4.2 on prod and staging from Open to Work in Progress.
ardumont updated the task description for T3918: Deploy swh-vault workers v1.4.2 on prod and staging.
ardumont moved T3918: Deploy swh-vault workers v1.4.2 on prod and staging from Backlog to Weekly backlog on the System administration board.
vlorentz added a revision to T3504: Make the git-bare cooker publicly available: D7120: Allow vault cooking from the UI.
Dec 2 2021
Dec 2 2021
vlorentz closed T3731: Try parallelizing directory_get_entries with an async swh-storage client, a subtask of T3504: Make the git-bare cooker publicly available, as Resolved.
Nov 26 2021
Nov 26 2021
Nov 22 2021
Nov 22 2021
vlorentz closed T3727: Show the progress of vault tasks, a subtask of T3550: Compute and show ETA for vault tasks, as Resolved.
vlorentz added a comment to T3731: Try parallelizing directory_get_entries with an async swh-storage client.
Making an async swh-storage (or RPC client for our framework, in general) turns out the not be worth it, threads are just as efficient and don't require complex hacks
vlorentz added revisions to T3731: Try parallelizing directory_get_entries with an async swh-storage client: D6650: git_bare: Fetch directories concurrently, using threads., D6651: WIP: Add AsyncRPCClient, D6652: WIP: Add AsyncRemoteStorage, D6653: git_bare: Add an alternative using asyncio instead of threads..
Nov 18 2021
Nov 18 2021
ardumont moved T3507: prod: vault: Deploy v1.0.0 from deployed/landed/monitoring to done on the System administration board.
Nov 17 2021
Nov 17 2021
ardumont moved T3734: Deploy latest swh.vault v1.3.0 from in-progress to deployed/landed/monitoring on the System administration board.
Nov 16 2021
Nov 16 2021