Page MenuHomeSoftware Heritage

Archive integrityTag
ActivePublic

Members

  • This project does not have any members.
  • View All

Watchers

  • This project does not have any watchers.
  • View All

Recent Activity

Wed, Jun 8

ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7968: Add missing coverage on `swh db version` cli.
Wed, Jun 8, 10:46 AM · Archive integrity, System administration

Tue, Jun 7

ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7965: postgres db: Create guest user at db initialization time.
Tue, Jun 7, 2:28 PM · Archive integrity, System administration

Thu, Jun 2

ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7949: db.BaseDb: Propose default get_current_version method implementation.
Thu, Jun 2, 5:21 PM · Archive integrity, System administration
ardumont added a revision to T4284: scrubber does not comply to what's expected by the swh db tooling: D7943: Revert "cli.db: Use attribute current_version instead of undeclared getter".
Thu, Jun 2, 11:42 AM · Archive integrity

Tue, May 31

ardumont closed T4228: scrubber: Investigate the apparent lock (staging) as Resolved.
Tue, May 31, 2:22 PM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

No more idle in transaction queries and corrupt_object still growing [1].
We can close this.

Tue, May 31, 2:22 PM · Archive integrity, System administration
ardumont closed T4284: scrubber does not comply to what's expected by the swh db tooling as Resolved.
Tue, May 31, 2:18 PM · Archive integrity
ardumont added a comment to T4284: scrubber does not comply to what's expected by the swh db tooling.

Scrubber got finally adapted so migration script is now ok.
Closing this.

Tue, May 31, 2:18 PM · Archive integrity
ardumont added a project to T4284: scrubber does not comply to what's expected by the swh db tooling: Archive integrity.
Tue, May 31, 2:17 PM · Archive integrity
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

We now have corrupt object being stored in the db:

11:26:40 swh-scrubber@db1:5432=> select count(*) from corrupt_object ;
+-------+
| count |
+-------+
| 59238 |
+-------+
(1 row)
Tue, May 31, 11:29 AM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

And v0.0.6 tagged and deployed.

Tue, May 31, 11:26 AM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

With [1] applied, the insert query got executed!

Tue, May 31, 10:58 AM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

Stopping all services and letting run only 1 checker service.

Tue, May 31, 10:58 AM · Archive integrity, System administration
ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7914: Wrap query in transaction.
Tue, May 31, 10:36 AM · Archive integrity, System administration
ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7913: db: Grant read access to guest user on all tables of the schema.
Tue, May 31, 9:19 AM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

Nothing bulged, same id process...
(Well, new ones appeared but old ones are still waiting for something)

Tue, May 31, 9:05 AM · Archive integrity, System administration

Mon, May 30

ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

We have plenty of process stuck with "idle in transaction". According to [1], this means
"waiting for client inside a BEGIN block", so there might be issues in the scrubber code
[2]?

Mon, May 30, 6:48 PM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

There seem to still exist staling queries [1] and nothing gets written to the db [2]:

Mon, May 30, 6:35 PM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

Finally, after installing postgresql-client-11 and stopping scrubber services [1]:

Mon, May 30, 6:10 PM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

It was still not ok as the upgrade tooling also expects the sql/upgrades/<final-version>.sql to exist and be packaged.
There was nothing there hence [1]

Mon, May 30, 6:00 PM · Archive integrity, System administration
ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7912: Reference the scrubber db model upgrade (from version 1 to 2).
Mon, May 30, 5:54 PM · Archive integrity, System administration
ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7906: db: Provide a default get_current_version method to db classes.
Mon, May 30, 4:14 PM · Archive integrity, System administration

May 30 2022

ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7905: Unify factory to use keyword 'postgresql' over deprecated 'local'.
May 30 2022, 3:18 PM · Archive integrity, System administration
ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7904: db: Bump to version 2.
May 30 2022, 3:12 PM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

And now time to unstuck the debian build...

May 30 2022, 2:56 PM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

And now time to unstuck the debian build...

May 30 2022, 2:24 PM · Archive integrity, System administration
ardumont added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7902: requirements: Add missing dependency.
May 30 2022, 2:07 PM · Archive integrity, System administration

May 25 2022

vlorentz added a revision to T4228: scrubber: Investigate the apparent lock (staging): D7896: Fix crash when using datastore_get_or_add for an existing datastore.
May 25 2022, 3:26 PM · Archive integrity, System administration
ardumont updated subscribers of T4228: scrubber: Investigate the apparent lock (staging).
May 25 2022, 2:26 PM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

@vlorentz And now we got [1] to unstuck.

May 25 2022, 2:21 PM · Archive integrity, System administration
ardumont moved T4228: scrubber: Investigate the apparent lock (staging) from in-progress to deployed/landed/monitoring on the System administration board.
May 25 2022, 2:07 PM · Archive integrity, System administration
ardumont changed the status of T4228: scrubber: Investigate the apparent lock (staging) from Open to Work in Progress.
May 25 2022, 2:07 PM · Archive integrity, System administration
ardumont added a comment to T4228: scrubber: Investigate the apparent lock (staging).

Well, i don't get it the apparent locking situation.
Services that do need to write to the db have the proper credentials for that.

May 25 2022, 2:01 PM · Archive integrity, System administration

May 10 2022

ardumont added a project to T4228: scrubber: Investigate the apparent lock (staging): Archive integrity.
May 10 2022, 3:37 PM · Archive integrity, System administration
ardumont added a hashtag to Archive integrity: #archive-integrity.
May 10 2022, 3:37 PM
ardumont set the icon for Archive integrity to Tag.
May 10 2022, 3:36 PM
ardumont created Archive integrity.
May 10 2022, 3:36 PM