one suggestion inline.
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Advanced Search
May 31 2022
(Use imperative form in tasks, it mostly matches how we name task in the forge ;)
one remark, I'd be neat you add the related task id on those diffs (if there is one).
No more idle in transaction queries and corrupt_object still growing [1].
We can close this.
Scrubber got finally adapted so migration script is now ok [1]
Closing this.
A couple of remarks/suggestions inline.
instead of swh-core's postgresql_fact one, since we actually do not
use its custom features any more in swh-storage.Well, initially the feature used was the reset between tables to make
it faster than it was (and it worked 'cause it was hellishly slow to drop
and create tables again).So, now storage uses the template to make the tests go as fast, is that it?
yes, when using the load argument of the postgresql_proc factory, it is used to initialize the template database used to create the db for each test.
For this to work, the dbname used when calling the postgresql_proc factory function must be the same as the one used to call the postgresql factory (so the removal of dbname="storage" in this diff; just use default db names unless specifically needed).
instead of swh-core's postgresql_fact one, since we actually do not
use its custom features any more in swh-storage.
We now have corrupt object being stored in the db:
11:26:40 swh-scrubber@db1:5432=> select count(*) from corrupt_object ; +-------+ | count | +-------+ | 59238 | +-------+ (1 row)
With @douardda, we agreed on irc to postpone this diff a bit the time david found back how to properly update the postgres template (to integrate the guest user).
So the part spawning back the pytest-plugin postgresql_fact becomes unnecessary [1]
And v0.0.6 tagged and deployed.
Reword commit message
With [1] applied, the insert query got executed!
Stopping all services and letting run only 1 checker service.
Nothing bulged, same id process...
(Well, new ones appeared but old ones are still waiting for something)
May 30 2022
We have plenty of processes stuck with "idle in transaction". According to [1], this means
"waiting for client inside a BEGIN block", so there might be issues in the scrubber code
[2]?
There seem to still exist staling queries [1] and nothing gets written to the db [2]:
Finally, after installing postgresql-client-11 and stopping scrubber services [1]:
It was still not ok as the upgrade tooling also expects the sql/upgrades/<final-version>.sql to exist and be packaged.
There was nothing there hence [1]
Thanks again!
Build is back to green thanks to you @vlorentz.
How great, now, we got the following with D7910 applied:
Add tests over the factory change