Description

The postgresql scrubber entry point is a command line which sequentially processes a potentially fairly large range of objects (currently, in production, a quarter of all objects at a time).

When the scrubber crashes, either for bugs or for external reasons, it restarts from the beginning. This biases it towards the start of each processed ranges.

It would be nicer to have the option to keep a state file, which would contain a recently successfully processed object range that the next iteration of the scrubber could restart from. This would make crash recovery more useful.

Revisions and Commits

rDSCRUB Datastore Scrubber
	D8609	rDSCRUB84fa17c00be8 storage_checker: Notify database when ranges are fully checked
	D8641	rDSCRUB282242873716 storage_checker: Do not re-check ranges already marked as checked
	D8608	rDSCRUBfef8a513dfea db: Add table checked_ranges

Related Objects
Search...

		Status	Assigned	Task
		Migrated	gitlab-migration	T4527 scrubber: keep a state file for postgresql datastores
		Migrated	gitlab-migration	T4639 Deploy swh-scrubber v0.1.1

Event Timeline

olasd triaged this task as High priority.Sep 9 2022, 6:49 PM

olasd created this task.

I think I'll implement it as a table in the scrubber's DB, this will make it easier to query the current status of scrubbing and add it to the Grafana dashboard

In T4527#92580, @vlorentz wrote:

I think I'll implement it as a table in the scrubber's DB, this will make it easier to query the current status of scrubbing and add it to the Grafana dashboard

Yeah, that's a good idea!

vlorentz mentioned this in D8608: db: Add table checked_ranges.Oct 4 2022, 2:17 PM

vlorentz mentioned this in D8609: storage_checker: Notify database when ranges are fully checked.

vlorentz added revisions: D8608: db: Add table checked_ranges, D8609: storage_checker: Notify database when ranges are fully checked.Oct 4 2022, 2:19 PM

vlorentz added a commit: rDSCRUBfef8a513dfea: db: Add table checked_ranges.Oct 7 2022, 2:56 PM

vlorentz added a revision: D8641: storage_checker: Do not re-check ranges already marked as checked.Oct 7 2022, 3:42 PM

vlorentz added a commit: rDSCRUB282242873716: storage_checker: Do not re-check ranges already marked as checked.Oct 17 2022, 3:07 PM

vlorentz added a commit: rDSCRUB84fa17c00be8: storage_checker: Notify database when ranges are fully checked.

ardumont closed subtask T4639: Deploy swh-scrubber v0.1.1 as Resolved.Oct 18 2022, 4:58 PM

gitlab-migration changed the status of subtask T4639: Deploy swh-scrubber v0.1.1 from Resolved to Migrated.Oct 19 2022, 6:08 PM

vlorentz closed this task as Resolved.Oct 24 2022, 9:34 AM