Improve integrity of ingested content
Closed, MigratedEdits Locked
Actions

Assigned To

Authored By

	rdicosmo
	Mar 15 2021, 8:47 PM

Description

This is a meta-task to track activities related to improving stability and reliability of the archive by preventing or detecting corruption.

rDLDG Git loader
	Closed		D6281 converters: Recompute hashes and check they match the originals
		D6504	rDLDGa40f1e00cd16 converters: Fix detection of tree entries with non-standard commit/tree mode.
		D6045	rDLDGc67ab0269ef2 converters: Preserve GPG signatures on releases

Status	Assigned	Task
Migrated	gitlab-migration	T3135 Improve integrity of ingested content
Migrated	gitlab-migration	T399 (Re-)Compute data checksums before insertion
Migrated	gitlab-migration	T3010 Enable the validating storage proxy in production
Migrated	gitlab-migration	T3011 Enable the validating storage proxy on staging
Migrated	gitlab-migration	T3012 Check all objects in the production storage/journal have a correct hash
Migrated	gitlab-migration	T75 Check integrity of directories, revisions, and releases
Migrated	gitlab-migration	T366 Refactor the person table
Migrated	gitlab-migration	T3594 Faithfully store weird git objects
Migrated	gitlab-migration	T3595 Support disordered directory entries in git
Migrated	gitlab-migration	T3596 Support "weird" permissions in directories
Migrated	gitlab-migration	T3598 Support revisions with "extra headers" not at the end
Migrated	gitlab-migration	T3752 Store/represent time offsets as strings
Migrated	gitlab-migration	T3819 Deploy swh.model 4.1.0 / swh.storage 0.41.0 to production
Migrated	gitlab-migration	T3753 Store original git manifests
Migrated	gitlab-migration	T3841 regularly scrub all the data stores of swh
Migrated	gitlab-migration	T4136 Add an "history completeness check"
		Restricted Maniphest Task
		Restricted Maniphest Task
Migrated	gitlab-migration	T4371 Deploy swh-scrubber on all storage instances
Migrated	gitlab-migration	T3878 Fix existing corrupt objects