⚓ T575 Pipe contents copied to azure to the indexer orchestrator

Status	Assigned	Task
		Unknown Object (Maniphest Task)
Migrated	gitlab-migration	T574 Pipeline copy content to azure and then compute multiple indexes independently (meta task)
Migrated	gitlab-migration	T575 Pipe contents copied to azure to the indexer orchestrator
Migrated	gitlab-migration	T582 Orchestrator - Optional filtering on existing contents

ardumont renamed this task from Pipe contents copied to azure to the indexer to Pipe contents copied to azure to the indexer orchestrator.Oct 5 2016, 11:04 AM

ardumont created this task.

ardumont created this object in space Restricted Space.

ardumont shifted this object from the Restricted Space space to the S1 Public space.Oct 5 2016, 11:08 AM

ardumont mentioned this in T579: Puppetize subset of indexers to start indexing computations.Oct 8 2016, 12:31 PM

As of today, this is the archiver (archiver-uffizi) which is in charge of copying data from our objstorages (uffizi, banco) to azure.
(currently this is deployed on uffizi).

An implementation idea is to add a hook function in the archiver which is called at the end of a successful batch copy.

And for the archiver used by the copy to azure (ArchiverToBackendWorker), implement said function to 'send those copied contents to a queue' (configured to be the indexer orchestrator).

ardumont created subtask T582: Orchestrator - Optional filtering on existing contents.Oct 13 2016, 2:15 PM

ardumont mentioned this in rDSTOf7becdeb8743: Add the means to pipe contents to another queue once copied.Oct 13 2016, 3:35 PM

ardumont mentioned this in T583: Update puppet manifests according to latest developments.Oct 13 2016, 3:40 PM

ardumont mentioned this in rSPSITE2c8a0d40f51a: data/defaults: Update archiver to azure configuration.Oct 13 2016, 3:51 PM

ardumont closed this task as Resolved.Oct 13 2016, 3:53 PM

ardumont closed subtask T582: Orchestrator - Optional filtering on existing contents as Resolved.