Maniphest T2207

Improve ingestion efficiency
Closed, MigratedEdits Locked
Actions

Assigned To

gitlab-migration

Authored By

	douardda
	Jan 20 2020, 2:18 PM

Tags

Subscribers

gitlab-migration

Description

KPI : Near-zero lag especially for the 2 major data sources: github and gitlab.com.

Related Objects
Search...

		Status	Assigned	Task
		Migrated	gitlab-migration	T2206 Quality of Service
		Migrated	gitlab-migration	T4080 Minimize archival lag w.r.t. upstream code hosting platforms
		Migrated	gitlab-migration	T2207 Improve ingestion efficiency
		Migrated	gitlab-migration	T846 Some objects from the original GitHub import have never actually been imported.
		Migrated	gitlab-migration	T3655 loader git: enable global deduplication of head branches before fetching them
		Migrated	gitlab-migration	T3635 git loader: enable "partial" global deduplication of revisions via the extid mapping table
		Migrated	gitlab-migration	T3654 loader git: load revisions in topological order
		Migrated	gitlab-migration	T3656 Survey revisions/releases with partially loaded history
		Migrated	gitlab-migration	T3660 Nodes with missing ancestors in SWH DAG / SWH-graph
		Migrated	gitlab-migration	T3544 Deal with GitHub removing support for git:// URLs

Event Timeline

douardda created this task.Jan 20 2020, 2:18 PM

vlorentz triaged this task as Normal priority.Jan 22 2020, 4:19 PM

vlorentz added projects: Origin-GitHub, Origin-GitLab.Jan 22 2020, 4:39 PM

douardda added a subtask: T846: Some objects from the original GitHub import have never actually been imported..Jan 23 2020, 2:01 PM

olasd changed the status of subtask T846: Some objects from the original GitHub import have never actually been imported. from Open to Work in Progress.Jan 23 2020, 6:37 PM

douardda added a subtask: T3655: loader git: enable global deduplication of head branches before fetching them.Feb 18 2022, 11:20 AM

douardda added a subtask: T3544: Deal with GitHub removing support for git:// URLs.

bchauvet mentioned this in T4080: Minimize archival lag w.r.t. upstream code hosting platforms.Mar 23 2022, 4:34 PM

bchauvet renamed this task from 0 lag to Improve ingestion efficiency .Apr 11 2022, 11:44 AM

bchauvet added a parent task: T4080: Minimize archival lag w.r.t. upstream code hosting platforms.

bchauvet updated the task description. (Show Details)

olasd closed subtask T3544: Deal with GitHub removing support for git:// URLs as Resolved.Apr 29 2022, 4:11 PM

gitlab-migration changed the status of subtask T3544: Deal with GitHub removing support for git:// URLs from Resolved to Migrated.Jan 8 2023, 4:35 PM

gitlab-migration closed subtask T846: Some objects from the original GitHub import have never actually been imported. as Migrated.Jan 8 2023, 4:59 PM

gitlab-migration closed subtask T3655: loader git: enable global deduplication of head branches before fetching them as Migrated.Jan 8 2023, 5:03 PM

This task has been migrated to GitLab.