Change Details

It's running slowly for now as the current execution is simply a loop over the mapping file then execute the ingestion. We could make it go faster leveraging the loader_oneshot and rabbitmq. For this, we need to: - iterate over the mapping file and send tasks `swh.loader.mercurial.tasks_from_disk.LoadMercurialFromDisk` - adapt the loader_oneshot so it consumes those tasks ^ That will make the ingestion go faster since the loader_oneshot is currently configured with a concurrency of 7.

It's running slowly for now as the current execution is simply a loop over the mapping file then execute the ingestion. We could make it go faster leveraging the loader_oneshot and rabbitmq. For this, we need to iterate over the mapping file and send tasks `swh.loader.mercurial.tasks.LoadMercurial` with the proper parameters. And that should be enough as the loader_oneshot already consumes those tasks ^. That will make the ingestion go faster since the loader_oneshot is currently configured with a concurrency of 7.

It's running slowly for now as the current execution is simply a loop over the mapping file then execute the ingestion. We could make it go faster leveraging the loader_oneshot and rabbitmq. For this, we need to: - iterate over the mapping file and send tasks `swh.loader.mercurial.tasks_from_disk.LoadMercurialFromDisk` - adapt` with the proper parameters. And that should be enough as the loader_oneshot so italready consumes those tasks ^. That will make the ingestion go faster since the loader_oneshot is currently configured with a concurrency of 7.