Diffusion Datasets 23853dbfacd4

luigi: Add DownloadFromS3 task
23853dbfacd4
Actions

Tags

None

Subscribers

None

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

luigi: Add DownloadFromS3 task

This will allow running swh-graph tasks easily on machines that didn't
export the graph themselves.

Details

Provenance

vlorentz	Authored on Nov 10 2022, 4:41 PM
vlorentz	Pushed on Nov 15 2022, 11:46 AM

Differential Revision

D8832: luigi: Add DownloadFromS3 task

Build Status

Buildable 32811
Build 51409: test-and-build	Jenkins console · Jenkins

Event Timeline

vlorentz committed rDDATASET23853dbfacd4: luigi: Add DownloadFromS3 task (authored by vlorentz).Nov 15 2022, 10:59 AM

vlorentz added an edge: D8832: luigi: Add DownloadFromS3 task.Nov 15 2022, 11:46 AM

Harbormaster completed building B32811: rDDATASET23853dbfacd4: luigi: Add DownloadFromS3 task.Nov 15 2022, 11:49 AM

swh-public-ci mentioned this in D8864: luigi: Remove copies of stamp files to/from S3.Nov 21 2022, 3:45 PM

swh-public-ci mentioned this in D8865: luigi: Add LocalExport task.Nov 21 2022, 3:54 PM

swh-public-ci mentioned this in D8866: luigi: Add swh-dataset version to export/meta.json.

Commit No Longer Exists

This commit no longer exists in the repository.