It should:
- export files to the local disk
- upload to S3
- define Athena tables
Description
Description
Revisions and Commits
Revisions and Commits
| rDGRPH Compressed graph representation | |||
| D8919 | rDGRPHb76801259953 Add CLI script to generate Luigi config and call it | ||
| rDDATASET Datasets | |||
| D8926 | rDDATASETc717f60fe08e luigi.RunExportAll: Default to exporting all formats | ||
| D8925 | rDDATASETeceaf73f0fba luigi.CreateAthena: Fix validation of DB name | ||
| D8924 | rDDATASET22f7ed11f688 exporters/orc: Fix crash on visit status with no type | ||
| D8829 | rDDATASET058e568492ba Add luigi tasks | ||
| D8828 | rDDATASETeea3e15bf7e4 cli: Move the main code of export_graph to its own function | ||
| D8827 | rDDATASET5087a463974e athena: Fix create_table to work with restricted permissions | ||
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Migrated | gitlab-migration | T2201 Indexing / mining | ||
| Migrated | gitlab-migration | T2204 Full-text search on source code (prototype) | ||
| Migrated | gitlab-migration | T2217 Plumbings | ||
| Migrated | gitlab-migration | T3096 Efficient and reliable download via the Vault | ||
| Migrated | gitlab-migration | T3550 Compute and show ETA for vault tasks | ||
| Migrated | gitlab-migration | T887 Vault: "snapshot" cooker | ||
| Migrated | gitlab-migration | T2220 swh-graph in production | ||
| Migrated | gitlab-migration | T4677 Add support for generating subdatasets in swh.dataset.luigi | ||
| Migrated | gitlab-migration | T4676 Add Luigi workflow in swh-dataset |