Feed Advanced Search

Advanced Search
Use Results
Edit Query
Hide Query

Jan 8 2023

gitlab-migration changed the status of T347: ingest antelink s3 contents in swh, a subtask of T322: Inject antepedia contents (backuped in sesi and not in swh) into swh, from Resolved to Migrated.

Jan 8 2023, 9:56 PM · Antelink loader

gitlab-migration changed the status of T308: retrieve content from s3 and store it in SWH storage from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:56 PM · Storage manager, Antelink loader

gitlab-migration closed T310: import antelink metadata as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:59 PM · Antelink loader

gitlab-migration changed the status of T356: s3 files injected in swh distribution per first char from Invalid to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T350: Backup downloaded s3 files to sesi's antelink store0 mount point, a subtask of T309: Delete duplicated antelink/antepedia content from s3, from Resolved to Migrated.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T350: Backup downloaded s3 files to sesi's antelink store0 mount point from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T320: sesi content downloader and injection in swh worker, a subtask of T322: Inject antepedia contents (backuped in sesi and not in swh) into swh, from Resolved to Migrated.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T322: Inject antepedia contents (backuped in sesi and not in swh) into swh from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T320: sesi content downloader and injection in swh worker from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T321: Download antepedia's s3 contents not in sesi nor in swh from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T322: Inject antepedia contents (backuped in sesi and not in swh) into swh, a subtask of T309: Delete duplicated antelink/antepedia content from s3, from Resolved to Migrated.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T317: Inject sesi files hashes in antelink db from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T319: S3 content files downloader and injection in swh, a subtask of T321: Download antepedia's s3 contents not in sesi nor in swh, from Resolved to Migrated.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T319: S3 content files downloader and injection in swh from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T317: Inject sesi files hashes in antelink db, a subtask of T321: Download antepedia's s3 contents not in sesi nor in swh, from Resolved to Migrated.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T317: Inject sesi files hashes in antelink db, a subtask of T322: Inject antepedia contents (backuped in sesi and not in swh) into swh, from Resolved to Migrated.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T316: List and compute hashes of actual sesi files, a subtask of T317: Inject sesi files hashes in antelink db, from Resolved to Migrated.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T316: List and compute hashes of actual sesi files from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Antelink loader

gitlab-migration changed the status of T309: Delete duplicated antelink/antepedia content from s3 from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:18 PM · Antelink loader

Nov 18 2016

ardumont updated the task description for T317: Inject sesi files hashes in antelink db.

Nov 18 2016, 2:10 PM · Antelink loader

May 13 2016

olasd changed the visibility for Antelink loader.

May 13 2016, 5:23 PM

olasd changed the visibility for T356: s3 files injected in swh distribution per first char.

May 13 2016, 5:09 PM · Antelink loader

olasd changed the visibility for T350: Backup downloaded s3 files to sesi's antelink store0 mount point.

May 13 2016, 5:09 PM · Antelink loader

olasd changed the visibility for T321: Download antepedia's s3 contents not in sesi nor in swh.

May 13 2016, 5:09 PM · Antelink loader

olasd changed the visibility for T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

May 13 2016, 5:09 PM · Antelink loader

olasd changed the visibility for T320: sesi content downloader and injection in swh worker.

May 13 2016, 5:09 PM · Antelink loader

olasd changed the visibility for T319: S3 content files downloader and injection in swh.

May 13 2016, 5:09 PM · Antelink loader

olasd changed the visibility for T317: Inject sesi files hashes in antelink db.

May 13 2016, 5:08 PM · Antelink loader

olasd changed the visibility for T316: List and compute hashes of actual sesi files.

May 13 2016, 5:08 PM · Antelink loader

olasd changed the visibility for T310: import antelink metadata.

May 13 2016, 5:08 PM · Antelink loader

olasd changed the visibility for T309: Delete duplicated antelink/antepedia content from s3.

May 13 2016, 5:08 PM · Antelink loader

olasd changed the visibility for T308: retrieve content from s3 and store it in SWH storage.

May 13 2016, 5:08 PM · Storage manager, Antelink loader

May 11 2016

ardumont closed T308: retrieve content from s3 and store it in SWH storage as Resolved.

May 11 2016, 1:51 PM · Storage manager, Antelink loader

Apr 27 2016

zack renamed T310: import antelink metadata from Inject metadata on antelink's content to import antelink metadata.

Apr 27 2016, 9:11 PM · Antelink loader

zack assigned T308: retrieve content from s3 and store it in SWH storage to ardumont.

Apr 27 2016, 9:02 PM · Storage manager, Antelink loader

zack renamed T308: retrieve content from s3 and store it in SWH storage from Retrieve content from s3 and store inside swh-storage to retrieve content from s3 and store it in SWH storage.

Apr 27 2016, 9:01 PM · Storage manager, Antelink loader

Apr 15 2016

ardumont closed T322: Inject antepedia contents (backuped in sesi and not in swh) into swh, a subtask of T309: Delete duplicated antelink/antepedia content from s3, as Resolved.

Apr 15 2016, 3:39 PM · Antelink loader

ardumont closed T322: Inject antepedia contents (backuped in sesi and not in swh) into swh as Resolved.

Apr 15 2016, 3:39 PM · Antelink loader

ardumont closed T309: Delete duplicated antelink/antepedia content from s3 as Resolved.

Apr 15 2016, 3:38 PM · Antelink loader

ardumont added a comment to T309: Delete duplicated antelink/antepedia content from s3.

s3 backup has been removed by Guillaume

Apr 15 2016, 3:38 PM · Antelink loader

ardumont closed T347: ingest antelink s3 contents in swh, a subtask of T322: Inject antepedia contents (backuped in sesi and not in swh) into swh, as Resolved.

Apr 15 2016, 1:30 PM · Antelink loader

ardumont changed the status of T319: S3 content files downloader and injection in swh from Duplicate to Resolved.

Apr 15 2016, 1:28 PM · Antelink loader

ardumont changed the status of T319: S3 content files downloader and injection in swh, a subtask of T321: Download antepedia's s3 contents not in sesi nor in swh, from Duplicate to Resolved.

Apr 15 2016, 1:28 PM · Antelink loader

Apr 7 2016

ardumont added a subtask for T322: Inject antepedia contents (backuped in sesi and not in swh) into swh: T347: ingest antelink s3 contents in swh.

Apr 7 2016, 11:56 AM · Antelink loader

Mar 30 2016

ardumont renamed T320: sesi content downloader and injection in swh worker from sesi content downloader and injection in swh to sesi content downloader and injection in swh worker.

Mar 30 2016, 1:54 PM · Antelink loader

ardumont closed T350: Backup downloaded s3 files to sesi's antelink store0 mount point, a subtask of T309: Delete duplicated antelink/antepedia content from s3, as Resolved.

Mar 30 2016, 1:46 PM · Antelink loader

ardumont closed T350: Backup downloaded s3 files to sesi's antelink store0 mount point as Resolved.

Mar 30 2016, 1:46 PM · Antelink loader

Mar 29 2016

ardumont closed T356: s3 files injected in swh distribution per first char as Invalid.

Mar 29 2016, 2:32 PM · Antelink loader

ardumont created T356: s3 files injected in swh distribution per first char.

Mar 29 2016, 2:28 PM · Antelink loader

Mar 14 2016

ardumont added a parent task for T322: Inject antepedia contents (backuped in sesi and not in swh) into swh: T309: Delete duplicated antelink/antepedia content from s3.

Mar 14 2016, 3:10 PM · Antelink loader

ardumont added subtasks for T309: Delete duplicated antelink/antepedia content from s3: T322: Inject antepedia contents (backuped in sesi and not in swh) into swh, T350: Backup downloaded s3 files to sesi's antelink store0 mount point.

Mar 14 2016, 3:10 PM · Antelink loader

ardumont added a parent task for T350: Backup downloaded s3 files to sesi's antelink store0 mount point: T309: Delete duplicated antelink/antepedia content from s3.

Mar 14 2016, 3:10 PM · Antelink loader

ardumont changed the status of T350: Backup downloaded s3 files to sesi's antelink store0 mount point from Open to Work in Progress.

Mar 14 2016, 2:41 PM · Antelink loader

ardumont updated the task description for T350: Backup downloaded s3 files to sesi's antelink store0 mount point.

Mar 14 2016, 12:30 PM · Antelink loader

ardumont created T350: Backup downloaded s3 files to sesi's antelink store0 mount point.

Mar 14 2016, 12:22 PM · Antelink loader

ardumont closed T321: Download antepedia's s3 contents not in sesi nor in swh as Resolved.

Mar 14 2016, 12:14 PM · Antelink loader

ardumont added a comment to T321: Download antepedia's s3 contents not in sesi nor in swh.

Check is currently running to determine if some files are still missing.

Mar 14 2016, 12:14 PM · Antelink loader

Mar 11 2016

ardumont added a comment to T321: Download antepedia's s3 contents not in sesi nor in swh.

Remaining 2M done.
Now we should be complete.

Mar 11 2016, 12:27 PM · Antelink loader

Mar 10 2016

zack removed a project from T316: List and compute hashes of actual sesi files: Developers.

Mar 10 2016, 5:54 PM · Antelink loader

zack removed a project from T309: Delete duplicated antelink/antepedia content from s3: Developers.

Mar 10 2016, 5:54 PM · Antelink loader

zack removed a project from T308: retrieve content from s3 and store it in SWH storage: Developers.

Mar 10 2016, 5:54 PM · Storage manager, Antelink loader

zack removed a project from T319: S3 content files downloader and injection in swh: Developers.

Mar 10 2016, 5:54 PM · Antelink loader

zack removed a project from T317: Inject sesi files hashes in antelink db: Developers.

Mar 10 2016, 5:54 PM · Antelink loader

zack removed a project from T320: sesi content downloader and injection in swh worker: Developers.

Mar 10 2016, 5:54 PM · Antelink loader

zack removed a project from T321: Download antepedia's s3 contents not in sesi nor in swh: Developers.

Mar 10 2016, 5:54 PM · Antelink loader

zack removed a project from T322: Inject antepedia contents (backuped in sesi and not in swh) into swh: Developers.

Mar 10 2016, 5:54 PM · Antelink loader

zack removed a project from T310: import antelink metadata: Staff.

Mar 10 2016, 5:51 PM · Antelink loader

zack removed a project from T308: retrieve content from s3 and store it in SWH storage: Staff.

Mar 10 2016, 5:49 PM · Storage manager, Antelink loader

zack removed a project from T309: Delete duplicated antelink/antepedia content from s3: Staff.

Mar 10 2016, 5:49 PM · Antelink loader

zack removed a project from T316: List and compute hashes of actual sesi files: Staff.

Mar 10 2016, 5:49 PM · Antelink loader

zack removed a project from T317: Inject sesi files hashes in antelink db: Staff.

Mar 10 2016, 5:49 PM · Antelink loader

zack removed a project from T319: S3 content files downloader and injection in swh: Staff.

Mar 10 2016, 5:49 PM · Antelink loader

zack removed a project from T321: Download antepedia's s3 contents not in sesi nor in swh: Staff.

Mar 10 2016, 5:49 PM · Antelink loader

zack removed a project from T320: sesi content downloader and injection in swh worker: Staff.

Mar 10 2016, 5:49 PM · Antelink loader

zack removed a project from T322: Inject antepedia contents (backuped in sesi and not in swh) into swh: Staff.

Mar 10 2016, 5:49 PM · Antelink loader

Mar 8 2016

ardumont renamed T322: Inject antepedia contents (backuped in sesi and not in swh) into swh from Download antepedia contents (backuped in sesi) not in swh and injects in swh to Inject antepedia contents (backuped in sesi and not in swh) into swh.

Mar 8 2016, 7:31 PM · Antelink loader

ardumont added a comment to T321: Download antepedia's s3 contents not in sesi nor in swh.

As of yesterday, 10M were done.

Mar 8 2016, 7:28 PM · Antelink loader

ardumont added a comment to T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

Multiple problems:

slow
ssh connection problems when too many concurrent jobs (either with ssh multiplex or with simple scp).

Mar 8 2016, 12:06 PM · Antelink loader

Mar 7 2016

ardumont added a comment to T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

The files mentioned are only the ones with size <= 100Mib for now.

Mar 7 2016, 11:03 AM · Antelink loader

ardumont added a comment to T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

worker01 is producing messages in the swh_antelink_sesi_download queue.
Each message represents either roughly 100Mib of compressed files or 1000 compressed files to scp.

Mar 7 2016, 10:59 AM · Antelink loader

ardumont renamed T322: Inject antepedia contents (backuped in sesi and not in swh) into swh from Download antepedia's sesi contents not in swh to Download antepedia contents (backuped in sesi) not in swh and injects in swh.

Mar 7 2016, 10:54 AM · Antelink loader

Feb 27 2016

ardumont added a comment to T321: Download antepedia's s3 contents not in sesi nor in swh.

worker01 is in charge

10 jobs working concurrently
1 job is downloading sequentially 1000 s3 files

Feb 27 2016, 8:10 PM · Antelink loader

ardumont claimed T321: Download antepedia's s3 contents not in sesi nor in swh.

Feb 27 2016, 8:04 PM · Antelink loader

ardumont changed the status of T321: Download antepedia's s3 contents not in sesi nor in swh from Open to Work in Progress.

Feb 27 2016, 8:04 PM · Antelink loader

Feb 23 2016

ardumont closed T317: Inject sesi files hashes in antelink db, a subtask of T322: Inject antepedia contents (backuped in sesi and not in swh) into swh, as Resolved.

Feb 23 2016, 7:14 PM · Antelink loader

ardumont closed T317: Inject sesi files hashes in antelink db as Resolved.

Feb 23 2016, 7:14 PM · Antelink loader

ardumont closed T317: Inject sesi files hashes in antelink db, a subtask of T321: Download antepedia's s3 contents not in sesi nor in swh, as Resolved.

Feb 23 2016, 7:14 PM · Antelink loader

ardumont updated the task description for T317: Inject sesi files hashes in antelink db.

Feb 23 2016, 6:36 PM · Antelink loader

ardumont closed T316: List and compute hashes of actual sesi files as Resolved.

Feb 23 2016, 6:35 PM · Antelink loader

ardumont closed T316: List and compute hashes of actual sesi files, a subtask of T317: Inject sesi files hashes in antelink db, as Resolved.

Feb 23 2016, 6:35 PM · Antelink loader

ardumont merged task T319: S3 content files downloader and injection in swh into T308: retrieve content from s3 and store it in SWH storage.

Feb 23 2016, 4:12 PM · Antelink loader

ardumont merged T319: S3 content files downloader and injection in swh into T308: retrieve content from s3 and store it in SWH storage.

Feb 23 2016, 4:12 PM · Storage manager, Antelink loader

ardumont added a parent task for T320: sesi content downloader and injection in swh worker: T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

Feb 23 2016, 4:10 PM · Antelink loader

ardumont added a parent task for T317: Inject sesi files hashes in antelink db: T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

Feb 23 2016, 4:10 PM · Antelink loader

ardumont added subtasks for T322: Inject antepedia contents (backuped in sesi and not in swh) into swh: T317: Inject sesi files hashes in antelink db, T320: sesi content downloader and injection in swh worker.

Feb 23 2016, 4:10 PM · Antelink loader

ardumont created T322: Inject antepedia contents (backuped in sesi and not in swh) into swh.

Feb 23 2016, 4:10 PM · Antelink loader

ardumont added subtasks for T321: Download antepedia's s3 contents not in sesi nor in swh: T317: Inject sesi files hashes in antelink db, T319: S3 content files downloader and injection in swh.

Feb 23 2016, 4:09 PM · Antelink loader

ardumont added a parent task for T317: Inject sesi files hashes in antelink db: T321: Download antepedia's s3 contents not in sesi nor in swh.

Feb 23 2016, 4:09 PM · Antelink loader

ardumont created T321: Download antepedia's s3 contents not in sesi nor in swh.

Feb 23 2016, 4:08 PM · Antelink loader

ardumont closed T320: sesi content downloader and injection in swh worker as Resolved.

Feb 23 2016, 4:07 PM · Antelink loader