loader plumbings used by other loaders
Details
Details
Description
Tue, Feb 19
Tue, Feb 19
Jan 8 2019
Jan 8 2019
Dec 13 2018
Dec 13 2018
Dec 5 2018
Dec 5 2018
Resolved by D770.
Dec 4 2018
Dec 4 2018
Nov 27 2018
Nov 27 2018
Oct 31 2018
Oct 31 2018
vlorentz renamed T1300: Release swh-loader-core v0.0.36 from Release swh-loader-core v0.0.35 to Release swh-loader-core v0.0.36.
Oct 10 2018
Oct 10 2018
Oct 4 2018
Oct 4 2018
zack closed T336: "save code now", a subtask of T1022: SWORD deposit requesting to save content existing on an external code hosting platform, as Resolved.
zack renamed T1238: Loader: refactor loader tests to avoid code duplication from Loader: Refactor the loader tests to reuse to Loader: refactor loader tests to avoid code duplication.
Oct 3 2018
Oct 3 2018
Jul 23 2018
Jul 23 2018
moranegg closed T960: draft specs for deposit with incomplete tarball , a subtask of T1021: SWORD deposit of metadata about an existing SWH object, as Resolved.
Jun 27 2018
Jun 27 2018
zack triaged T1022: SWORD deposit requesting to save content existing on an external code hosting platform as Normal priority.
Apr 11 2018
Apr 11 2018
rdicosmo updated the task description for T1022: SWORD deposit requesting to save content existing on an external code hosting platform.
Mar 9 2018
Mar 9 2018
Well, after much digging documentations and some tryouts. Finally gave in to solution 3.
Mar 7 2018
Mar 7 2018
ardumont added a comment to T982: failing worker consumes remaining tasks without processing them.
|---------------------------------+---------------------------------------+---------------------------------+------------------------------------| | Possible solution | Pros | Cons | Question | |---------------------------------+---------------------------------------+---------------------------------+------------------------------------| | 8. Celery signal on postrun | - separation of concern | Not working, in OOM kill | Documentation [3] | | | | scenario, task is killed | | | | | (tested) | | |---------------------------------+---------------------------------------+---------------------------------+------------------------------------|
Mar 5 2018
Mar 5 2018
ardumont added a comment to T982: failing worker consumes remaining tasks without processing them.
Still asserting the possibilities (and reading documentation):
ardumont added a comment to T982: failing worker consumes remaining tasks without processing them.
So far, the solutions i foresee:
ardumont triaged T982: failing worker consumes remaining tasks without processing them as Normal priority.
Jan 31 2018
Jan 31 2018
Dec 7 2017
Dec 7 2017
ardumont closed T871: Migrate swh-storage api functions relative to indexers to swh-indexer as Resolved.
ardumont closed T872: Deploy and restart indexers, a subtask of T871: Migrate swh-storage api functions relative to indexers to swh-indexer, as Resolved.
ardumont updated the task description for T872: Deploy and restart indexers.
ardumont updated the task description for T872: Deploy and restart indexers.
ardumont updated the task description for T872: Deploy and restart indexers.
ardumont changed the status of T872: Deploy and restart indexers, a subtask of T871: Migrate swh-storage api functions relative to indexers to swh-indexer, from Open to Work in Progress.
ardumont updated the task description for T872: Deploy and restart indexers.
ardumont updated the task description for T871: Migrate swh-storage api functions relative to indexers to swh-indexer.
ardumont updated the task description for T871: Migrate swh-storage api functions relative to indexers to swh-indexer.
ardumont updated the task description for T871: Migrate swh-storage api functions relative to indexers to swh-indexer.
ardumont updated the task description for T871: Migrate swh-storage api functions relative to indexers to swh-indexer.
ardumont updated the task description for T871: Migrate swh-storage api functions relative to indexers to swh-indexer.
Oct 30 2017
Oct 30 2017
ardumont added a comment to T804: swh-loader-core: doesn't properly batch object insertions (for big requests).
The size limit should superimpose itself to the number of items limit, but the behavior should be the same.
olasd added a comment to T804: swh-loader-core: doesn't properly batch object insertions (for big requests).
I'm going to try to clarify my point:
Oct 24 2017
Oct 24 2017
ardumont added a comment to T804: swh-loader-core: doesn't properly batch object insertions (for big requests).
The core loader uses a queue system to bundle loading of objects.
Oct 13 2017
Oct 13 2017