vlorentz renamed T1337: Add intrinsic metadata indexers to puppet config from Add intrinsic metadata indexer to puppet config to Add intrinsic metadata indexers to puppet config.

Nov 14 2018, 11:55 AM · Indexer

vlorentz created T1337: Add intrinsic metadata indexers to puppet config.

Nov 14 2018, 11:55 AM · Indexer

vlorentz added a parent task for T1324: Deploy metadata indexers in production: T1336: Deploy search over origin intrinsic metadata.

Nov 14 2018, 10:25 AM · Indexer

vlorentz added a parent task for T1231: Create an origin indexer that lists the most recent revision in HEAD branch: T1335: Add an index on origin_intrinsic_metadata.metadata.

Nov 14 2018, 10:17 AM · Indexer, Metadata workflow

Nov 13 2018

zack added a project to T1324: Deploy metadata indexers in production: Indexer.

Nov 13 2018, 2:34 PM · Indexer

ardumont added a subtask for T1227: General improvments of the indexer: Schedule indexer tasks: T1324: Deploy metadata indexers in production.

Nov 13 2018, 1:43 PM · Indexer, Scheduling utilities

ardumont removed a subtask for T1227: General improvments of the indexer: Schedule indexer tasks: T991: Indexers: Send range of ids instead of list of ids.

Nov 13 2018, 1:42 PM · Indexer, Scheduling utilities

ardumont added a subtask for T1310: Simplify indexer design: move away from the pipeline approach: T991: Indexers: Send range of ids instead of list of ids.

Nov 13 2018, 1:42 PM · Indexer, Scheduling utilities

ardumont edited parent tasks for T991: Indexers: Send range of ids instead of list of ids, added: T1310: Simplify indexer design: move away from the pipeline approach; removed: T1227: General improvments of the indexer: Schedule indexer tasks.

Nov 13 2018, 1:42 PM · Indexer

ardumont removed a parent task for T991: Indexers: Send range of ids instead of list of ids: T1326: metadata indexer: Deploy origin head.

Nov 13 2018, 1:41 PM · Indexer

ardumont removed a subtask for T1326: metadata indexer: Deploy origin head: T991: Indexers: Send range of ids instead of list of ids.

Nov 13 2018, 1:41 PM · Metadata workflow, Indexer

vlorentz triaged T1328: Add Ruby/Gem metadata indexer as Normal priority.

Nov 13 2018, 1:26 PM · Restricted Project, Indexer

vlorentz triaged T1327: Add Python metadata indexer as Normal priority.

Nov 13 2018, 1:26 PM · Restricted Project, Indexer

vlorentz closed T1326: metadata indexer: Deploy origin head as Invalid.

Duplicate of T1324

Nov 13 2018, 1:23 PM · Metadata workflow, Indexer

vlorentz claimed T1326: metadata indexer: Deploy origin head.

Nov 13 2018, 12:29 PM · Metadata workflow, Indexer

vlorentz added a parent task for T991: Indexers: Send range of ids instead of list of ids: T1326: metadata indexer: Deploy origin head.

Nov 13 2018, 12:28 PM · Indexer

vlorentz added a subtask for T1326: metadata indexer: Deploy origin head: T991: Indexers: Send range of ids instead of list of ids.

Nov 13 2018, 12:28 PM · Metadata workflow, Indexer

vlorentz triaged T1326: metadata indexer: Deploy origin head as Normal priority.

Nov 13 2018, 12:28 PM · Metadata workflow, Indexer

ardumont closed T1311: indexer: Remove orchestrators, a subtask of T1310: Simplify indexer design: move away from the pipeline approach, as Resolved.

Nov 13 2018, 12:17 PM · Indexer, Scheduling utilities

ardumont closed T1311: indexer: Remove orchestrators as Resolved.

Nov 13 2018, 12:17 PM · Indexer, Scheduling utilities

Nov 12 2018

vlorentz triaged T1316: Add support for more CodeMeta contexts URLs as Low priority.

Nov 12 2018, 4:58 PM · Indexer

vlorentz created T1316: Add support for more CodeMeta contexts URLs.

Nov 12 2018, 4:58 PM · Indexer

Nov 9 2018

ardumont claimed T1310: Simplify indexer design: move away from the pipeline approach.

Nov 9 2018, 11:38 AM · Indexer, Scheduling utilities

ardumont claimed T1311: indexer: Remove orchestrators.

Nov 9 2018, 11:37 AM · Indexer, Scheduling utilities

ardumont claimed T991: Indexers: Send range of ids instead of list of ids.

Nov 9 2018, 11:37 AM · Indexer

ardumont claimed T1227: General improvments of the indexer: Schedule indexer tasks.

Nov 9 2018, 11:36 AM · Indexer, Scheduling utilities

Nov 7 2018

ardumont triaged T1312: indexer: Adapt textual content indexer to actually filter textual content themselves as Normal priority.

Nov 7 2018, 10:12 AM · Indexer, Scheduling utilities

ardumont triaged T1311: indexer: Remove orchestrators as Normal priority.

Nov 7 2018, 10:11 AM · Indexer, Scheduling utilities

ardumont triaged T1310: Simplify indexer design: move away from the pipeline approach as Normal priority.

Nov 7 2018, 10:10 AM · Indexer, Scheduling utilities

ardumont added a comment to T1229: Indexers: Make orchestrators use swh-scheduler for scheduling.

So in the end, we will remove the orchestrator anyway.
That will simplify the indexer scheduling (which could not really happen easily in the current state).

Nov 7 2018, 10:04 AM · Indexer, Scheduling utilities

Nov 5 2018

vlorentz closed T1289: CodeMeta support in the metadata indexer as Resolved by committing rDCIDX59dca309b166: Always output valid JSON-LD..

Nov 5 2018, 5:32 PM · Indexer, Metadata workflow

Oct 30 2018

vlorentz added a revision to T1289: CodeMeta support in the metadata indexer: D620: Translate from pom.xml and codemeta.json..

Oct 30 2018, 1:25 PM · Indexer, Metadata workflow

Oct 29 2018

vlorentz added a revision to T1289: CodeMeta support in the metadata indexer: D617: Always output valid JSON-LD..

Oct 29 2018, 4:16 PM · Indexer, Metadata workflow

ardumont closed T1229: Indexers: Make orchestrators use swh-scheduler for scheduling as Resolved.

Yes, it should.

Oct 29 2018, 10:14 AM · Indexer, Scheduling utilities

ardumont closed T1229: Indexers: Make orchestrators use swh-scheduler for scheduling, a subtask of T1227: General improvments of the indexer: Schedule indexer tasks, as Resolved.

Oct 29 2018, 10:14 AM · Indexer, Scheduling utilities

vlorentz closed T1291: indexer: serialization issue on latest deployed indexer as Resolved by committing rDCIDX45c2e035a99f: Use swh.scheduler instead of celery in the OriginHeadIndexer..

Oct 29 2018, 9:55 AM · Indexer

vlorentz closed T1290: Indexers: Use swh.scheduler instead of directly relying on Celery as Resolved by committing rDCIDX45c2e035a99f: Use swh.scheduler instead of celery in the OriginHeadIndexer..

Oct 29 2018, 9:55 AM · Indexer, Scheduling utilities

vlorentz closed T1290: Indexers: Use swh.scheduler instead of directly relying on Celery, a subtask of T1229: Indexers: Make orchestrators use swh-scheduler for scheduling, as Resolved.

Oct 29 2018, 9:55 AM · Indexer, Scheduling utilities

vlorentz closed T1290: Indexers: Use swh.scheduler instead of directly relying on Celery, a subtask of T1291: indexer: serialization issue on latest deployed indexer, as Resolved.

Oct 29 2018, 9:55 AM · Indexer

vlorentz added a comment to T1229: Indexers: Make orchestrators use swh-scheduler for scheduling.

So this task will be solved by D606, right?

Oct 29 2018, 9:12 AM · Indexer, Scheduling utilities

Oct 27 2018

ardumont updated subscribers of P326 Deploy the latest indexers m.o (new v0.0.55).

Note: To deploy the new indexers, there are still work to do in another part of our stack, the puppet repository, notably the swh-site repository.
See for example:

Oct 27 2018, 11:28 AM · Indexer

ardumont updated the title for P326 Deploy the latest indexers m.o (new v0.0.55) from Deploy the latest indexers (v0.0.55) to Deploy the latest indexers m.o (new v0.0.55).

Oct 27 2018, 11:16 AM · Indexer

ardumont created P326 Deploy the latest indexers m.o (new v0.0.55).

Oct 27 2018, 11:16 AM · Indexer

Oct 26 2018

ardumont added a subtask for T1291: indexer: serialization issue on latest deployed indexer: T1290: Indexers: Use swh.scheduler instead of directly relying on Celery.

Oct 26 2018, 9:35 PM · Indexer

ardumont added a parent task for T1290: Indexers: Use swh.scheduler instead of directly relying on Celery: T1291: indexer: serialization issue on latest deployed indexer.

Oct 26 2018, 9:35 PM · Indexer, Scheduling utilities

ardumont renamed T1229: Indexers: Make orchestrators use swh-scheduler for scheduling from Indexers: Analysis further the possiblity to remove the orchestrator layer to the benefit of the scheduler to Indexers: Make orchestrators use swh-scheduler for scheduling.

Oct 26 2018, 8:05 PM · Indexer, Scheduling utilities

ardumont added a comment to T1291: indexer: serialization issue on latest deployed indexer.

\m/ <- sounds good ;)

Oct 26 2018, 11:11 AM · Indexer

vlorentz added a revision to T1291: indexer: serialization issue on latest deployed indexer: D608: Use swh.scheduler instead of celery in the OriginHeadIndexer..

Oct 26 2018, 10:43 AM · Indexer

vlorentz added a revision to T1290: Indexers: Use swh.scheduler instead of directly relying on Celery: D608: Use swh.scheduler instead of celery in the OriginHeadIndexer..

Oct 26 2018, 10:43 AM · Indexer, Scheduling utilities

vlorentz added a comment to T1291: indexer: serialization issue on latest deployed indexer.

Alternatively, you can wait for me to fix T1290 (I'll do it this morning). My plan is to remove the need for result serialization.

Oct 26 2018, 9:17 AM · Indexer

ardumont added a comment to T1291: indexer: serialization issue on latest deployed indexer.

I'm missing something somewhere though.

Oct 26 2018, 9:13 AM · Indexer

Oct 25 2018

ardumont closed T1293: indexer: Configuration setup missing from orchestrator as Resolved by committing rDCIDXee54c5ad3523: swh.indexer.orchestrator: Fix orchestrator initialization step.

Oct 25 2018, 11:00 PM · Indexer

ardumont updated the task description for T1292: indexer: Issue on latest task changes.

Oct 25 2018, 10:15 PM · Indexer

ardumont added a comment to T1291: indexer: serialization issue on latest deployed indexer.

This was fixed

Oct 25 2018, 10:08 PM · Indexer

ardumont closed T1294: indexer: Task must inherit from swh.scheduler.tasks as Resolved.

Oct 25 2018, 10:06 PM · Indexer

ardumont added a revision to T1293: indexer: Configuration setup missing from orchestrator: D607: swh.indexer.orchestrator: Fix missing init step.

Oct 25 2018, 10:04 PM · Indexer

ardumont added a comment to T1294: indexer: Task must inherit from swh.scheduler.tasks.

Fixed in 4b44f3af3e0d300ee15719c94946bad2279e7732

Oct 25 2018, 10:00 PM · Indexer

ardumont triaged T1294: indexer: Task must inherit from swh.scheduler.tasks as Normal priority.

Oct 25 2018, 10:00 PM · Indexer

ardumont triaged T1293: indexer: Configuration setup missing from orchestrator as Normal priority.

Oct 25 2018, 9:56 PM · Indexer

vlorentz added a comment to T1291: indexer: serialization issue on latest deployed indexer.

This was fixed by D600

Oct 25 2018, 9:50 PM · Indexer

ardumont closed T1292: indexer: Issue on latest task changes as Resolved.

Oct 25 2018, 9:45 PM · Indexer

ardumont added a comment to T1292: indexer: Issue on latest task changes.

Fixed in 2a926fc4aab51d9c6e41eb62c7dce97aa8d62b89

Oct 25 2018, 9:44 PM · Indexer

ardumont triaged T1292: indexer: Issue on latest task changes as Normal priority.

Oct 25 2018, 9:44 PM · Indexer

ardumont updated the task description for T1291: indexer: serialization issue on latest deployed indexer.

Oct 25 2018, 9:42 PM · Indexer

ardumont updated the task description for T1291: indexer: serialization issue on latest deployed indexer.

Oct 25 2018, 9:41 PM · Indexer

ardumont triaged T1291: indexer: serialization issue on latest deployed indexer as Normal priority.

Oct 25 2018, 9:37 PM · Indexer

vlorentz placed T1229: Indexers: Make orchestrators use swh-scheduler for scheduling up for grabs.

Oct 25 2018, 3:58 PM · Indexer, Scheduling utilities

vlorentz renamed T1290: Indexers: Use swh.scheduler instead of directly relying on Celery from Use swh.scheduler instead of directly relying on Celery to Indexers: Use swh.scheduler instead of directly relying on Celery.

Oct 25 2018, 3:57 PM · Indexer, Scheduling utilities

vlorentz triaged T1290: Indexers: Use swh.scheduler instead of directly relying on Celery as Normal priority.

Oct 25 2018, 3:57 PM · Indexer, Scheduling utilities

ardumont added a comment to T1229: Indexers: Make orchestrators use swh-scheduler for scheduling.

Indexers can be made configurable as well

Oct 25 2018, 3:52 PM · Indexer, Scheduling utilities

vlorentz closed T1236: Refactor metadata translator to parse different types of files as Resolved.

Resolved by D591.

Oct 25 2018, 2:23 PM · Metadata workflow, Indexer

vlorentz closed T1236: Refactor metadata translator to parse different types of files, a subtask of T1235: Refactor metadata detector tool to add easily new file names to detect, as Resolved.

Oct 25 2018, 2:23 PM · Metadata workflow, Indexer

vlorentz closed T1235: Refactor metadata detector tool to add easily new file names to detect as Resolved by committing rDCIDX86695b3f6464: Make mappings into a hierarchy of classes that can be easily extended..

Oct 25 2018, 2:22 PM · Metadata workflow, Indexer

vlorentz closed T1235: Refactor metadata detector tool to add easily new file names to detect, a subtask of T1289: CodeMeta support in the metadata indexer, as Resolved.

Oct 25 2018, 2:22 PM · Indexer, Metadata workflow

moranegg added a comment to T1237: Update mappings to CodeMeta crosswalk table .

Noted Invalid because D591 by @vlorentz uses the most up-to-date mappings with the CodeMeta crosswalk table.

Oct 25 2018, 1:55 PM · Metadata workflow, Indexer

moranegg closed T1237: Update mappings to CodeMeta crosswalk table , a subtask of T1236: Refactor metadata translator to parse different types of files, as Invalid.

Oct 25 2018, 1:52 PM · Metadata workflow, Indexer

moranegg closed T1237: Update mappings to CodeMeta crosswalk table as Invalid.

Oct 25 2018, 1:52 PM · Metadata workflow, Indexer

vlorentz claimed T1289: CodeMeta support in the metadata indexer.

Oct 25 2018, 12:20 PM · Indexer, Metadata workflow

vlorentz added a parent task for T1289: CodeMeta support in the metadata indexer: T1232: Search over intrinsic metadata associated to an origin.

Oct 25 2018, 12:19 PM · Indexer, Metadata workflow

vlorentz added a subtask for T1289: CodeMeta support in the metadata indexer: T1235: Refactor metadata detector tool to add easily new file names to detect.

Oct 25 2018, 12:19 PM · Indexer, Metadata workflow

vlorentz added a parent task for T1235: Refactor metadata detector tool to add easily new file names to detect: T1289: CodeMeta support in the metadata indexer.

Oct 25 2018, 12:19 PM · Metadata workflow, Indexer

vlorentz triaged T1289: CodeMeta support in the metadata indexer as Normal priority.

Oct 25 2018, 12:18 PM · Indexer, Metadata workflow

Oct 24 2018

vlorentz added a revision to T1235: Refactor metadata detector tool to add easily new file names to detect: D591: Make mappings into a hierarchy of classes that can be easily extended..

Oct 24 2018, 6:39 PM · Metadata workflow, Indexer

vlorentz claimed T1235: Refactor metadata detector tool to add easily new file names to detect.

Oct 24 2018, 6:39 PM · Metadata workflow, Indexer

vlorentz closed T1228: Create a component that sends a list of revision sha1 to the metadataRevisionIndexer as Resolved.

Resolved by D558.

Oct 24 2018, 3:12 PM · Indexer, Metadata workflow

vlorentz added a comment to T1229: Indexers: Make orchestrators use swh-scheduler for scheduling.

Indexers can be made configurable as well

Oct 24 2018, 2:30 PM · Indexer, Scheduling utilities

ardumont added a comment to T1229: Indexers: Make orchestrators use swh-scheduler for scheduling.

My plan is to move the logic from the orchestrator into the indexers themselves

Oct 24 2018, 1:26 PM · Indexer, Scheduling utilities

vlorentz added a comment to T1229: Indexers: Make orchestrators use swh-scheduler for scheduling.

My plan is to move the logic from the orchestrator into the indexers themselves

Oct 24 2018, 12:27 PM · Indexer, Scheduling utilities

ardumont updated subscribers of T1229: Indexers: Make orchestrators use swh-scheduler for scheduling.

@vlorentz @olasd Beware though, the orchestrator is supposed to per indexer (as in its configuration, e.g. P235):

Oct 24 2018, 12:02 PM · Indexer, Scheduling utilities

Oct 23 2018

vlorentz claimed T1236: Refactor metadata translator to parse different types of files.

Oct 23 2018, 12:22 PM · Metadata workflow, Indexer

moranegg added a comment to T1237: Update mappings to CodeMeta crosswalk table .

This task was a temporary task to manually review the CodeMeta mapping, which I forgot to do but I can do it rapidly.
I think we should open a new task for you or use T1236 which have a more technical aspect of how to use the CodeMeta crosswalk table.

Oct 23 2018, 11:46 AM · Metadata workflow, Indexer

Oct 22 2018

vlorentz claimed T1229: Indexers: Make orchestrators use swh-scheduler for scheduling.

Oct 22 2018, 2:05 PM · Indexer, Scheduling utilities

Advanced SearchUse ResultsEdit QueryHide Query

Nov 20 2018

Nov 19 2018

Nov 14 2018

Nov 13 2018

Nov 12 2018

Nov 9 2018

Nov 7 2018

Nov 5 2018

Oct 30 2018

Oct 29 2018

Oct 27 2018

Oct 26 2018

Oct 25 2018

Oct 24 2018

Oct 23 2018

Oct 22 2018

Advanced Search
Use Results
Edit Query
Hide Query