What's next, as a summary, subsequent subtasks should be created later:

• Refactor a bit the journal client to update a docstring and inline one function (done, that'd be the 2 previous commits mentioned here just below that comment ^).
• Deactivate failing visits (delegating to listers the act of activating back those origins which gets live again). I have diffs which deal with this that needs some rebase and work according to latest change (I need to get back to it) [1].
• Deploy the current scheduler implementation (master) when that previous point is done. (That's gonna be my goal to reach prior to some vacation break).

including the next-gen scheduler runner not yet puppetized [4]

All got done except this part ^.
This needs first the following:

• D5809 to be rebased on latest master branch (v0.17)
• the saatchi venv (in swhscheduler home) to be updated with it

including the next-gen scheduler runner not yet puppetized [4]

Following actions in order:

Ensure the journal client is doing its new job

Deactivate failing visits (delegating to listers the act of activating back those
origins which gets live again). I have diffs which deal with this that needs some
rebase and work according to latest change (I need to get back to it) [1].

(^ for a while ;)

Status on this, after the recent refactoring we did with @olasd to simplify the actual
implementation (backend and journal client). There remains to:

The 'hg' ingestion started now that the latest loader mercurial got deployed.

The 'git' ingestion caught up [1] so now let's make the svn origins finish [2]. In
effect, making the loader run as before. Activating back the svn queue consumption where
it remains few origins to consume.

So, to improve the current situation, the svn ingestion got put in stand-by to let the
git ingestion progress as well.

So a bit of status report.

if it would be worth submitting these recursive origins with "save code now" so we can try to get submodule updates close to the update of the main repository

I also wonder if we have a somewhat common approach to handle the SVN externals as well.

I think this is worthwhile in general, at least for repositories that are still live.

Updated stats in descending order on the no_last_update column:

Relatedly to this task, some work has been started to make the pypi lister list its
origins with the last_update information in the diff D5977 / T3399 (review got done
and the implementation needs to be improved but still ;).

Status on the latest development for this task, "Baseline for the recurrence of origin
visits" chapter has been implemented in the following stacked diffs (in review):

Updates:

• A ticket has been submitted in the Sonatype JIRA to let them know we will fetch maven poms and src jars soon.
• An email has been sent on the maven-dev mailing list with a few kind answers, mainly stating to let Sonatype know through a JIRA issue.
• Hervé Bouthemy provided some precious insights about the best way to use the poms; it seems we can get a near-complete list of maven repositories worldwide by parsing some pom arguments and following dependencies up. It should probably not be used directly by the lister (which should provide only the list of src jars and scm attributes to the loaders), but we can output it somewhere to feed the lister manually.

Summary of the data available in the listed_origins table, broken down by lister and "known state" of origins: