Rebase

Rebase

LGTM!

HEAD is not required anymore

Update default branch mechanism

Does this work yet?

Incorporate latest suggestions

Incorporate suggestions by olasd

Add documentation and sort heads

Rebase and adjust the naming scheme

I think @marmoute's intention was to more closely convey the semantics of Mercurial's branching system. A branch tip or head are not a branches themselves, so it would be "wrong" to put them under branches/. Thus, since the plural of "a branch head" is "branch heads" I don't feel like the second change would be appropriate either.

Here's what I gather to be the most up-to-date version:

In D5815#148012, @olasd wrote:

Hum, looks like you squashed the changes related to the new branch structure with this the change described in the commit message. Would it be possible to split them off? If not, then the commit message needs to actually describe the change in branch structure, rather than the incidental change to the data generation script.

Remove unrelated changes

I have implemented @marmoute's version of the mapping at D5816. Since the exact naming scheme is just one search-and-replace away, we can still change it easily. Implementing this has highlighted a flaw in the handling of multiple open heads, which is now fixed.

Agreed! That would look like:

The run from this week-end, detailed in T3336, appears to have worked fine. (just making sure it's obvious from this task also)

Great news! Let me know if I can help in any way.

For posterity, I have tested that all corrupted and "verify failed" repositories in the archive load correctly, as well as the humongous Mozilla-unified, PyPy and about a few thousand random other ones from the archive. Aside from the incremental loading issues detailed in T3336 (that should be fixed in today's run), everything seems fine.

Batch RPC calls

D5793 should fix the remaining issue. We had a discussion with @marmoute about whether considering closed branches (done in D5790) is *actually* a good idea in terms of presentation, but D5793 fixes the underlying issue, so we'll see about this issue next week.

Refactor expected stats in new test

In D5790#147332, @ardumont wrote:

I guess it's some kind of read-only branch. Once marked
closed by the hg commit cli, we can't push any more
on it, right?

(Hopefully) Addressed in D5785

Sorry for the delayed response. I'm assuming we'd like it better if the lister continued anyway in case of a "fatal" connection error, with maybe some sort of retry?

The mapping file is located (on the boatbucket machine) at /srv/boatbucket/mapping-to-repos.txt. It does *not* contain the (very few) outright corrupted repositories, I might have to do some digging and even bother the BB team again to get the URL for those.

Rebase

In D5740#146045, @vlorentz wrote:

When this happens, will the page be visited again by the incremental lister?

More thorough testing

Include suggestions

I haven't gotten around to doing the changes yet, will do so shortly.

Rebase

Remove pytest-postgresql hotfix

Rebase + remove useless loader call in test

Better fix for environment manipulation