That's been done for a while now.

Just to be sure I'm getting your proposal right: this will ensure that the normalization is done aligning with what a checkout will do (rather than the other way around), right?
If so, I agree this is definitely the way to go, as we will be guaranteeing that we archive what users would get out of a (badly stored) SVN revision.

In T570#22071, @anlambert wrote:

Based on my understanding, this property does not seem to be taken into account
when using the remote access replay api and thus the reconstructed files may
contain different line endings as those generated by 'svn export', so there is a
tree divergence.

I took some time to dig into that issue last week and this what I understood of it.

I took some time to dig into that issue last week and this what I understood of it.

good catch !

• Restored in time.
• Fix the input dumps to mount in order
• Restarted the dump mounting routine

Well, the dump is all messed up to be polite now...

Well, the dump is all messed up to be polite now...
Restoring to an old point in time.

Need to notify our asf contact but i'll make sure there are no other holes first.

Well, yeah, that file is missing.

Well, nothing too serious. Wrong even possibly missing file dump!

Status on this, everything were fine up until today.

I don't see anything new here.

I don't see anything new here. Subversion offers no integrity guarantees, it applies to the ASF repos like it applies to any other SVN repo out there. We need to decide a policy about when (if at all), re-do full ingestions of Subversion repos (which will allow to re-inject modified objects at the cost of forking the resulting history on Software Heritage) or just say *shrug* and never re-ingest in a non-incremental way any Subversion repo we have previously ingested.

During our latest exchange with our asf contact (Greg Stein), i ask about history modification and here is his answer:

There, asf has the same inconsistency error, which is now detected:

Update on this. As the tested dumps so far were going well.
I have automated the remaining dumps to mount.
It's currently running.

Of course i forgot to mention (because i only remember it now), it's not only CRLF, it's sometimes a mix with CR/CRLF/LF as well...

Ok digging further, all repositories are corrupted, or at least not consistent.
And this inconsistency is unfortunately permitted in the svn toolchain (at least, it was at some point).

So far, other dumps with the error. All of those errors have the same reason, a mix of CRLF lines terminator where export does not expect that:

This issue is reproduced on 6 repositories (so far, still running locally on some other big repositories).
Those fails are commits holding non-unicode named characters in tree or filename (japanese for example).
What's failing is not clear at all though.

It appears that in this case, the properties must be changed not to the symlink but to its source.

It's more empty repository case than a repository starting its commit range at 0...

This is in stand-by during the snapshot migration.

The 1st suggestion is currently tested and so far so good (more than 700k revision has been done so far).

Repeating my initial comment.

Well, trying to mount the repository on the side seems to be a task on its own:

Those dumps are currently being retrieved in uffizi:/srv/storage/space/mirrors/asf.

Even svn's own tools break on such cases (svnsync must be iteratively called to continue).

origins-to-cleanup file: /srv/storage/space/lists/svn/INDEX-svn-dumps-to-cleanup

P202 checked and ok locally.
Now asked for review as it will remove data from the main db.

After discussion with the team, it has been decided to remove from the re-scheduling the svn dumps whose compressed size exceeds 2Gib.
This reflects the same decision took for git repositories.

T896 will help.

Reopened as new missing children task was created.

Scheduled back from saatchi (as i needed the producer credentials to access the queue properties):

I close this in favor of T879

Status, there are no longer missing objects:

Well, there are errors.

• Make sure nothing is amiss