Page MenuHomeSoftware Heritage

Save code now report error for svn type
Open, NormalPublic

Description

Monitoring save code now [1] for type svn report error [2]
from the worker stacktrace, this looks like an off by one when starting up visits from
an already visited repository:

Apr 17 17:07:24 worker2 python3[2637389]: [2021-04-17 17:07:24,952: INFO/ForkPoolWorker-1] Load origin 'https://github.com/cran/SCRABBLE' with type 'svn'
Apr 17 17:07:26 worker2 python3[2637389]: [2021-04-17 17:07:26,765: ERROR/ForkPoolWorker-1] Loading failure, updating to `failed` status
                                          Traceback (most recent call last):
                                            File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 330, in load
                                              self.prepare()
                                            File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 756, in prepare
                                              dump_path = self.dump_svn_revisions(self.svn_url, last_loaded_svn_rev)
                                            File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 741, in dump_svn_revisions
                                              % (last_dumped_rev, last_loaded_svn_rev)
                                          Exception: Last dumped subversion revision (0) is lesser than the last one loaded into the archive (0).

This needs some fixing.

[2] Ui view

Exception: Last dumped subversion revision (0) is lesser than the last one loaded into the archive (0).

[1] T1481

Event Timeline

ardumont triaged this task as Normal priority.Apr 19 2021, 9:01 AM
ardumont created this task.
ardumont updated the task description. (Show Details)

The error is related to that github origin erroneously submitted with a svn visit type.

Github only supports a small subset of svn client commands and svnrdump does not work for github repos.

(swh) anlambert@carnavalet:/tmp$ svnrdump dump https://github.com/cran/SCRABBLE
SVN-fs-dump-format-version: 3

UUID: 72fe256d-5aa7-4cf9-fc0d-ce57b7db7bff

Revision-number: 0
Prop-content-length: 104
Content-length: 104

K 10
git-commit
V 0

K 7
git-ref
V 0

K 10
svn:author
V 0

K 8
svn:date
V 0

K 7
svn:log
V 0

PROPS-END

* Dumped revision 0.
svnrdump: E200007: The requested report is unknown.

Nevertheless, that error should have been detected on the first loading so some adaptation is needed in the svn loader.

Thanks for the heads up ;)

In any case, independently from this, for the monitoring, I was set on modifying the
actual svn origin used to something else not hosted on github. I was of a mind to first
figuring out how to mirror, or at least fork some exising svn repository in our forge
and check that regularly with the monitoring [1]

So this goes in the direction of ridding oneself of such error ^.

[1] The monitoring's purpose being to check the plumbing is ok and not necessarily to
check the time it took to ingest an oversized repository. That's a variable out of our
control and I did not want flip flopping in the monitoring for no good reason...

In any case, independently from this, for the monitoring, I was set on modifying the
actual svn origin used to something else not hosted on github.

That check is now switched to another origin not hosted on github and those are reported
green by icinga.

Nevertheless, that error should have been detected on the first loading so some adaptation is needed in the svn loader.

That's the remaining part for this task to be closed.