Page MenuHomeSoftware Heritage

loader-svn: Some SVN origins have occurrences that point to non-existent objects
Closed, ResolvedPublic

Description

The following four SVN origins have occurrences that point to revisions that haven't been loaded :

http://ich-sys.googlecode.com/svn/ {'cab5adf110bafe53285ca78f913a860710e83cf5'}
http://project-a.googlecode.com/svn/ {'e89361aaf48051916049a0a14463e0fb78ea8ee6'}
http://shenzipattendesign.googlecode.com/svn/ {'13ae440423bf81c5def18417501839ecd455ce52'}
http://zipeg-gae.googlecode.com/svn/ {'9e3a0cdce5b93b68ad68b7a479d4c8abe889fd5a'}

This needs to be investigated and fixed.

Event Timeline

olasd created this task.Nov 14 2017, 11:27 AM
ardumont added a comment.EditedNov 23 2017, 3:13 PM

For information, for the first url, It's more a bug in the origin_url computation which results with the same origin_url for 2 differents dumps with the same name:

ardumont@uffizi:~% grep ich-sys /srv/storage/space/mirrors/code.google.com/sources/INDEX-svn-dumps
http://ich-sys.googlecode.com/svn/ /srv/storage/space/mirrors/code.google.com/sources/v2/code.google.com/i/ich-sys/ich-sys-repo.svndump.gz
http://ich-sys.googlecode.com/svn/ /srv/storage/space/mirrors/code.google.com/sources/v2/eclipselabs.org/i/ich-sys/ich-sys-repo.svndump.gz

The missing revision is found on the second origin.

ardumont changed the task status from Open to Work in Progress.Dec 8 2017, 1:39 PM
ardumont claimed this task.Dec 8 2017, 1:42 PM
ardumont changed the status of subtask T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps from Open to Work in Progress.
ardumont renamed this task from Some SVN origins have occurrences that point to non-existent objects to loader-svn: Some SVN origins have occurrences that point to non-existent objects.Dec 8 2017, 1:50 PM
ardumont closed this task as Resolved.Dec 11 2017, 10:20 AM

Status, there are no longer missing objects:

http://ich-sys.googlecode.com/svn/ {'xcd13af289bf79be7f225df8a1748e8dac87f9044'}
http://code.google.com/eclipselabs/i/ich-sys/svn/ {'cab5adf110bafe53285ca78f913a860710e83cf5'}
http://project-a.googlecode.com/svn/ {'e89361aaf48051916049a0a14463e0fb78ea8ee6'}
http://code.google.com/eclipselabs/p/project-a/svn/ {'e89361aaf48051916049a0a14463e0fb78ea8ee6'}
http://shenzipattendesign.googlecode.com/svn/ {'13ae440423bf81c5def18417501839ecd455ce52'}
http://zipeg-gae.googlecode.com/svn/ {'9e3a0cdce5b93b68ad68b7a479d4c8abe889fd5a'}

ich-sys now has a different revision target since indeed, it clashed with another dump.

INDEX.svndump:/srv/storage/space/mirrors/code.google.com/sources/v2/code.google.com/i/ich-sys/ich-sys-repo.svndump.gz
INDEX.svndump:/srv/storage/space/mirrors/code.google.com/sources/v2/eclipselabs.org/i/ich-sys/ich-sys-repo.svndump.gz

Checking the new and distinct origin (the one from eclipselabs), we found back the initial targetted revision.

Note: INDEX.svndump is a derivative file only filtering the svndumps from googlecode (stored at uffizi).

Source, main db:

softwareheritage=> select o.id, o.type, o.url, rev.id, rev.metadata from occurrence occ inner join origin o on (occ.origin = o.id and o.type='svn') inner join revision rev on (occ.target = rev.id and occ.target_type='revision') where o.url
='http://ich-sys.googlecode.com/svn/';
    id    | type |                url                 |                     id                     |                                                metadata
----------+------+------------------------------------+--------------------------------------------+---------------------------------------------------------------------------------------------------------
 55073740 | svn  | http://ich-sys.googlecode.com/svn/ | \xcd13af289bf79be7f225df8a1748e8dac87f9044 | {"extra_headers": [["svn_repo_uuid", "c2f1d9da-fff0-4c43-9df6-0e52ed2426b6"], ["svn_revision", "129"]]}
(1 row)

softwareheritage=> select o.id, o.type, o.url, rev.id, rev.metadata from occurrence occ inner join origin o on (occ.origin = o.id and o.type='svn') inner join revision rev on (occ.target = rev.id and occ.target_type='revision') where o.url='http://project-a.googlecode.com/svn/';
    id    | type |                 url                  |                     id                     |                                                metadata
----------+------+--------------------------------------+--------------------------------------------+--------------------------------------------------------------------------------------------------------
 54595134 | svn  | http://project-a.googlecode.com/svn/ | \xe89361aaf48051916049a0a14463e0fb78ea8ee6 | {"extra_headers": [["svn_repo_uuid", "39805ea9-f6bc-4665-bec4-a34ea9db31db"], ["svn_revision", "24"]]}
(1 row)

softwareheritage=> select o.id, o.type, o.url, rev.id, rev.metadata from occurrence occ inner join origin o on (occ.origin = o.id and o.type='svn') inner join revision rev on (occ.target = rev.id and occ.target_type='revision') where o.url='http://shenzipattendesign.googlecode.com/svn/';
    id    | type |                      url                      |                     id                     |                                               metadata
----------+------+-----------------------------------------------+--------------------------------------------+-------------------------------------------------------------------------------------------------------
 55582630 | svn  | http://shenzipattendesign.googlecode.com/svn/ | \x13ae440423bf81c5def18417501839ecd455ce52 | {"extra_headers": [["svn_repo_uuid", "25e1d323-984f-4d73-b2ca-3507d4f494fe"], ["svn_revision", "8"]]}
(1 row)

softwareheritage=> select o.id, o.type, o.url, rev.id, rev.metadata from occurrence occ inner join origin o on (occ.origin = o.id and o.type='svn') inner join revision rev on (occ.target = rev.id and occ.target_type='revision') where o.url='http://zipeg-gae.googlecode.com/svn/';
    id    | type |                 url                  |                     id                     |                                                metadata
----------+------+--------------------------------------+--------------------------------------------+--------------------------------------------------------------------------------------------------------
 55394265 | svn  | http://zipeg-gae.googlecode.com/svn/ | \x9e3a0cdce5b93b68ad68b7a479d4c8abe889fd5a | {"extra_headers": [["svn_repo_uuid", "df9e112f-810d-4914-ad69-4291329273d4"], ["svn_revision", "19"]]}
(1 row)
softwareheritage=> select o.id, o.type, o.url, rev.id, rev.metadata from occurrence occ inner join origin o on (occ.origin = o.id and o.type='svn') inner join revision rev on (occ.target = rev.id and occ.target_type='revision') where o.url='http://code.google.com/eclipselabs/i/ich-sys/svn/';
    id    | type |                        url                        |                     id                     |                                               metadata
----------+------+---------------------------------------------------+--------------------------------------------+-------------------------------------------------------------------------------------------------------
 65643618 | svn  | http://code.google.com/eclipselabs/i/ich-sys/svn/ | \xcab5adf110bafe53285ca78f913a860710e83cf5 | {"extra_headers": [["svn_repo_uuid", "65a79042-1f59-4223-b7ab-b49cb54978d3"], ["svn_revision", "3"]]}
(1 row)
softwareheritage=> select o.id, o.type, o.url, rev.id, rev.metadata from occurrence occ inner join origin o on (occ.origin = o.id and o.type='svn') inner join revision rev on (occ.target = rev.id and occ.target_type='revision') where o.url='http://code.google.com/eclipselabs/p/project-a/svn/';
    id    | type |                         url                         |                     id                     |                                                metadata
----------+------+-----------------------------------------------------+--------------------------------------------+--------------------------------------------------------------------------------------------------------
 65644141 | svn  | http://code.google.com/eclipselabs/p/project-a/svn/ | \xe89361aaf48051916049a0a14463e0fb78ea8ee6 | {"extra_headers": [["svn_repo_uuid", "39805ea9-f6bc-4665-bec4-a34ea9db31db"], ["svn_revision", "24"]]}
(1 row)
ardumont reopened this task as Open.Dec 13 2017, 11:42 AM

Reopened as new missing children task was created.