Page MenuHomeSoftware Heritage
Feed Advanced Search

Jan 8 2023

gitlab-migration closed T958: googlecode import: Clean up googlecode origin's origin_visits, a subtask of T617: ingest Google Code Subversion repositories, as Migrated.
Jan 8 2023, 10:21 PM · Archive coverage, Origin-GoogleCode, SVN Loader
gitlab-migration closed T958: googlecode import: Clean up googlecode origin's origin_visits as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:21 PM · SVN Loader, Origin-GoogleCode, Archive content
gitlab-migration changed the status of T948: googlecode import: Loading failure on symbolic link edge cases, a subtask of T879: Reschedule googlecode svn origins from scratch, from Resolved to Migrated.
Jan 8 2023, 9:57 PM · Origin-GoogleCode, SVN Loader, Archive content
gitlab-migration changed the status of T955: googlecode import: hglib.error.CommandError during loading from Wontfix to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:57 PM · Origin-GoogleCode, Archive content, Mercurial loader
gitlab-migration changed the status of T947: googlecode import: Some dumps are just empty repository from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:57 PM · Origin-GoogleCode, SVN Loader, Archive content
gitlab-migration changed the status of T948: googlecode import: Loading failure on symbolic link edge cases from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:57 PM · Origin-GoogleCode, SVN Loader, Archive content
gitlab-migration changed the status of T947: googlecode import: Some dumps are just empty repository, a subtask of T879: Reschedule googlecode svn origins from scratch, from Resolved to Migrated.
Jan 8 2023, 9:57 PM · Origin-GoogleCode, SVN Loader, Archive content
gitlab-migration changed the status of T896: Clean up wrong origins, a subtask of T879: Reschedule googlecode svn origins from scratch, from Resolved to Migrated.
Jan 8 2023, 9:57 PM · Origin-GoogleCode, SVN Loader, Archive content
gitlab-migration changed the status of T896: Clean up wrong origins from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:57 PM · Origin-GoogleCode, SVN Loader, Archive content
gitlab-migration changed the status of T879: Reschedule googlecode svn origins from scratch, a subtask of T617: ingest Google Code Subversion repositories, from Resolved to Migrated.
Jan 8 2023, 9:57 PM · Archive coverage, Origin-GoogleCode, SVN Loader
gitlab-migration changed the status of T879: Reschedule googlecode svn origins from scratch from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:57 PM · Origin-GoogleCode, SVN Loader, Archive content
gitlab-migration changed the status of T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading, a subtask of T879: Reschedule googlecode svn origins from scratch, from Resolved to Migrated.
Jan 8 2023, 4:24 PM · Origin-GoogleCode, SVN Loader, Archive content
gitlab-migration changed the status of T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:24 PM · Origin-GoogleCode, SVN Loader
gitlab-migration changed the status of T876: loader-svn: Reschedule origins with missing data from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:23 PM · Origin-GoogleCode, SVN Loader
gitlab-migration changed the status of T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:23 PM · Origin-GoogleCode, SVN Loader
gitlab-migration changed the status of T847: loader-svn: Some SVN origins have occurrences that point to non-existent objects, a subtask of T617: ingest Google Code Subversion repositories, from Resolved to Migrated.
Jan 8 2023, 4:23 PM · Archive coverage, Origin-GoogleCode, SVN Loader
gitlab-migration changed the status of T676: Google Code SVN import: Examine ingestion logs for errors and list them if any, a subtask of T617: ingest Google Code Subversion repositories, from Resolved to Migrated.
Jan 8 2023, 4:20 PM · Archive coverage, Origin-GoogleCode, SVN Loader
gitlab-migration changed the status of T617: ingest Google Code Subversion repositories from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:20 PM · Archive coverage, Origin-GoogleCode, SVN Loader
gitlab-migration changed the status of T328: svn / subversion loader, a subtask of T617: ingest Google Code Subversion repositories, from Resolved to Migrated.
Jan 8 2023, 4:18 PM · Archive coverage, Origin-GoogleCode, SVN Loader

Nov 25 2020

zack closed T617: ingest Google Code Subversion repositories as Resolved.
Nov 25 2020, 1:49 PM · Archive coverage, Origin-GoogleCode, SVN Loader

Jul 3 2019

ardumont placed T617: ingest Google Code Subversion repositories up for grabs.
Jul 3 2019, 3:26 PM · Archive coverage, Origin-GoogleCode, SVN Loader
ardumont placed T958: googlecode import: Clean up googlecode origin's origin_visits up for grabs.
Jul 3 2019, 3:26 PM · SVN Loader, Origin-GoogleCode, Archive content

May 25 2019

zack closed T328: svn / subversion loader, a subtask of T617: ingest Google Code Subversion repositories, as Resolved.
May 25 2019, 5:01 PM · Archive coverage, Origin-GoogleCode, SVN Loader

Sep 30 2018

ardumont added a comment to T958: googlecode import: Clean up googlecode origin's origin_visits.

And also make sure the one visit date is the right one:

Sep 30 2018, 11:39 AM · SVN Loader, Origin-GoogleCode, Archive content

Sep 28 2018

ardumont closed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading as Resolved.
Sep 28 2018, 3:04 PM · Origin-GoogleCode, SVN Loader
ardumont closed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading, a subtask of T879: Reschedule googlecode svn origins from scratch, as Resolved.
Sep 28 2018, 3:04 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.
  • new loader svn packaged and deployed
  • origins in error scheduled back
  • workers logs (kibana dashboard) [1]
Sep 28 2018, 3:04 PM · Origin-GoogleCode, SVN Loader

Sep 27 2018

ardumont renamed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading from loader-svn: googlecode import: UnicodeDecodeError in user svn properties to loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.
Sep 27 2018, 6:12 PM · Origin-GoogleCode, SVN Loader
ardumont renamed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading from loader-svn: googlecode import: UnicodeDecodeError to loader-svn: googlecode import: UnicodeDecodeError in user svn properties.
Sep 27 2018, 6:12 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Ok, so going for that fix.

Sep 27 2018, 4:51 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

\m/

Sep 27 2018, 4:48 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

So here are the results:

Sep 27 2018, 4:43 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

I was trying to play with your Python scripts to query kibana logs but it's been a while since I
did not write a query for elastic search and their json format is still awful.

Sep 27 2018, 4:24 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Today is not a good day for me ;)

Sep 27 2018, 3:53 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Great to have kibana back!

Sep 27 2018, 3:15 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Another try with another svn repo gives me the following output:

Sep 27 2018, 2:37 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Great to have kibana back!

Sep 27 2018, 2:29 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

I had to configure back the new kibana0 (was banco before) to start parsing those logs back.

Sep 27 2018, 1:56 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

This issue is reproduced on 6 repositories (so far, still running locally on some other big repositories).

To my old self, what are those other 5 repositories?

Sep 27 2018, 10:50 AM · Origin-GoogleCode, SVN Loader

Sep 25 2018

ardumont renamed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading from googlecode import: UnicodeDecodeError to loader-svn: googlecode import: UnicodeDecodeError.
Sep 25 2018, 2:18 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Digging deeper to try and improve the result to return (at the moment, empty string).
I tried to use chardet to detect the language to try and decode the bytes.
This fails as nothing appropriate is found.

Sep 25 2018, 2:18 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

By analyzing the repository dump file in emacs

Sep 25 2018, 2:18 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Thinking a bit more about the issue, there might a way to workaround it in our client code instead
of hacking in subvertpy.

Sep 25 2018, 2:18 PM · Origin-GoogleCode, SVN Loader

Sep 24 2018

anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Tracking down the issue in subvertpy source code, the error occurs in the subvertpy._ra C extension module.
More precisely, an exception is raised at line 1068 in file subvertpy/editor.c when Python tries to decode a svn property value from 'utf-8' encoding.

1062  static svn_error_t *py_cb_editor_change_prop(void *dir_baton, const char *name, const svn_string_t *value, apr_pool_t *pool)
1063  {
1064          PyObject *self = (PyObject *)dir_baton, *ret;
1065          PyGILState_STATE state = PyGILState_Ensure();
1066
1067          if (value != NULL) {
1068                  ret = PyObject_CallMethod(self, "change_prop", "sz#", name, value->data, value->len);
1069          } else {
1070                  ret = PyObject_CallMethod(self, "change_prop", "sO", name, Py_None);
1071          }
1072          CB_CHECK_PYRETVAL(ret);
1073          Py_DECREF(ret);
1074          PyGILState_Release(state);
1075          return NULL;
1076  }
Sep 24 2018, 4:55 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

(If you are not interested in details, no need to look further, it's a run extract ;)

Sep 24 2018, 4:24 PM · Origin-GoogleCode, SVN Loader
ardumont lowered the priority of T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading from High to Normal.

This one is still true. Still banging my head on this.

Sep 24 2018, 4:14 PM · Origin-GoogleCode, SVN Loader

Sep 19 2018

ardumont closed T879: Reschedule googlecode svn origins from scratch, a subtask of T617: ingest Google Code Subversion repositories, as Resolved.
Sep 19 2018, 1:56 PM · Archive coverage, Origin-GoogleCode, SVN Loader
ardumont closed T879: Reschedule googlecode svn origins from scratch as Resolved.

That's been done for a while now.

Sep 19 2018, 1:56 PM · Origin-GoogleCode, SVN Loader, Archive content

Jun 19 2018

zack edited projects for T617: ingest Google Code Subversion repositories, added: Archive coverage; removed Archive content.
Jun 19 2018, 3:28 PM · Archive coverage, Origin-GoogleCode, SVN Loader

Feb 14 2018

ardumont closed T955: googlecode import: hglib.error.CommandError during loading as Wontfix.

I close this as another mirror exists that we already browsed multiple times.

Feb 14 2018, 10:23 AM · Origin-GoogleCode, Archive content, Mercurial loader

Feb 13 2018

ardumont added a comment to T955: googlecode import: hglib.error.CommandError during loading.

Basic checks on the archive is fine:

Feb 13 2018, 3:06 PM · Origin-GoogleCode, Archive content, Mercurial loader
ardumont added projects to T617: ingest Google Code Subversion repositories: SVN Loader, Origin-GoogleCode.
Feb 13 2018, 2:31 PM · Archive coverage, Origin-GoogleCode, SVN Loader
ardumont created T958: googlecode import: Clean up googlecode origin's origin_visits.
Feb 13 2018, 1:45 PM · SVN Loader, Origin-GoogleCode, Archive content
ardumont renamed T955: googlecode import: hglib.error.CommandError during loading from import googlecode: hglib.error.CommandError during loading to googlecode import: hglib.error.CommandError during loading.
Feb 13 2018, 12:21 PM · Origin-GoogleCode, Archive content, Mercurial loader
ardumont created T955: googlecode import: hglib.error.CommandError during loading.
Feb 13 2018, 12:11 PM · Origin-GoogleCode, Archive content, Mercurial loader

Feb 6 2018

ardumont closed T948: googlecode import: Loading failure on symbolic link edge cases as Resolved by committing rDLDSVN06bcb409d9af: swh.loader.svn: Fix corner edge case on symbolic link.
Feb 6 2018, 3:35 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont closed T947: googlecode import: Some dumps are just empty repository, a subtask of T879: Reschedule googlecode svn origins from scratch, as Resolved.
Feb 6 2018, 3:35 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont closed T948: googlecode import: Loading failure on symbolic link edge cases, a subtask of T879: Reschedule googlecode svn origins from scratch, as Resolved.
Feb 6 2018, 3:35 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont closed T947: googlecode import: Some dumps are just empty repository as Resolved by committing rDLDSVNde3c7a031f8b: swh.loader.svn: Deal with empty svn repository.
Feb 6 2018, 3:35 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

This issue is reproduced on 6 repositories (so far, still running locally on some other big repositories).
Those fails are commits holding non-unicode named characters in tree or filename (japanese for example).
What's failing is not clear at all though.

Feb 6 2018, 11:49 AM · Origin-GoogleCode, SVN Loader

Feb 5 2018

ardumont added a comment to T948: googlecode import: Loading failure on symbolic link edge cases.

It appears that in this case, the properties must be changed not to the symlink but to its source.

Feb 5 2018, 5:42 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont created T948: googlecode import: Loading failure on symbolic link edge cases.
Feb 5 2018, 3:46 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont changed the status of T947: googlecode import: Some dumps are just empty repository, a subtask of T879: Reschedule googlecode svn origins from scratch, from Open to Work in Progress.
Feb 5 2018, 1:45 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont renamed T947: googlecode import: Some dumps are just empty repository from googlecode import: Some dumps starts their log to revision 0 to googlecode import: Some dumps are just empty repository.
Feb 5 2018, 1:45 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont added a comment to T947: googlecode import: Some dumps are just empty repository.

It's more empty repository case than a repository starting its commit range at 0...

Feb 5 2018, 1:37 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont created T947: googlecode import: Some dumps are just empty repository.
Feb 5 2018, 11:43 AM · Origin-GoogleCode, SVN Loader, Archive content

Feb 2 2018

ardumont created T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.
Feb 2 2018, 1:52 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T879: Reschedule googlecode svn origins from scratch.

This is in stand-by during the snapshot migration.

Feb 2 2018, 1:44 PM · Origin-GoogleCode, SVN Loader, Archive content

Dec 14 2017

ardumont edited P203 Clean up wrong googlecode origins (googlecode instead of eclipselabs or apache-extras) - T897 related.
Dec 14 2017, 3:10 PM · Origin-GoogleCode, Storage manager
ardumont edited P203 Clean up wrong googlecode origins (googlecode instead of eclipselabs or apache-extras) - T897 related.
Dec 14 2017, 3:09 PM · Origin-GoogleCode, Storage manager
ardumont closed T896: Clean up wrong origins, a subtask of T879: Reschedule googlecode svn origins from scratch, as Resolved.
Dec 14 2017, 3:03 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont closed T896: Clean up wrong origins as Resolved.
Dec 14 2017, 3:03 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont created P203 Clean up wrong googlecode origins (googlecode instead of eclipselabs or apache-extras) - T897 related.
Dec 14 2017, 2:55 PM · Origin-GoogleCode, Storage manager
ardumont edited P202 Clean up wrong origins (url is filled with the size instead of the actual origin url) - T896 related.
Dec 14 2017, 2:50 PM · Origin-GoogleCode, Storage manager
ardumont added a comment to T896: Clean up wrong origins.

P202 checked and ok locally.
Now asked for review as it will remove data from the main db.

Dec 14 2017, 12:06 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont changed the status of T896: Clean up wrong origins from Open to Work in Progress.
Dec 14 2017, 12:06 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont changed the status of T896: Clean up wrong origins, a subtask of T879: Reschedule googlecode svn origins from scratch, from Open to Work in Progress.
Dec 14 2017, 12:06 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont added a comment to T879: Reschedule googlecode svn origins from scratch.

After discussion with the team, it has been decided to remove from the re-scheduling the svn dumps whose compressed size exceeds 2Gib.
This reflects the same decision took for git repositories.

Dec 14 2017, 12:05 PM · Origin-GoogleCode, SVN Loader, Archive content

Dec 13 2017

ardumont added a project to P202 Clean up wrong origins (url is filled with the size instead of the actual origin url) - T896 related: Origin-GoogleCode.
Dec 13 2017, 6:57 PM · Origin-GoogleCode, Storage manager
ardumont created T896: Clean up wrong origins.
Dec 13 2017, 11:40 AM · Origin-GoogleCode, SVN Loader, Archive content

Dec 11 2017

ardumont added a comment to T879: Reschedule googlecode svn origins from scratch.

Scheduled back from saatchi (as i needed the producer credentials to access the queue properties):

Dec 11 2017, 5:08 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont raised the priority of T879: Reschedule googlecode svn origins from scratch from Normal to High.
Dec 11 2017, 11:03 AM · Origin-GoogleCode, SVN Loader, Archive content
ardumont changed the status of T879: Reschedule googlecode svn origins from scratch from Open to Work in Progress.
Dec 11 2017, 11:03 AM · Origin-GoogleCode, SVN Loader, Archive content
ardumont updated the task description for T879: Reschedule googlecode svn origins from scratch.
Dec 11 2017, 11:01 AM · Origin-GoogleCode, SVN Loader, Archive content
ardumont created T879: Reschedule googlecode svn origins from scratch.
Dec 11 2017, 10:59 AM · Origin-GoogleCode, SVN Loader, Archive content
ardumont closed T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps as Resolved.
Dec 11 2017, 10:21 AM · Origin-GoogleCode, SVN Loader
ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 11 2017, 10:20 AM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.

Well, there are errors.

Dec 11 2017, 9:42 AM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
  • Make sure nothing is amiss
Dec 11 2017, 9:18 AM · Origin-GoogleCode, SVN Loader

Dec 9 2017

ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 9 2017, 11:25 AM · Origin-GoogleCode, SVN Loader
ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 9 2017, 11:22 AM · Origin-GoogleCode, SVN Loader
ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 9 2017, 11:16 AM · Origin-GoogleCode, SVN Loader
ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 9 2017, 11:14 AM · Origin-GoogleCode, SVN Loader

Dec 8 2017

ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 8 2017, 6:16 PM · Origin-GoogleCode, SVN Loader
ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 8 2017, 5:26 PM · Origin-GoogleCode, SVN Loader
ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 8 2017, 5:12 PM · Origin-GoogleCode, SVN Loader
ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 8 2017, 4:18 PM · Origin-GoogleCode, SVN Loader
ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 8 2017, 4:11 PM · Origin-GoogleCode, SVN Loader
ardumont added a project to T876: loader-svn: Reschedule origins with missing data: Origin-GoogleCode.
Dec 8 2017, 3:39 PM · Origin-GoogleCode, SVN Loader
ardumont updated the task description for T863: loader-svn: Fix origin clashes for homonym but distinct svn dumps.
Dec 8 2017, 3:38 PM · Origin-GoogleCode, SVN Loader