Page MenuHomeSoftware Heritage

Origin-GoogleCodeTag
ActivePublic

Members

  • This project does not have any members.

Watchers

  • This project does not have any watchers.

Recent Activity

Sep 30 2018

ardumont added a comment to T958: googlecode import: Clean up googlecode origin's origin_visits.

And also make sure the one visit date is the right one:

Sep 30 2018, 11:39 AM · SVN Loader, Origin-GoogleCode, Archive content

Sep 28 2018

ardumont closed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading as Resolved.
Sep 28 2018, 3:04 PM · Origin-GoogleCode, SVN Loader
ardumont closed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading, a subtask of T879: Reschedule googlecode svn origins from scratch, as Resolved.
Sep 28 2018, 3:04 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.
  • new loader svn packaged and deployed
  • origins in error scheduled back
  • workers logs (kibana dashboard) [1]
Sep 28 2018, 3:04 PM · Origin-GoogleCode, SVN Loader

Sep 27 2018

ardumont renamed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading from loader-svn: googlecode import: UnicodeDecodeError in user svn properties to loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.
Sep 27 2018, 6:12 PM · Origin-GoogleCode, SVN Loader
ardumont renamed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading from loader-svn: googlecode import: UnicodeDecodeError to loader-svn: googlecode import: UnicodeDecodeError in user svn properties.
Sep 27 2018, 6:12 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Ok, so going for that fix.

Sep 27 2018, 4:51 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

\m/

Sep 27 2018, 4:48 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

So here are the results:

Sep 27 2018, 4:43 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

I was trying to play with your Python scripts to query kibana logs but it's been a while since I
did not write a query for elastic search and their json format is still awful.

Sep 27 2018, 4:24 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Today is not a good day for me ;)

Sep 27 2018, 3:53 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Great to have kibana back!

Sep 27 2018, 3:15 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Another try with another svn repo gives me the following output:

Sep 27 2018, 2:37 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Great to have kibana back!

Sep 27 2018, 2:29 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

I had to configure back the new kibana0 (was banco before) to start parsing those logs back.

Sep 27 2018, 1:56 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

This issue is reproduced on 6 repositories (so far, still running locally on some other big repositories).

To my old self, what are those other 5 repositories?

Sep 27 2018, 10:50 AM · Origin-GoogleCode, SVN Loader

Sep 25 2018

ardumont renamed T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading from googlecode import: UnicodeDecodeError to loader-svn: googlecode import: UnicodeDecodeError.
Sep 25 2018, 2:18 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Digging deeper to try and improve the result to return (at the moment, empty string).
I tried to use chardet to detect the language to try and decode the bytes.
This fails as nothing appropriate is found.

Sep 25 2018, 2:18 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

By analyzing the repository dump file in emacs

Sep 25 2018, 2:18 PM · Origin-GoogleCode, SVN Loader
anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Thinking a bit more about the issue, there might a way to workaround it in our client code instead
of hacking in subvertpy.

Sep 25 2018, 2:18 PM · Origin-GoogleCode, SVN Loader

Sep 24 2018

anlambert added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

Tracking down the issue in subvertpy source code, the error occurs in the subvertpy._ra C extension module.
More precisely, an exception is raised at line 1068 in file subvertpy/editor.c when Python tries to decode a svn property value from 'utf-8' encoding.

1062  static svn_error_t *py_cb_editor_change_prop(void *dir_baton, const char *name, const svn_string_t *value, apr_pool_t *pool)
1063  {
1064          PyObject *self = (PyObject *)dir_baton, *ret;
1065          PyGILState_STATE state = PyGILState_Ensure();
1066
1067          if (value != NULL) {
1068                  ret = PyObject_CallMethod(self, "change_prop", "sz#", name, value->data, value->len);
1069          } else {
1070                  ret = PyObject_CallMethod(self, "change_prop", "sO", name, Py_None);
1071          }
1072          CB_CHECK_PYRETVAL(ret);
1073          Py_DECREF(ret);
1074          PyGILState_Release(state);
1075          return NULL;
1076  }
Sep 24 2018, 4:55 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

(If you are not interested in details, no need to look further, it's a run extract ;)

Sep 24 2018, 4:24 PM · Origin-GoogleCode, SVN Loader
ardumont lowered the priority of T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading from High to Normal.

This one is still true. Still banging my head on this.

Sep 24 2018, 4:14 PM · Origin-GoogleCode, SVN Loader

Sep 19 2018

ardumont closed T879: Reschedule googlecode svn origins from scratch, a subtask of T617: ingest Google Code Subversion repositories, as Resolved.
Sep 19 2018, 1:56 PM · Archive coverage, Origin-GoogleCode, SVN Loader
ardumont closed T879: Reschedule googlecode svn origins from scratch as Resolved.

That's been done for a while now.

Sep 19 2018, 1:56 PM · Origin-GoogleCode, SVN Loader, Archive content

Jun 19 2018

zack edited projects for T617: ingest Google Code Subversion repositories, added: Archive coverage; removed Archive content.
Jun 19 2018, 3:28 PM · Archive coverage, Origin-GoogleCode, SVN Loader

Feb 14 2018

ardumont closed T955: googlecode import: hglib.error.CommandError during loading as Wontfix.

I close this as another mirror exists that we already browsed multiple times.

Feb 14 2018, 10:23 AM · Origin-GoogleCode, Archive content, Mercurial loader

Feb 13 2018

ardumont added a comment to T955: googlecode import: hglib.error.CommandError during loading.

Basic checks on the archive is fine:

Feb 13 2018, 3:06 PM · Origin-GoogleCode, Archive content, Mercurial loader
ardumont added projects to T617: ingest Google Code Subversion repositories: SVN Loader, Origin-GoogleCode.
Feb 13 2018, 2:31 PM · Archive coverage, Origin-GoogleCode, SVN Loader
ardumont created T958: googlecode import: Clean up googlecode origin's origin_visits.
Feb 13 2018, 1:45 PM · SVN Loader, Origin-GoogleCode, Archive content
ardumont renamed T955: googlecode import: hglib.error.CommandError during loading from import googlecode: hglib.error.CommandError during loading to googlecode import: hglib.error.CommandError during loading.
Feb 13 2018, 12:21 PM · Origin-GoogleCode, Archive content, Mercurial loader
ardumont created T955: googlecode import: hglib.error.CommandError during loading.
Feb 13 2018, 12:11 PM · Origin-GoogleCode, Archive content, Mercurial loader

Feb 6 2018

ardumont closed T948: googlecode import: Loading failure on symbolic link edge cases as Resolved by committing rDLDSVN06bcb409d9af: swh.loader.svn: Fix corner edge case on symbolic link.
Feb 6 2018, 3:35 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont closed T947: googlecode import: Some dumps are just empty repository, a subtask of T879: Reschedule googlecode svn origins from scratch, as Resolved.
Feb 6 2018, 3:35 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont closed T948: googlecode import: Loading failure on symbolic link edge cases, a subtask of T879: Reschedule googlecode svn origins from scratch, as Resolved.
Feb 6 2018, 3:35 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont closed T947: googlecode import: Some dumps are just empty repository as Resolved by committing rDLDSVNde3c7a031f8b: swh.loader.svn: Deal with empty svn repository.
Feb 6 2018, 3:35 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont added a comment to T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.

This issue is reproduced on 6 repositories (so far, still running locally on some other big repositories).
Those fails are commits holding non-unicode named characters in tree or filename (japanese for example).
What's failing is not clear at all though.

Feb 6 2018, 11:49 AM · Origin-GoogleCode, SVN Loader

Feb 5 2018

ardumont added a comment to T948: googlecode import: Loading failure on symbolic link edge cases.

It appears that in this case, the properties must be changed not to the symlink but to its source.

Feb 5 2018, 5:42 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont created T948: googlecode import: Loading failure on symbolic link edge cases.
Feb 5 2018, 3:46 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont changed the status of T947: googlecode import: Some dumps are just empty repository, a subtask of T879: Reschedule googlecode svn origins from scratch, from Open to Work in Progress.
Feb 5 2018, 1:45 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont renamed T947: googlecode import: Some dumps are just empty repository from googlecode import: Some dumps starts their log to revision 0 to googlecode import: Some dumps are just empty repository.
Feb 5 2018, 1:45 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont added a comment to T947: googlecode import: Some dumps are just empty repository.

It's more empty repository case than a repository starting its commit range at 0...

Feb 5 2018, 1:37 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont created T947: googlecode import: Some dumps are just empty repository.
Feb 5 2018, 11:43 AM · Origin-GoogleCode, SVN Loader, Archive content

Feb 2 2018

ardumont created T946: loader-svn: googlecode import: UnicodeDecodeError in user svn properties fails the loading.
Feb 2 2018, 1:52 PM · Origin-GoogleCode, SVN Loader
ardumont added a comment to T879: Reschedule googlecode svn origins from scratch.

This is in stand-by during the snapshot migration.

Feb 2 2018, 1:44 PM · Origin-GoogleCode, SVN Loader, Archive content

Dec 14 2017

ardumont edited P203 Clean up wrong googlecode origins (googlecode instead of eclipselabs or apache-extras) - T897 related.
Dec 14 2017, 3:10 PM · Origin-GoogleCode, Storage manager
ardumont edited P203 Clean up wrong googlecode origins (googlecode instead of eclipselabs or apache-extras) - T897 related.
Dec 14 2017, 3:09 PM · Origin-GoogleCode, Storage manager
ardumont closed T896: Clean up wrong origins, a subtask of T879: Reschedule googlecode svn origins from scratch, as Resolved.
Dec 14 2017, 3:03 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont closed T896: Clean up wrong origins as Resolved.
Dec 14 2017, 3:03 PM · Origin-GoogleCode, SVN Loader, Archive content
ardumont created P203 Clean up wrong googlecode origins (googlecode instead of eclipselabs or apache-extras) - T897 related.
Dec 14 2017, 2:55 PM · Origin-GoogleCode, Storage manager