Feed Advanced Search

Advanced Search
Use Results
Edit Query
Hide Query

Oct 26 2017

ardumont renamed T816: Gitorious import: loose object parsing error with corrupted file as empty one from Gitorious import: to Gitorious import: loose object parsing error.

Oct 26 2017, 11:20 AM · Git loader, Origin-Gitorious

ardumont updated the task description for T815: Gitorious import: Release time conversion issue when no release date is provided.

Oct 26 2017, 11:12 AM · Origin-Gitorious, Git loader

ardumont added a project to T814: Gitorious import: unexisting object retrieval makes the loading fail: Origin-Gitorious.

Oct 26 2017, 11:11 AM · Origin-Gitorious, Git loader

ardumont added a parent task for T814: Gitorious import: unexisting object retrieval makes the loading fail: T674: Gitorious import: Examine ingestion logs for errors and list them if any.

Oct 26 2017, 11:11 AM · Origin-Gitorious, Git loader

ardumont added a parent task for T815: Gitorious import: Release time conversion issue when no release date is provided: T674: Gitorious import: Examine ingestion logs for errors and list them if any.

Oct 26 2017, 11:11 AM · Origin-Gitorious, Git loader

ardumont created T815: Gitorious import: Release time conversion issue when no release date is provided.

Oct 26 2017, 11:10 AM · Origin-Gitorious, Git loader

ardumont added a comment to T814: Gitorious import: unexisting object retrieval makes the loading fail.

git fsck on that repository shows that this entry is actually wrong.

Oct 26 2017, 11:01 AM · Origin-Gitorious, Git loader

ardumont created T814: Gitorious import: unexisting object retrieval makes the loading fail.

Oct 26 2017, 10:28 AM · Origin-Gitorious, Git loader

Oct 6 2017

zack closed T89: refactor common behavior between git loader and dir loader as Resolved.

(since quite a while, thanks to swh.loader.core)

Oct 6 2017, 2:59 PM · Git loader, Directory loader

Sep 15 2017

ardumont closed T673: ingest Google Code Git repositories, a subtask of T675: Google Code Git import: Examine ingestion logs for errors and list them if any, as Resolved.

Sep 15 2017, 3:27 PM · Git loader

Jul 28 2017

ardumont created P170 gitorious - remaining loader git disk errors.

Jul 28 2017, 11:06 AM · Git loader

ardumont closed T675: Google Code Git import: Examine ingestion logs for errors and list them if any as Resolved.

Jul 28 2017, 10:38 AM · Git loader

ardumont added a comment to T675: Google Code Git import: Examine ingestion logs for errors and list them if any.

After rescheduling of thos origins (the one we can do something about), here are the remaining errors.

Jul 28 2017, 10:38 AM · Git loader

Jul 26 2017

ardumont added a comment to T675: Google Code Git import: Examine ingestion logs for errors and list them if any.

After much learning on how to read and extract logs from our kibana instance, here is the error repartition.

Jul 26 2017, 12:56 PM · Git loader

Feb 15 2017

ardumont changed the status of T673: ingest Google Code Git repositories from Open to Work in Progress.

Feb 15 2017, 7:53 PM · Archive coverage

ardumont changed the status of T673: ingest Google Code Git repositories, a subtask of T675: Google Code Git import: Examine ingestion logs for errors and list them if any, from Open to Work in Progress.

Feb 15 2017, 7:53 PM · Git loader

ardumont added a comment to T673: ingest Google Code Git repositories.

Visit dates have been fixed for the origins already injected.

Feb 15 2017, 7:34 PM · Archive coverage

ardumont added a comment to T673: ingest Google Code Git repositories.

starting-date: 2017-02-15 14:42:27,724

Feb 15 2017, 5:16 PM · Archive coverage

zack added a subtask for T675: Google Code Git import: Examine ingestion logs for errors and list them if any: T673: ingest Google Code Git repositories.

Feb 15 2017, 4:14 PM · Git loader

zack added a parent task for T673: ingest Google Code Git repositories: T675: Google Code Git import: Examine ingestion logs for errors and list them if any.

Feb 15 2017, 4:14 PM · Archive coverage

zack removed a parent task for T675: Google Code Git import: Examine ingestion logs for errors and list them if any: T673: ingest Google Code Git repositories.

Feb 15 2017, 4:13 PM · Git loader

zack removed a subtask for T673: ingest Google Code Git repositories: T675: Google Code Git import: Examine ingestion logs for errors and list them if any.

Feb 15 2017, 4:13 PM · Archive coverage

zack removed a subtask for T673: ingest Google Code Git repositories: T682: Ingest Google Code Mercurial repositories.

Feb 15 2017, 4:03 PM · Archive coverage

ardumont updated the task description for T673: ingest Google Code Git repositories.

Feb 15 2017, 2:01 PM · Archive coverage

ardumont added a subtask for T673: ingest Google Code Git repositories: T682: Ingest Google Code Mercurial repositories.

Feb 15 2017, 1:59 PM · Archive coverage

ardumont added a comment to T673: ingest Google Code Git repositories.

As in T617, the origin date to use for injection is 'Tue, 3 May 2016 17:16:32 +0200'. We retrieved all googlecode repositories together (git, svn, hg).

Feb 15 2017, 1:47 PM · Archive coverage

Feb 14 2017

ardumont added a comment to T673: ingest Google Code Git repositories.

at last, generate a full_mapping.txt (mirroring the one from gitorious) mentioning <origin_url> <path-to-git-repository-tree-or-archive>.

Feb 14 2017, 8:47 PM · Archive coverage

Feb 13 2017

ardumont updated the task description for T673: ingest Google Code Git repositories.

Feb 13 2017, 4:36 PM · Archive coverage

Feb 12 2017

zack moved T673: ingest Google Code Git repositories from Restricted Project Column to Restricted Project Column on the Restricted Project board.

Feb 12 2017, 6:37 PM · Archive coverage

zack renamed T673: ingest Google Code Git repositories from inject googlecode's git repositories into swh to ingest Google Code Git repositories.

Feb 12 2017, 6:14 PM · Archive coverage

Feb 10 2017

ardumont renamed T675: Google Code Git import: Examine ingestion logs for errors and list them if any from Google Code Git import: Reference errors after ingestion to Google Code Git import: Examine ingestion logs for errors and list them if any.

Feb 10 2017, 3:18 PM · Git loader

olasd renamed T675: Google Code Git import: Examine ingestion logs for errors and list them if any from Reference errors after ingestion to Google Code Git import: Reference errors after ingestion.

Feb 10 2017, 2:38 PM · Git loader

ardumont created T675: Google Code Git import: Examine ingestion logs for errors and list them if any.

Feb 10 2017, 12:43 PM · Git loader

zack added a parent task for T673: ingest Google Code Git repositories: T367: ingest Google Code repositories.

Feb 10 2017, 12:28 PM · Archive coverage

ardumont claimed T673: ingest Google Code Git repositories.

Feb 10 2017, 12:26 PM · Archive coverage

ardumont updated the task description for T673: ingest Google Code Git repositories.

Feb 10 2017, 12:26 PM · Archive coverage

ardumont created T673: ingest Google Code Git repositories.

Feb 10 2017, 12:20 PM · Archive coverage

Oct 26 2016

ardumont edited P119 Problem when running reader-git in uffizi.

Oct 26 2016, 7:30 PM · Git loader

ardumont edited P119 Problem when running reader-git in uffizi.

Oct 26 2016, 7:29 PM · Git loader

ardumont edited P119 Problem when running reader-git in uffizi.

Oct 26 2016, 5:03 PM · Git loader

ardumont edited projects for P119 Problem when running reader-git in uffizi, added: Git loader; removed Indexer.

Oct 26 2016, 4:23 PM · Git loader

Aug 29 2016

ardumont closed T539: Update loaders to register origin_visit's state as Resolved.

Aug 29 2016, 3:31 PM · Directory loader, Tarball loader, Git loader, SVN Loader

ardumont added a comment to T539: Update loaders to register origin_visit's state.

This tasks takes care of:

loader-core
loader-dir (depends on loader-core)
loader-tar (depends on loader-dir and loader-core)
loader-git
loader-svn

Aug 29 2016, 3:31 PM · Directory loader, Tarball loader, Git loader, SVN Loader

Aug 23 2016

ardumont updated the task description for T539: Update loaders to register origin_visit's state.

Aug 23 2016, 11:46 AM · Directory loader, Tarball loader, Git loader, SVN Loader

May 13 2016

olasd changed the visibility for Git loader.

May 13 2016, 5:22 PM

olasd changed the visibility for T340: add missing "archive_type" property to revision.metadata JSON for all imported dsc.

May 13 2016, 5:09 PM · Git loader

olasd changed the visibility for T271: Update clients on impacts + upgrade respective package dependencies.

May 13 2016, 5:08 PM · Git loader, Directory loader, Data Model, Web app

olasd changed the visibility for T117: factor out common code from git/dir loader.

May 13 2016, 5:06 PM · Directory loader, Git loader

olasd changed the visibility for T102: Add synthetic flag to false for swh-loader-git.

May 13 2016, 5:06 PM · Git loader

olasd changed the visibility for T89: refactor common behavior between git loader and dir loader.

May 13 2016, 5:06 PM · Git loader, Directory loader

olasd changed the visibility for T76: Reload repositories whose import failed due to connection issues.

May 13 2016, 5:06 PM · Git loader

olasd changed the visibility for T73: Reload repositories with null tag names.

May 13 2016, 5:05 PM · Git loader

olasd changed the visibility for T68: support for git tags that point to arbitrary git objects, instead of revisions.

May 13 2016, 5:05 PM · Git loader

olasd changed the visibility for T65: Support authors with non-utf8-encoded names.

May 13 2016, 5:05 PM · Storage manager, Git loader

olasd changed the visibility for T64: Support tags with empty or non-utf8 messages.

May 13 2016, 5:05 PM · Git loader

olasd changed the visibility for T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too).

May 13 2016, 5:05 PM · Git cloner, Git loader

olasd changed the visibility for T36: performance estimation: how long will it take to git-bulk-load all the GitHub repos we have.

May 13 2016, 5:05 PM · Git loader

olasd changed the visibility for T17: handle github assets in git loader.

May 13 2016, 5:05 PM · Git loader

Mar 10 2016

zack removed a project from T102: Add synthetic flag to false for swh-loader-git: Developers.

Mar 10 2016, 5:54 PM · Git loader

zack removed projects from T17: handle github assets in git loader: Developers, Staff.

Mar 10 2016, 5:53 PM · Git loader

zack removed projects from T68: support for git tags that point to arbitrary git objects, instead of revisions: Developers, Staff.

Mar 10 2016, 5:53 PM · Git loader

zack removed projects from T117: factor out common code from git/dir loader: Developers, Staff.

Mar 10 2016, 5:53 PM · Directory loader, Git loader

zack removed projects from T89: refactor common behavior between git loader and dir loader: Developers, Staff.

Mar 10 2016, 5:53 PM · Git loader, Directory loader

zack removed projects from T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too): Developers, Staff.

Mar 10 2016, 5:52 PM · Git cloner, Git loader

zack removed projects from T36: performance estimation: how long will it take to git-bulk-load all the GitHub repos we have: Staff, Developers.

Mar 10 2016, 5:51 PM · Git loader

zack removed projects from T73: Reload repositories with null tag names: Developers, Staff.

Mar 10 2016, 5:51 PM · Git loader

zack removed projects from T65: Support authors with non-utf8-encoded names: Staff, Developers.

Mar 10 2016, 5:51 PM · Storage manager, Git loader

zack removed projects from T64: Support tags with empty or non-utf8 messages: Developers, Staff.

Mar 10 2016, 5:51 PM · Git loader

zack removed projects from T76: Reload repositories whose import failed due to connection issues: Staff, Developers.

Mar 10 2016, 5:51 PM · Git loader

zack removed projects from T271: Update clients on impacts + upgrade respective package dependencies: Developers, Staff.

Mar 10 2016, 5:51 PM · Git loader, Directory loader, Data Model, Web app

zack removed projects from T340: add missing "archive_type" property to revision.metadata JSON for all imported dsc: Developers, Staff.

Mar 10 2016, 5:51 PM · Git loader

zack removed a project from T102: Add synthetic flag to false for swh-loader-git: Staff.

Mar 10 2016, 5:49 PM · Git loader

Mar 4 2016

zack created T340: add missing "archive_type" property to revision.metadata JSON for all imported dsc.

Mar 4 2016, 6:21 PM · Git loader

Feb 22 2016

olasd set the image for Git loader to Unknown Object (File).

Feb 22 2016, 8:17 PM

olasd closed T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too) as Resolved.

A new git updater, based on @ardumont's proof of concept, is now available in rDLDGIT.

Feb 22 2016, 6:31 PM · Git cloner, Git loader

Feb 9 2016

olasd closed T68: support for git tags that point to arbitrary git objects, instead of revisions as Resolved.

This is now supported.

Feb 9 2016, 2:26 PM · Git loader

Jan 27 2016

ardumont renamed T271: Update clients on impacts + upgrade respective package dependencies from Update api conversion on impacted fields to Update clients on impacts + upgrade respective package dependencies.

Jan 27 2016, 4:51 PM · Git loader, Directory loader, Data Model, Web app

Jan 22 2016

ardumont changed the status of T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too) from Work in Progress to Open.

Jan 22 2016, 10:05 AM · Git cloner, Git loader

Jan 21 2016

ardumont added a comment to T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too).

For information, sample test_update.py adapted in swh-loader-git https://forge.softwareheritage.org/diffusion/DLDG/browse/master/swh/loader/git/updater.py to use the swh-storage.

Jan 21 2016, 12:08 PM · Git cloner, Git loader

ardumont changed the status of T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too) from Open to Work in Progress.

Jan 21 2016, 12:08 PM · Git cloner, Git loader

ardumont renamed T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too) from smart, all-in-one git cloner/loader to smart, all-in-one git cloner/loader/ (+ dealing with updates too).

Jan 21 2016, 12:08 PM · Git cloner, Git loader

ardumont added a comment to T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too).

Related but not limited to:
58903e5 * origin/master origin/HEAD Open occurrence_get(origin_id) to retrieve latest occurrences per origin
bc23eb9 * sql/upgrades/043: add 042→043 upgrade script
d05afde * revision_log from multiple root revisions
3a40f00 * sql/upgrades/042: add 041→042 upgrade script
f54fd8d * Open release_get_by to retrieve a release by origin.
5dc4244 * revision_get_by: branch name filtering is optional
7e623c8 * sql/upgrades/040: add 040→041 upgrade script
7e2dcbc * Open directory_get to retrieve information on directory by id

Jan 21 2016, 12:08 PM · Git cloner, Git loader

Jan 18 2016

ardumont closed T116: Add storage endpoints to help with repository updates, a subtask of T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too), as Resolved.

Jan 18 2016, 2:39 PM · Git cloner, Git loader

Oct 29 2015

zack closed T36: performance estimation: how long will it take to git-bulk-load all the GitHub repos we have as Resolved.

Oct 29 2015, 10:04 AM · Git loader

Oct 22 2015

zack closed T117: factor out common code from git/dir loader as Invalid.

duplicate of T89

Oct 22 2015, 10:54 AM · Directory loader, Git loader

zack renamed T89: refactor common behavior between git loader and dir loader from Extract common behavior between swh-loader-git and swh-loader-dir ~> swh-loader-core to refactor common behavior between git loader and dir loader.

Oct 22 2015, 10:53 AM · Git loader, Directory loader

zack created T117: factor out common code from git/dir loader.

Oct 22 2015, 10:50 AM · Directory loader, Git loader

Oct 21 2015

olasd added a subtask for T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too): T116: Add storage endpoints to help with repository updates.

Oct 21 2015, 5:59 PM · Git cloner, Git loader

olasd removed a parent task for T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too): T116: Add storage endpoints to help with repository updates.

Oct 21 2015, 5:58 PM · Git cloner, Git loader

olasd added a parent task for T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too): T116: Add storage endpoints to help with repository updates.

Oct 21 2015, 5:58 PM · Git cloner, Git loader

Oct 20 2015

olasd added a comment to T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too).

Started playing with dulwich's git smart protocol client.

test_update.py3 KBDownload

Oct 20 2015, 7:18 PM · Git cloner, Git loader

Oct 16 2015

ardumont added a project to T102: Add synthetic flag to false for swh-loader-git: Developers.

Oct 16 2015, 12:45 PM · Git loader

ardumont closed T102: Add synthetic flag to false for swh-loader-git as Resolved.

Oct 16 2015, 12:38 PM · Git loader

ardumont created T102: Add synthetic flag to false for swh-loader-git.

Oct 16 2015, 12:36 PM · Git loader

Oct 12 2015

olasd closed T76: Reload repositories whose import failed due to connection issues as Resolved.

Those repositories have been rescheduled with the rest of the 13 million repos that hadn't been imported yet.

Oct 12 2015, 10:17 AM · Git loader

Oct 11 2015

olasd closed T73: Reload repositories with null tag names as Resolved.

The tasks are currently running.

Oct 11 2015, 12:57 PM · Git loader

Oct 6 2015

olasd created T76: Reload repositories whose import failed due to connection issues.

Oct 6 2015, 2:42 PM · Git loader

olasd closed T65: Support authors with non-utf8-encoded names as Resolved by committing rDSTOe80adb87444c: swh-schema: person (name, email) -> bytea.

Oct 6 2015, 2:02 PM · Storage manager, Git loader

olasd added projects to T73: Reload repositories with null tag names: Developers, Git loader.

Oct 6 2015, 12:43 PM · Git loader

olasd closed T64: Support tags with empty or non-utf8 messages as Resolved.

Done in rDLDG61e2b1ecd061

Oct 6 2015, 12:28 PM · Git loader