Feed Advanced Search

Advanced Search
Use Results
Edit Query
Hide Query

Oct 21 2015

olasd added a parent task for T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too): T116: Add storage endpoints to help with repository updates.

Oct 21 2015, 5:58 PM · Git cloner, Git loader

Oct 20 2015

olasd added a comment to T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too).

Started playing with dulwich's git smart protocol client.

test_update.py3 KBDownload

Oct 20 2015, 7:18 PM · Git cloner, Git loader

Oct 16 2015

ardumont added a project to T102: Add synthetic flag to false for swh-loader-git: Developers.

Oct 16 2015, 12:45 PM · Git loader

ardumont closed T102: Add synthetic flag to false for swh-loader-git as Resolved.

Oct 16 2015, 12:38 PM · Git loader

ardumont created T102: Add synthetic flag to false for swh-loader-git.

Oct 16 2015, 12:36 PM · Git loader

Oct 12 2015

olasd closed T76: Reload repositories whose import failed due to connection issues as Resolved.

Those repositories have been rescheduled with the rest of the 13 million repos that hadn't been imported yet.

Oct 12 2015, 10:17 AM · Git loader

Oct 11 2015

olasd closed T73: Reload repositories with null tag names as Resolved.

The tasks are currently running.

Oct 11 2015, 12:57 PM · Git loader

Oct 6 2015

olasd created T76: Reload repositories whose import failed due to connection issues.

Oct 6 2015, 2:42 PM · Git loader

olasd closed T65: Support authors with non-utf8-encoded names as Resolved by committing rDSTOe80adb87444c: swh-schema: person (name, email) -> bytea.

Oct 6 2015, 2:02 PM · Storage manager, Git loader

olasd added projects to T73: Reload repositories with null tag names: Developers, Git loader.

Oct 6 2015, 12:43 PM · Git loader

olasd closed T64: Support tags with empty or non-utf8 messages as Resolved.

Done in rDLDG61e2b1ecd061

Oct 6 2015, 12:28 PM · Git loader

Oct 5 2015

zack created T68: support for git tags that point to arbitrary git objects, instead of revisions.

Oct 5 2015, 2:24 PM · Git loader

Oct 4 2015

zack added a parent task for T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too): T66: clone and load fork GitHub repositories.

Oct 4 2015, 5:40 PM · Git cloner, Git loader

olasd updated the task description for T65: Support authors with non-utf8-encoded names.

Oct 4 2015, 5:31 PM · Storage manager, Git loader

olasd created T65: Support authors with non-utf8-encoded names.

Oct 4 2015, 5:30 PM · Storage manager, Git loader

olasd created T64: Support tags with empty or non-utf8 messages.

Oct 4 2015, 5:29 PM · Git loader

zack added a comment to T36: performance estimation: how long will it take to git-bulk-load all the GitHub repos we have.

In T36#629, @olasd wrote:

sadly, 14k is only .1% ;)

Oct 4 2015, 1:47 PM · Git loader

zack added a comment to T36: performance estimation: how long will it take to git-bulk-load all the GitHub repos we have.

In T36#626, @olasd wrote:

IPython notebook to play with the result times scatter plot :
result_times.ipynb2 KBDownload

Oct 4 2015, 12:47 PM · Git loader

olasd added a comment to T36: performance estimation: how long will it take to git-bulk-load all the GitHub repos we have.

In T36#628, @zack wrote:

Based on that data, here are the current average/stddev processing times per repository based on the first ~14k random repositories loaded (~1% of our total):

Oct 4 2015, 1:34 AM · Git loader

Oct 3 2015

zack added a comment to T36: performance estimation: how long will it take to git-bulk-load all the GitHub repos we have.

(Thanks for making me play for the first time with a IPython notebook, it's a pretty impressive environment to play with scientific data.)

Oct 3 2015, 8:17 PM · Git loader

olasd added a comment to T36: performance estimation: how long will it take to git-bulk-load all the GitHub repos we have.

IPython notebook to play with the result times scatter plot :

result_times.ipynb2 KBDownload

Oct 3 2015, 5:34 PM · Git loader

Oct 2 2015

zack added a project to T17: handle github assets in git loader: Git loader.

Oct 2 2015, 11:04 PM · Git loader

zack added projects to T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too): Git loader, Git cloner.

Oct 2 2015, 10:59 PM · Git cloner, Git loader

zack added a project to T36: performance estimation: how long will it take to git-bulk-load all the GitHub repos we have: Git loader.

Oct 2 2015, 10:58 PM · Git loader

zack removed a member for Git loader: zack.

Oct 2 2015, 10:55 PM

zack added hashtags to Git loader: #swh-loader-git, #loader-git.

Oct 2 2015, 10:49 PM

zack created Git loader.

Oct 2 2015, 10:49 PM