Page MenuHomeSoftware Heritage
Feed Advanced Search

Advanced Search
Use Results
Hide Query

May 12 2016

qcampos created T403: Improve content archiver testing coverage.
May 12 2016, 12:09 PM · Restricted Project, Storage manager
qcampos updated the task description for T402: Dev: Improve starting server routine.
May 12 2016, 12:08 PM · Restricted Project, Storage manager
qcampos created T402: Dev: Improve starting server routine.
May 12 2016, 12:07 PM · Restricted Project, Storage manager
qcampos created T401: Content archiver - Asynchronous version.
May 12 2016, 12:06 PM · Restricted Project, Storage manager
qcampos created T400: Content archiver synchronous version.
May 12 2016, 12:05 PM · Restricted Project, Storage manager

May 11 2016

olasd created T399: (Re-)Compute data checksums before insertion.
May 11 2016, 7:16 PM · Storage manager
ardumont closed T308: retrieve content from s3 and store it in SWH storage as Resolved.
May 11 2016, 1:51 PM · Storage manager, Antelink loader

May 9 2016

qcampos added a revision to T240: content archiver: D23: Content archiver.
May 9 2016, 4:56 PM · Storage manager

May 4 2016

qcampos closed T381: HTTP client/server version of swh.storage.objstorage as Resolved.
May 4 2016, 3:36 PM · Storage manager
qcampos closed T381: HTTP client/server version of swh.storage.objstorage, a subtask of T240: content archiver, as Resolved.
May 4 2016, 3:36 PM · Storage manager
qcampos added a revision to T381: HTTP client/server version of swh.storage.objstorage: D10: Adding the http client/server for swh.storage.objstorage.
May 4 2016, 3:06 PM · Storage manager

Apr 27 2016

zack moved T75: Check integrity of directories, revisions, and releases from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Apr 27 2016, 9:21 PM · Archive content, Restricted Project
zack created T381: HTTP client/server version of swh.storage.objstorage.
Apr 27 2016, 9:15 PM · Storage manager
zack assigned T304: content integrity checker to qcampos.
Apr 27 2016, 9:13 PM · Storage manager
zack assigned T240: content archiver to qcampos.
Apr 27 2016, 9:13 PM · Storage manager
zack added a project to T75: Check integrity of directories, revisions, and releases: Restricted Project.
Apr 27 2016, 9:03 PM · Archive content, Restricted Project
zack lowered the priority of T111: Performance analysis of read queries from High to Normal.
Apr 27 2016, 9:02 PM · Storage manager
zack assigned T308: retrieve content from s3 and store it in SWH storage to ardumont.
Apr 27 2016, 9:02 PM · Storage manager, Antelink loader
zack renamed T308: retrieve content from s3 and store it in SWH storage from Retrieve content from s3 and store inside swh-storage to retrieve content from s3 and store it in SWH storage.
Apr 27 2016, 9:01 PM · Storage manager, Antelink loader
zack moved T240: content archiver from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Apr 27 2016, 10:29 AM · Storage manager
zack added a project to T240: content archiver: Restricted Project.
Apr 27 2016, 10:28 AM · Storage manager

Apr 19 2016

olasd abandoned D1: Simplify swh_revision_find_occurrence.
Apr 19 2016, 3:38 PM · Storage manager

Apr 7 2016

olasd closed T366: Refactor the person table, a subtask of T75: Check integrity of directories, revisions, and releases, as Resolved.
Apr 7 2016, 6:30 PM · Archive content, Restricted Project
olasd closed T366: Refactor the person table as Resolved by committing rDSTO740bef462123: sql/swh-schema: add a fullname field to person.
Apr 7 2016, 6:30 PM · Storage manager
zack added a comment to T366: Refactor the person table.

after F2F discussion: ACK! /o\

Apr 7 2016, 2:42 PM · Storage manager
olasd created T366: Refactor the person table.
Apr 7 2016, 2:28 PM · Storage manager

Apr 6 2016

olasd added a comment to T75: Check integrity of directories, revisions, and releases.

We currently have 3'431'504 revisions that need fixing (on a total of 469'428'397, that is 0.73%).

Apr 6 2016, 2:04 PM · Archive content, Restricted Project

Apr 5 2016

olasd added a comment to T75: Check integrity of directories, revisions, and releases.

The scripts checking the integrity of revisions and fixing some of them have been committed to swh-storage (in a new utils directory).

Apr 5 2016, 4:46 PM · Archive content, Restricted Project

Mar 31 2016

rdicosmo added a comment to T75: Check integrity of directories, revisions, and releases.

Ok, one more reason why we must keep the original data from the dumps.

Mar 31 2016, 4:06 PM · Archive content, Restricted Project

Mar 30 2016

olasd added a comment to T75: Check integrity of directories, revisions, and releases.

The dulwich pull request has been merged, and the corresponding package has been added to the local swh archive for importers.

Mar 30 2016, 3:11 PM · Archive content, Restricted Project

Mar 10 2016

zack removed a project from T103: Add synthetic flag in release/revision tables: Developers.
Mar 10 2016, 5:54 PM · Storage manager
zack removed a project from T111: Performance analysis of read queries: Developers.
Mar 10 2016, 5:54 PM · Storage manager
zack removed a project from T121: Populate fetch_history: Developers.
Mar 10 2016, 5:54 PM · Storage manager
zack removed a project from T163: Add entry points to create / request entities in swh.storage: Developers.
Mar 10 2016, 5:54 PM · Storage manager
zack removed a project from T209: Retrieve content with other checksums than sha1: Developers.
Mar 10 2016, 5:54 PM · Storage manager
zack removed a project from T216: /api/1/revision/<sha1> - Single revision with no parent: Developers.
Mar 10 2016, 5:54 PM · Storage manager, Web app
zack removed a project from T272: Fix double-encoded release.name entries: Developers.
Mar 10 2016, 5:54 PM · Data Model, Storage manager
zack removed a project from T269: Deal nicely with swh-storage's backend down time in clients: Developers.
Mar 10 2016, 5:54 PM · Web app, Storage manager
zack removed a project from T264: directory_get: Retrieve information on a directory per hash.: Developers.
Mar 10 2016, 5:54 PM · Web app, Storage manager
zack removed a project from T308: retrieve content from s3 and store it in SWH storage: Developers.
Mar 10 2016, 5:54 PM · Storage manager, Antelink loader
zack removed projects from T45: Fix swh.storage.storage.occurrence_add for overlapping intervals: Staff, Developers.
Mar 10 2016, 5:53 PM · Storage manager
zack removed projects from T72: occurrence history table: use an array of timestamps instead of intervals: Developers, Staff.
Mar 10 2016, 5:53 PM · Storage manager
zack removed projects from T3: Discuss the project <-> origin mapping: Staff, Developers.
Mar 10 2016, 5:52 PM · Storage manager
zack removed projects from T22: DB schema: add metadata field to revision table: Developers, Staff.
Mar 10 2016, 5:52 PM · Storage manager
zack removed projects from T41: add actual content to swh-storage-testdata: Developers, Staff.
Mar 10 2016, 5:52 PM · Storage manager
zack removed projects from T49: DB schema: add missing unicity constraint on origin (type, url): Developers, Staff.
Mar 10 2016, 5:52 PM · Restricted Project, Storage manager
zack removed projects from T61: loading: trigger to update occurrence table: Developers, Staff.
Mar 10 2016, 5:52 PM · Storage manager
zack removed projects from T62: DB schema: add directory→tarball provenance information: Staff, Developers.
Mar 10 2016, 5:52 PM · Storage manager
zack removed projects from T78: Unable to perfectly round-trip releases pointed at by occurrences: Staff, Developers.
Mar 10 2016, 5:52 PM · Storage manager
zack removed projects from T88: DB: add index on content.ctime: Developers, Staff.
Mar 10 2016, 5:52 PM · Storage manager
zack removed projects from T116: Add storage endpoints to help with repository updates: Developers, Staff.
Mar 10 2016, 5:52 PM · Storage manager
zack removed projects from T210: Unify release_get with revision_get internal api: Developers, Staff.
Mar 10 2016, 5:52 PM · Web app, Storage manager
zack removed projects from T65: Support authors with non-utf8-encoded names: Staff, Developers.
Mar 10 2016, 5:51 PM · Storage manager, Git loader
zack removed projects from T75: Check integrity of directories, revisions, and releases: Developers, Staff.
Mar 10 2016, 5:51 PM · Archive content, Restricted Project
zack removed projects from T240: content archiver: Developers, Staff.
Mar 10 2016, 5:51 PM · Storage manager
zack removed projects from T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches: Developers, Staff.
Mar 10 2016, 5:51 PM · Language-SQL, Storage manager
zack removed projects from T299: Return the person's identifier along the person's data: Developers, Staff.
Mar 10 2016, 5:51 PM · Storage manager, Web app
zack removed projects from T304: content integrity checker: Developers, Staff.
Mar 10 2016, 5:51 PM · Storage manager
zack removed a project from T103: Add synthetic flag in release/revision tables: Staff.
Mar 10 2016, 5:49 PM · Storage manager
zack removed a project from T121: Populate fetch_history: Staff.
Mar 10 2016, 5:49 PM · Storage manager
zack removed a project from T111: Performance analysis of read queries: Staff.
Mar 10 2016, 5:49 PM · Storage manager
zack removed a project from T163: Add entry points to create / request entities in swh.storage: Staff.
Mar 10 2016, 5:49 PM · Storage manager
zack removed a project from T209: Retrieve content with other checksums than sha1: Staff.
Mar 10 2016, 5:49 PM · Storage manager
zack removed a project from T216: /api/1/revision/<sha1> - Single revision with no parent: Staff.
Mar 10 2016, 5:49 PM · Storage manager, Web app
zack removed a project from T269: Deal nicely with swh-storage's backend down time in clients: Staff.
Mar 10 2016, 5:49 PM · Web app, Storage manager
zack removed a project from T264: directory_get: Retrieve information on a directory per hash.: Staff.
Mar 10 2016, 5:49 PM · Web app, Storage manager
zack removed a project from T272: Fix double-encoded release.name entries: Staff.
Mar 10 2016, 5:49 PM · Data Model, Storage manager
zack removed a project from T308: retrieve content from s3 and store it in SWH storage: Staff.
Mar 10 2016, 5:49 PM · Storage manager, Antelink loader

Mar 4 2016

zack closed T62: DB schema: add directory→tarball provenance information as Resolved.

This has been fixed a while ago, for both debian and tarball ingestion.

Mar 4 2016, 6:26 PM · Storage manager

Feb 29 2016

olasd closed T61: loading: trigger to update occurrence table as Resolved.

Done during the big postgres 9.5 upgrade window.

Feb 29 2016, 1:58 PM · Storage manager

Feb 23 2016

ardumont merged T319: S3 content files downloader and injection in swh into T308: retrieve content from s3 and store it in SWH storage.
Feb 23 2016, 4:12 PM · Storage manager, Antelink loader

Feb 22 2016

olasd set the image for Storage manager to Unknown Object (File).
Feb 22 2016, 8:19 PM

Feb 17 2016

olasd added a comment to T75: Check integrity of directories, revisions, and releases.
In T75#3503, @olasd wrote:

It currently breaks on *completely* empty messages, but the patch seems fairly simple.

Feb 17 2016, 12:18 PM · Archive content, Restricted Project

Feb 9 2016

ardumont renamed T308: retrieve content from s3 and store it in SWH storage from swh-loader-antelink bootstrap - Retrieve content from s3 and store inside swh-storage to Retrieve content from s3 and store inside swh-storage.
Feb 9 2016, 12:29 PM · Storage manager, Antelink loader
ardumont updated the task description for T308: retrieve content from s3 and store it in SWH storage.
Feb 9 2016, 12:20 PM · Storage manager, Antelink loader
ardumont added a project to T308: retrieve content from s3 and store it in SWH storage: Storage manager.
Feb 9 2016, 12:18 PM · Storage manager, Antelink loader
ardumont added a project to T309: Delete duplicated antelink/antepedia content from s3: Storage manager.
Feb 9 2016, 12:18 PM · Antelink loader

Feb 4 2016

zack added a project to T240: content archiver: Storage manager.
Feb 4 2016, 3:00 PM · Storage manager
zack created T304: content integrity checker.
Feb 4 2016, 3:00 PM · Storage manager

Jan 29 2016

ardumont closed T299: Return the person's identifier along the person's data as Resolved by committing rDWAPPS4f57e6862526: Returns person's identifier on api + Hide person's emails in views endpoint.
Jan 29 2016, 5:43 PM · Storage manager, Web app
ardumont claimed T299: Return the person's identifier along the person's data.
Jan 29 2016, 3:25 PM · Storage manager, Web app

Jan 28 2016

olasd closed T72: occurrence history table: use an array of timestamps instead of intervals as Resolved.

This schema change is now done in production.

Jan 28 2016, 5:23 PM · Storage manager
ardumont added a parent task for T72: occurrence history table: use an array of timestamps instead of intervals: T286: view /browse/origin/ORIGIN_ID/ - calendar-like UI.
Jan 28 2016, 4:21 PM · Storage manager

Jan 27 2016

ardumont closed T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches as Resolved by committing rDSTO78024bf20d6f: Deal with occurrence_history.branch, release.name as bytes.
Jan 27 2016, 3:16 PM · Language-SQL, Storage manager
ardumont claimed T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches.
Jan 27 2016, 12:01 PM · Language-SQL, Storage manager
zack added a comment to T72: occurrence history table: use an array of timestamps instead of intervals.
In T72#3501, @olasd wrote:

full SQL code with the new schemata for origin_visit, occurrence_history and occurrence. Those three tables are implicitly relevant only for the "Software Heritage" authority.

Jan 27 2016, 10:40 AM · Storage manager

Jan 26 2016

zack added a comment to T75: Check integrity of directories, revisions, and releases.
In T75#3505, @olasd wrote:

Reading the dulwich code a bit further, it turns out that git commits can have more header attributes than we initally expected.

Jan 26 2016, 10:45 PM · Archive content, Restricted Project
olasd added a comment to T75: Check integrity of directories, revisions, and releases.

Reading the dulwich code a bit further, it turns out that git commits can have more header attributes than we initally expected.

Jan 26 2016, 4:26 PM · Archive content, Restricted Project
olasd added a comment to T75: Check integrity of directories, revisions, and releases.

Dulwich seems to handle some of those special cases just fine.

Jan 26 2016, 12:59 PM · Archive content, Restricted Project
olasd added a comment to T72: occurrence history table: use an array of timestamps instead of intervals.

full SQL code with the new schemata for origin_visit, occurrence_history and occurrence. Those three tables are implicitly relevant only for the "Software Heritage" authority.

Jan 26 2016, 12:16 PM · Storage manager
rdicosmo added a comment to T78: Unable to perfectly round-trip releases pointed at by occurrences.

Roberto Di Cosmo (via mobile/cell)
Le 26 janv. 2016 07:47, "olasd (Nicolas Dandrimont)" <
forge@softwareheritage.org> a écrit :

Jan 26 2016, 9:10 AM · Storage manager
olasd closed T78: Unable to perfectly round-trip releases pointed at by occurrences as Resolved.

This has now been deployed in swh.storage v0.0.30: occurrences and releases can now point to arbitrary objects.

Jan 26 2016, 7:46 AM · Storage manager

Jan 22 2016

olasd closed T272: Fix double-encoded release.name entries as Resolved.

This has now been done.

Jan 22 2016, 6:31 PM · Data Model, Storage manager
olasd added a comment to T72: occurrence history table: use an array of timestamps instead of intervals.

Still thinking about this .

Jan 22 2016, 12:13 PM · Storage manager

Jan 21 2016

olasd added a comment to T272: Fix double-encoded release.name entries.

Currentlly running

Jan 21 2016, 8:51 PM · Data Model, Storage manager
olasd added a comment to T75: Check integrity of directories, revisions, and releases.

I just noticed that empty messages with empty lines are stored as an empty bytea, whereas empty messages without the empty line are stored as NULL. So there's that.

Jan 21 2016, 8:40 PM · Archive content, Restricted Project
olasd added a comment to T75: Check integrity of directories, revisions, and releases.

Some example releases:

Jan 21 2016, 8:22 PM · Archive content, Restricted Project
olasd changed the status of T75: Check integrity of directories, revisions, and releases from Open to Work in Progress.

I have done some investigations on this in light of T272. Bottom line: not good: git is very proficient in the corner cases department.

Jan 21 2016, 7:41 PM · Archive content, Restricted Project
olasd created T272: Fix double-encoded release.name entries.
Jan 21 2016, 4:38 PM · Data Model, Storage manager
ardumont added a parent task for T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches: T271: Update clients on impacts + upgrade respective package dependencies.
Jan 21 2016, 4:33 PM · Language-SQL, Storage manager
First
Prev
Next
About · Developer information · Legal