Page MenuHomeSoftware Heritage
Feed Advanced Search

Advanced Search
Use Results
Hide Query

Jan 28 2016

olasd closed T72: occurrence history table: use an array of timestamps instead of intervals as Resolved.

This schema change is now done in production.

Jan 28 2016, 5:23 PM · Storage manager
olasd closed T72: occurrence history table: use an array of timestamps instead of intervals, a subtask of T286: view /browse/origin/ORIGIN_ID/ - calendar-like UI, as Resolved.
Jan 28 2016, 5:23 PM · Web app
ardumont created T301: Improve date parsing error messages.
Jan 28 2016, 4:37 PM · Web app
ardumont added a project to T300: Hide person's email on revision/release/person view: Web app.
Jan 28 2016, 4:36 PM · Web app
ardumont created T299: Return the person's identifier along the person's data.
Jan 28 2016, 4:32 PM · Storage manager, Web app
ardumont added a parent task for T298: Add /api/1/revision/<sha1_git>/raw/: T297: Try to decode the revision's message data and fail gracefully.
Jan 28 2016, 4:30 PM · Web app
ardumont added a subtask for T297: Try to decode the revision's message data and fail gracefully: T298: Add /api/1/revision/<sha1_git>/raw/.
Jan 28 2016, 4:30 PM · Web app
ardumont created T298: Add /api/1/revision/<sha1_git>/raw/.
Jan 28 2016, 4:30 PM · Web app
ardumont created T297: Try to decode the revision's message data and fail gracefully.
Jan 28 2016, 4:29 PM · Web app
ardumont created T296: Try to decode the content's raw data and fail gracefully.
Jan 28 2016, 4:28 PM · Web app
ardumont created T295: Update /browse/directory/<path>/ to show content when path resolves to a content.
Jan 28 2016, 4:23 PM · Web app
ardumont added a subtask for T286: view /browse/origin/ORIGIN_ID/ - calendar-like UI: T72: occurrence history table: use an array of timestamps instead of intervals.
Jan 28 2016, 4:21 PM · Web app
ardumont added a parent task for T72: occurrence history table: use an array of timestamps instead of intervals: T286: view /browse/origin/ORIGIN_ID/ - calendar-like UI.
Jan 28 2016, 4:21 PM · Storage manager
ardumont added a subtask for T294: Align /browse/directory/<sha1_git>/ template to other /directory/ apis: T293: Open /browse/directory/<sha1_git>/[<path>/] view.
Jan 28 2016, 4:20 PM · Web app
ardumont added a parent task for T293: Open /browse/directory/<sha1_git>/[<path>/] view: T294: Align /browse/directory/<sha1_git>/ template to other /directory/ apis.
Jan 28 2016, 4:20 PM · Web app
ardumont created T294: Align /browse/directory/<sha1_git>/ template to other /directory/ apis.
Jan 28 2016, 4:20 PM · Web app
ardumont created T293: Open /browse/directory/<sha1_git>/[<path>/] view.
Jan 28 2016, 4:18 PM · Web app
ardumont created T292: Open /api/1/directory/<sha1_git>/[<path>/] api.
Jan 28 2016, 4:17 PM · Web app
ardumont created T291: Remove default identifier on api/view.
Jan 28 2016, 4:13 PM · Web app
ardumont created T290: Open /api/1/revision/<sha1_git_root>/history/<sha1_git>/log/ api.
Jan 28 2016, 4:11 PM · Web app
ardumont created T289: Open /browse/revision/<sha1_git_root>/history/<sha1_git>/log/ view.
Jan 28 2016, 4:10 PM · Web app
ardumont created T287: Open /browse/revision/origin/<origin_id>/[branch/<branch_name>][ts/<ts>]/log/ view.
Jan 28 2016, 4:09 PM · Web app
ardumont created T286: view /browse/origin/ORIGIN_ID/ - calendar-like UI.
Jan 28 2016, 4:07 PM · Web app

Jan 27 2016

ardumont renamed T271: Update clients on impacts + upgrade respective package dependencies from Update api conversion on impacted fields to Update clients on impacts + upgrade respective package dependencies.
Jan 27 2016, 4:51 PM · Git loader, Directory loader, Data Model, Web app
ardumont closed T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches as Resolved by committing rDSTO78024bf20d6f: Deal with occurrence_history.branch, release.name as bytes.
Jan 27 2016, 3:16 PM · Language-SQL, Storage manager
ardumont closed T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches, a subtask of T271: Update clients on impacts + upgrade respective package dependencies, as Resolved.
Jan 27 2016, 3:16 PM · Git loader, Directory loader, Data Model, Web app
ardumont claimed T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches.
Jan 27 2016, 12:01 PM · Language-SQL, Storage manager
zack added a comment to T72: occurrence history table: use an array of timestamps instead of intervals.
In T72#3501, @olasd wrote:

full SQL code with the new schemata for origin_visit, occurrence_history and occurrence. Those three tables are implicitly relevant only for the "Software Heritage" authority.

Jan 27 2016, 10:40 AM · Storage manager

Jan 26 2016

zack added a comment to T75: Check integrity of directories, revisions, and releases.
In T75#3505, @olasd wrote:

Reading the dulwich code a bit further, it turns out that git commits can have more header attributes than we initally expected.

Jan 26 2016, 10:45 PM · Archive content, Restricted Project
olasd added a comment to T75: Check integrity of directories, revisions, and releases.

Reading the dulwich code a bit further, it turns out that git commits can have more header attributes than we initally expected.

Jan 26 2016, 4:26 PM · Archive content, Restricted Project
olasd added a comment to T75: Check integrity of directories, revisions, and releases.

Dulwich seems to handle some of those special cases just fine.

Jan 26 2016, 12:59 PM · Archive content, Restricted Project
olasd added a comment to T72: occurrence history table: use an array of timestamps instead of intervals.

full SQL code with the new schemata for origin_visit, occurrence_history and occurrence. Those three tables are implicitly relevant only for the "Software Heritage" authority.

Jan 26 2016, 12:16 PM · Storage manager
rdicosmo added a comment to T78: Unable to perfectly round-trip releases pointed at by occurrences.

Roberto Di Cosmo (via mobile/cell)
Le 26 janv. 2016 07:47, "olasd (Nicolas Dandrimont)" <
forge@softwareheritage.org> a écrit :

Jan 26 2016, 9:10 AM · Storage manager
olasd closed T78: Unable to perfectly round-trip releases pointed at by occurrences as Resolved.

This has now been deployed in swh.storage v0.0.30: occurrences and releases can now point to arbitrary objects.

Jan 26 2016, 7:46 AM · Storage manager
olasd closed T78: Unable to perfectly round-trip releases pointed at by occurrences, a subtask of T67: prototype: git clone from SWH, as Resolved.
Jan 26 2016, 7:46 AM · Vault, General

Jan 25 2016

ardumont closed T280: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/history/<SHA1>/directory/[<PATH>] view as Resolved by committing rDWAPPS0dc8a834876b: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>]….
Jan 25 2016, 4:53 PM · Web app
ardumont closed T285: Open /browse/revision/<sha1_git>/log/ as Resolved by committing rDWAPPS16f97f8d8e80: Open /browse/revision/<sha1_git>/log/.
Jan 25 2016, 3:53 PM · Web app
ardumont created T285: Open /browse/revision/<sha1_git>/log/.
Jan 25 2016, 3:52 PM · Web app
ardumont closed T281: Open /revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/directory/[<PATH>] view as Resolved by committing rDWAPPS43f9c827cd45: Open /revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/directory/….
Jan 25 2016, 2:04 PM · Web app
ardumont closed T278: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/ view as Resolved by committing rDWAPPSdb71b926e66a: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/ view.
Jan 25 2016, 10:23 AM · Web app
ardumont closed T279: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/history/<SHA1>/ view as Resolved by committing rDWAPPSb5dd1178cc2f: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>]….
Jan 25 2016, 10:23 AM · Web app

Jan 22 2016

olasd closed T272: Fix double-encoded release.name entries as Resolved.

This has now been done.

Jan 22 2016, 6:31 PM · Data Model, Storage manager
ardumont claimed T278: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/ view.
Jan 22 2016, 5:14 PM · Web app
ardumont closed T277: Open /browse/revision/<SHA1_GIT_ROOT>/history/<SHA1_GIT>/directory/[<PATH>] view as Resolved by committing rDWAPPSa4f574de5736: Open /browse/revision/<SHA1_GIT_ROOT>/history/<SHA1_GIT>/directory/[<PATH>] view.
Jan 22 2016, 5:09 PM · Web app
ardumont closed T276: Open /browse/revision/<sha1_git_root>/directory/<path>/ view as Resolved by committing rDWAPPS45720eb71a04: Open /browse/revision/<sha1_git_root>/directory/<path>/.
Jan 22 2016, 2:58 PM · Web app
olasd added a comment to T72: occurrence history table: use an array of timestamps instead of intervals.

Still thinking about this .

Jan 22 2016, 12:13 PM · Storage manager
ardumont closed T275: Open /browse/revision/<sha1_git_root>/history/<sha1_git>/ view as Resolved by committing rDWAPPS9fd582d85697: Open /browse/revision/<sha1_git_root>/history/<sha1_git>/.
Jan 22 2016, 11:52 AM · Web app
ardumont created T281: Open /revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/directory/[<PATH>] view.
Jan 22 2016, 11:51 AM · Web app
ardumont created T280: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/history/<SHA1>/directory/[<PATH>] view.
Jan 22 2016, 11:50 AM · Web app
ardumont created T279: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/history/<SHA1>/ view.
Jan 22 2016, 11:50 AM · Web app
ardumont created T278: Open /browse/revision/origin/<ORIG_ID>[/branch/<BRANCH>][/ts/<TIMESTAMP>]/ view.
Jan 22 2016, 11:49 AM · Web app
ardumont created T277: Open /browse/revision/<SHA1_GIT_ROOT>/history/<SHA1_GIT>/directory/[<PATH>] view.
Jan 22 2016, 11:48 AM · Web app
ardumont created T276: Open /browse/revision/<sha1_git_root>/directory/<path>/ view.
Jan 22 2016, 11:46 AM · Web app
ardumont created T275: Open /browse/revision/<sha1_git_root>/history/<sha1_git>/ view.
Jan 22 2016, 11:45 AM · Web app
ardumont changed the status of T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too) from Work in Progress to Open.
Jan 22 2016, 10:05 AM · Git cloner, Git loader
ardumont changed the status of T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too), a subtask of T66: clone and load fork GitHub repositories, from Work in Progress to Open.
Jan 22 2016, 10:05 AM · Restricted Project, General

Jan 21 2016

olasd added a comment to T272: Fix double-encoded release.name entries.

Currentlly running

Jan 21 2016, 8:51 PM · Data Model, Storage manager
olasd added a comment to T75: Check integrity of directories, revisions, and releases.

I just noticed that empty messages with empty lines are stored as an empty bytea, whereas empty messages without the empty line are stored as NULL. So there's that.

Jan 21 2016, 8:40 PM · Archive content, Restricted Project
olasd added a comment to T75: Check integrity of directories, revisions, and releases.

Some example releases:

Jan 21 2016, 8:22 PM · Archive content, Restricted Project
olasd changed the status of T75: Check integrity of directories, revisions, and releases from Open to Work in Progress.

I have done some investigations on this in light of T272. Bottom line: not good: git is very proficient in the corner cases department.

Jan 21 2016, 7:41 PM · Archive content, Restricted Project
zack added a comment to T273: Darcs loader.

As a starting point, I've briefly discussed with Darcs developers how the push/pull protocol works in Darcs. Unfortunately the protocol doesn't seem to be documented anywhere. The relevant entry points in the code are:

Jan 21 2016, 5:17 PM · General
zack created T273: Darcs loader.
Jan 21 2016, 5:03 PM · General
olasd created T272: Fix double-encoded release.name entries.
Jan 21 2016, 4:38 PM · Data Model, Storage manager
ardumont added a subtask for T271: Update clients on impacts + upgrade respective package dependencies: T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches.
Jan 21 2016, 4:33 PM · Git loader, Directory loader, Data Model, Web app
ardumont added a parent task for T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches: T271: Update clients on impacts + upgrade respective package dependencies.
Jan 21 2016, 4:33 PM · Language-SQL, Storage manager
ardumont removed a subtask for T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches: T271: Update clients on impacts + upgrade respective package dependencies.
Jan 21 2016, 4:32 PM · Language-SQL, Storage manager
ardumont removed a parent task for T271: Update clients on impacts + upgrade respective package dependencies: T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches.
Jan 21 2016, 4:32 PM · Git loader, Directory loader, Data Model, Web app
ardumont created T271: Update clients on impacts + upgrade respective package dependencies.
Jan 21 2016, 4:30 PM · Git loader, Directory loader, Data Model, Web app
zack created T270: TEXT → BYTEA conversion for columns that represent VCS tags/branches.
Jan 21 2016, 4:24 PM · Language-SQL, Storage manager
olasd added a comment to T72: occurrence history table: use an array of timestamps instead of intervals.

Our main query on occurrences is looking for occurrences that are

  • from a given origin
  • on a given branch (or all branches)
  • that are the newest, or the closest to a given timestamp.
Jan 21 2016, 12:08 PM · Storage manager
ardumont added a comment to T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too).

For information, sample test_update.py adapted in swh-loader-git https://forge.softwareheritage.org/diffusion/DLDG/browse/master/swh/loader/git/updater.py to use the swh-storage.

Jan 21 2016, 12:08 PM · Git cloner, Git loader
ardumont closed T269: Deal nicely with swh-storage's backend down time in clients as Resolved.
Jan 21 2016, 12:08 PM · Web app, Storage manager
ardumont changed the status of T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too), a subtask of T66: clone and load fork GitHub repositories, from Open to Work in Progress.
Jan 21 2016, 12:08 PM · Restricted Project, General
ardumont changed the status of T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too) from Open to Work in Progress.
Jan 21 2016, 12:08 PM · Git cloner, Git loader
ardumont renamed T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too) from smart, all-in-one git cloner/loader to smart, all-in-one git cloner/loader/ (+ dealing with updates too).
Jan 21 2016, 12:08 PM · Git cloner, Git loader
ardumont added a comment to T269: Deal nicely with swh-storage's backend down time in clients.

+ v0.0.21 deployed on archive

Jan 21 2016, 12:08 PM · Web app, Storage manager
ardumont added a comment to T269: Deal nicely with swh-storage's backend down time in clients.
  • 68a8283 Deal nicely with communication downtime with storage
  • 3afbd2d Deal more appropriately with storage error
Jan 21 2016, 12:08 PM · Web app, Storage manager
ardumont created T269: Deal nicely with swh-storage's backend down time in clients.
Jan 21 2016, 12:08 PM · Web app, Storage manager
ardumont added a comment to T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too).

Related but not limited to:
58903e5 * origin/master origin/HEAD Open occurrence_get(origin_id) to retrieve latest occurrences per origin
bc23eb9 * sql/upgrades/043: add 042→043 upgrade script
d05afde * revision_log from multiple root revisions
3a40f00 * sql/upgrades/042: add 041→042 upgrade script
f54fd8d * Open release_get_by to retrieve a release by origin.
5dc4244 * revision_get_by: branch name filtering is optional
7e623c8 * sql/upgrades/040: add 040→041 upgrade script
7e2dcbc * Open directory_get to retrieve information on directory by id

Jan 21 2016, 12:08 PM · Git cloner, Git loader

Jan 18 2016

ardumont closed T163: Add entry points to create / request entities in swh.storage as Resolved.
Jan 18 2016, 4:09 PM · Storage manager
ardumont closed T163: Add entry points to create / request entities in swh.storage, a subtask of T164: Port GitHub lister to swh.storage instead of sqlalchemy, as Resolved.
Jan 18 2016, 4:09 PM · GitHub lister
ardumont closed T166: functional tests for the web UI, with actual data as Resolved.
Jan 18 2016, 4:08 PM · Web app
ardumont created T264: directory_get: Retrieve information on a directory per hash..
Jan 18 2016, 2:44 PM · Web app, Storage manager
ardumont added a comment to T264: directory_get: Retrieve information on a directory per hash..

storage:

  • 7e623c8 * master origin/master origin/HEAD sql/upgrades/040: add 040→041 upgrade script
  • 7e2dcbc * Open directory_get to retrieve information on directory by id
  • ac380c9 * Rename directory_get to directory_ls
Jan 18 2016, 2:39 PM · Web app, Storage manager
ardumont closed T116: Add storage endpoints to help with repository updates, a subtask of T51: smart, all-in-one git cloner/loader/ (+ dealing with updates too), as Resolved.
Jan 18 2016, 2:39 PM · Git cloner, Git loader
ardumont closed T116: Add storage endpoints to help with repository updates as Resolved.
Jan 18 2016, 2:39 PM · Storage manager
ardumont changed the status of T264: directory_get: Retrieve information on a directory per hash. from Open to Work in Progress.
Jan 18 2016, 2:39 PM · Web app, Storage manager
ardumont closed T264: directory_get: Retrieve information on a directory per hash., a subtask of T116: Add storage endpoints to help with repository updates, as Resolved.
Jan 18 2016, 2:39 PM · Storage manager
ardumont changed the status of T264: directory_get: Retrieve information on a directory per hash., a subtask of T116: Add storage endpoints to help with repository updates, from Open to Work in Progress.
Jan 18 2016, 2:39 PM · Storage manager
ardumont closed T264: directory_get: Retrieve information on a directory per hash. as Resolved.
Jan 18 2016, 2:39 PM · Web app, Storage manager
ardumont closed T206: /api/1/revision/?timestamp=<TIMESTAMP>&origin=<ORIGIN> as Invalid.
Jan 18 2016, 2:39 PM · Web app
ardumont closed T219: Package swh-web-ui with flask-api declared as requirement as Resolved.
Jan 18 2016, 2:39 PM · Web app
ardumont added a comment to T163: Add entry points to create / request entities in swh.storage.
  • 'Single entity query' done in 9fe94d3, ebe3a29 (entity_get)
Jan 18 2016, 2:39 PM · Storage manager
ardumont added a comment to T116: Add storage endpoints to help with repository updates.

Current status on this:

  • find objects of any type by sha1_git (release/revision/directory/content)
Jan 18 2016, 2:39 PM · Storage manager

Jan 15 2016

ardumont closed T247: Open /api/1/entity/<uuid>, a subtask of T246: Open /browse/entity/<uuid> view, as Resolved.
Jan 15 2016, 4:44 PM · Web app
ardumont closed T247: Open /api/1/entity/<uuid> as Resolved by committing rDWAPPS9b59b77bd9d0: Open /api/1/entity/<string:uuid>/.
Jan 15 2016, 4:44 PM · Web app
ardumont changed the status of T247: Open /api/1/entity/<uuid> from Open to Work in Progress.
Jan 15 2016, 3:44 PM · Web app
ardumont changed the status of T247: Open /api/1/entity/<uuid>, a subtask of T246: Open /browse/entity/<uuid> view, from Open to Work in Progress.
Jan 15 2016, 3:44 PM · Web app
zack added a comment to T78: Unable to perfectly round-trip releases pointed at by occurrences.

What about having "occurrences" for all kinds of objects in a VCS, releases, tags, revisions, etc. ?

Jan 15 2016, 9:28 AM · Storage manager
rdicosmo added a comment to T78: Unable to perfectly round-trip releases pointed at by occurrences.

What about having "occurrences" for all kinds of objects in a VCS, releases, tags, revisions, etc. ?
We would definitely need to look at other VCS to get a general model; for example, in Darcs patches are first class citizens: do we have a way of accomodating this in our data model?

Jan 15 2016, 9:08 AM · Storage manager
First
Prev
Next
About · Developer information · Legal