Page MenuHomeSoftware Heritage
Feed Advanced Search

Jan 22 2018

olasd closed T936: Deploy a storage backend on azure, a subtask of T888: Deploy the Vault and a DB replica on Azure, as Resolved.
Jan 22 2018, 8:51 PM · System administration, Restricted Project, Vault

Jan 17 2018

seirl closed T913: documentation for the vault as Resolved.

Review tracking in T937

Jan 17 2018, 2:44 PM · Web app, Documentation, Vault, Restricted Project
seirl moved T913: documentation for the vault from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 17 2018, 2:42 PM · Web app, Documentation, Vault, Restricted Project
seirl moved T894: Vault: add limits for the max size of the objects that can be cooked from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 17 2018, 2:42 PM · Restricted Project, Vault
seirl moved T894: Vault: add limits for the max size of the objects that can be cooked from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 17 2018, 2:42 PM · Restricted Project, Vault

Jan 12 2018

seirl added a revision to T928: Rewrite the Vault Cookers I/O pipeline with file objects: D286: cookers: rewrite the I/O pipeline with file objects instead of generators.
Jan 12 2018, 3:53 PM · Vault
seirl triaged T928: Rewrite the Vault Cookers I/O pipeline with file objects as Normal priority.
Jan 12 2018, 3:03 PM · Vault
seirl added a parent task for T928: Rewrite the Vault Cookers I/O pipeline with file objects: T885: Vault: use objstorage streaming to store and fetch bundles.
Jan 12 2018, 2:49 PM · Vault
seirl added a subtask for T885: Vault: use objstorage streaming to store and fetch bundles: T928: Rewrite the Vault Cookers I/O pipeline with file objects.
Jan 12 2018, 2:49 PM · Vault
seirl created T928: Rewrite the Vault Cookers I/O pipeline with file objects.
Jan 12 2018, 2:49 PM · Vault
seirl moved T894: Vault: add limits for the max size of the objects that can be cooked from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 12 2018, 2:42 PM · Restricted Project, Vault
seirl moved T743: Vault: polish web-ui API from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 12 2018, 2:42 PM · Restricted Project, Web app, Vault
seirl moved T845: Failure mode for Vault tasks from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 12 2018, 2:42 PM · Restricted Project, Vault
seirl closed T894: Vault: add limits for the max size of the objects that can be cooked as Resolved.

Done in rDVAUe8419e9b9f16 (at least the "just in time" version, the other is more a research project than a release blocker...)

Jan 12 2018, 2:39 PM · Restricted Project, Vault
seirl added a revision to T894: Vault: add limits for the max size of the objects that can be cooked: D285: cookers: add a max bundle size.
Jan 12 2018, 2:38 PM · Restricted Project, Vault
seirl closed T845: Failure mode for Vault tasks as Resolved.

For now we allow to retry as soon as the bundle status is set as "failed". rDVAU3a3c337107e6 implements the failure state.

Jan 12 2018, 1:09 PM · Restricted Project, Vault

Jan 11 2018

seirl closed T890: Vault web API: don't schedule cooking tasks with GET requests as Resolved.
Jan 11 2018, 2:35 PM · Web app, Vault

Jan 10 2018

seirl added a revision to T845: Failure mode for Vault tasks: D284: Add failure state for bundle tasks.
Jan 10 2018, 5:31 PM · Restricted Project, Vault

Jan 9 2018

seirl closed T743: Vault: polish web-ui API as Resolved.

New URL scheme was done in rDWAPPS195058a21902, dispatch between GET and POST in rDWAPPS03406f1d07af , and I made some changes in the Vault to behave nicely with the RPC and the documented exceptions in rDVAUd95bf6d37bec .

Jan 9 2018, 4:37 PM · Restricted Project, Web app, Vault
zack moved T888: Deploy the Vault and a DB replica on Azure from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 9 2018, 12:46 PM · System administration, Restricted Project, Vault
zack moved T743: Vault: polish web-ui API from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 9 2018, 12:45 PM · Restricted Project, Web app, Vault
zack assigned T894: Vault: add limits for the max size of the objects that can be cooked to seirl.
Jan 9 2018, 12:43 PM · Restricted Project, Vault

Jan 8 2018

seirl added a project to T743: Vault: polish web-ui API: Restricted Project.
Jan 8 2018, 12:45 PM · Restricted Project, Web app, Vault
seirl removed a project from T893: Vault: get progress and fetch by bundle ID: Restricted Project.
Jan 8 2018, 12:41 PM · Vault

Jan 4 2018

seirl closed T706: establish openvpn link with unibo for the vault as Resolved.
Jan 4 2018, 1:09 AM · Vault, System administration

Jan 3 2018

zack added a subtask for T888: Deploy the Vault and a DB replica on Azure: T883: set up a replica of the main DB on azure.
Jan 3 2018, 3:28 PM · System administration, Restricted Project, Vault
zack moved T845: Failure mode for Vault tasks from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 3 2018, 3:27 PM · Restricted Project, Vault
zack added a project to T888: Deploy the Vault and a DB replica on Azure: System administration.
Jan 3 2018, 3:26 PM · System administration, Restricted Project, Vault
zack moved T888: Deploy the Vault and a DB replica on Azure from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 3 2018, 3:26 PM · System administration, Restricted Project, Vault
zack moved T893: Vault: get progress and fetch by bundle ID from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 3 2018, 3:26 PM · Vault
zack moved T894: Vault: add limits for the max size of the objects that can be cooked from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 3 2018, 3:26 PM · Restricted Project, Vault
zack moved T913: documentation for the vault from Restricted Project Column to Restricted Project Column on the Restricted Project board.
Jan 3 2018, 3:26 PM · Web app, Documentation, Vault, Restricted Project
zack triaged T888: Deploy the Vault and a DB replica on Azure as Normal priority.
Jan 3 2018, 3:22 PM · System administration, Restricted Project, Vault
zack lowered the priority of T913: documentation for the vault from High to Normal.
Jan 3 2018, 3:19 PM · Web app, Documentation, Vault, Restricted Project
zack added projects to T913: documentation for the vault: Vault, Documentation, Web app.
Jan 3 2018, 3:19 PM · Web app, Documentation, Vault, Restricted Project
zack raised the priority of T894: Vault: add limits for the max size of the objects that can be cooked from Normal to High.
Jan 3 2018, 3:13 PM · Restricted Project, Vault
zack added a project to T845: Failure mode for Vault tasks: Restricted Project.
Jan 3 2018, 3:13 PM · Restricted Project, Vault
zack added a project to T893: Vault: get progress and fetch by bundle ID: Restricted Project.
Jan 3 2018, 3:13 PM · Vault
zack edited projects for T889: Web API: add a user-friendly way to retrieve Vault bundles in /browse, added: Web app; removed Website.
Jan 3 2018, 3:12 PM · Web app, Vault
anlambert changed the status of T889: Web API: add a user-friendly way to retrieve Vault bundles in /browse from Open to Work in Progress.
Jan 3 2018, 3:07 PM · Web app, Vault

Dec 18 2017

zack added a comment to T743: Vault: polish web-ui API.
In T743#16529, @seirl wrote:
  • shouldn't the format be a parameter instead of a part of the URL? i.e., ?format=foo instead of /foo at the end?

I see some reasons why not:

Dec 18 2017, 2:33 PM · Restricted Project, Web app, Vault
seirl added a comment to T743: Vault: polish web-ui API.
  • shouldn't the format be a parameter instead of a part of the URL? i.e., ?format=foo instead of /foo at the end?
Dec 18 2017, 1:42 PM · Restricted Project, Web app, Vault
zack updated subscribers of T743: Vault: polish web-ui API.

First of all thanks for this useful summary @seirl !

Dec 18 2017, 12:04 PM · Restricted Project, Web app, Vault

Dec 15 2017

seirl updated subscribers of T743: Vault: polish web-ui API.
Dec 15 2017, 6:26 PM · Restricted Project, Web app, Vault
seirl added a comment to T743: Vault: polish web-ui API.

My current idea for the user-facing api is below. I have highlighted in bold the
things that require decisions or comments.

Dec 15 2017, 6:25 PM · Restricted Project, Web app, Vault
olasd added a comment to T904: Factor-out common DB wrappers in swh.core.

You can add swh.indexer.storage.db.Db to your list.

Dec 15 2017, 6:11 PM · Core & foundations, Storage manager, Scheduling utilities, Vault
ardumont updated the task description for T904: Factor-out common DB wrappers in swh.core.
Dec 15 2017, 5:40 PM · Core & foundations, Storage manager, Scheduling utilities, Vault
seirl created T904: Factor-out common DB wrappers in swh.core.
Dec 15 2017, 5:28 PM · Core & foundations, Storage manager, Scheduling utilities, Vault
seirl assigned T889: Web API: add a user-friendly way to retrieve Vault bundles in /browse to anlambert.
Dec 15 2017, 1:57 PM · Web app, Vault
seirl closed T744: Vault: test installation on Unibo as Resolved.
Dec 15 2017, 1:49 PM · Vault
seirl triaged T894: Vault: add limits for the max size of the objects that can be cooked as Normal priority.
Dec 15 2017, 1:43 PM · Restricted Project, Vault

Dec 14 2017

seirl closed T886: Vault: use swh.scheduler oneshot tasks for cooking dispatching as Resolved.
Dec 14 2017, 9:18 PM · Vault
seirl closed T886: Vault: use swh.scheduler oneshot tasks for cooking dispatching, a subtask of T744: Vault: test installation on Unibo, as Resolved.
Dec 14 2017, 9:18 PM · Vault
seirl added a revision to T886: Vault: use swh.scheduler oneshot tasks for cooking dispatching: D278: backend: use swh.scheduler to dispatch cooking tasks.
Dec 14 2017, 4:31 PM · Vault

Dec 12 2017

seirl created T894: Vault: add limits for the max size of the objects that can be cooked.
Dec 12 2017, 4:24 PM · Restricted Project, Vault
seirl closed T742: Vault: add backend tests as Resolved.
Dec 12 2017, 4:18 PM · Vault
seirl added a parent task for T893: Vault: get progress and fetch by bundle ID: T892: Vault: cooking with additional options.
Dec 12 2017, 4:12 PM · Vault
seirl added a subtask for T892: Vault: cooking with additional options: T893: Vault: get progress and fetch by bundle ID.
Dec 12 2017, 4:12 PM · Vault
seirl raised the priority of T893: Vault: get progress and fetch by bundle ID from Low to Normal.
Dec 12 2017, 4:12 PM · Vault
seirl created T893: Vault: get progress and fetch by bundle ID.
Dec 12 2017, 4:12 PM · Vault
seirl added a parent task for T892: Vault: cooking with additional options: T891: Vault: cooking with --depth option.
Dec 12 2017, 4:09 PM · Vault
seirl added a subtask for T891: Vault: cooking with --depth option: T892: Vault: cooking with additional options.
Dec 12 2017, 4:09 PM · Vault
seirl created T892: Vault: cooking with additional options.
Dec 12 2017, 4:08 PM · Vault
seirl created T891: Vault: cooking with --depth option.
Dec 12 2017, 4:06 PM · Vault
seirl created T890: Vault web API: don't schedule cooking tasks with GET requests.
Dec 12 2017, 4:03 PM · Web app, Vault
seirl created T889: Web API: add a user-friendly way to retrieve Vault bundles in /browse.
Dec 12 2017, 4:02 PM · Web app, Vault
seirl added a subtask for T744: Vault: test installation on Unibo: T886: Vault: use swh.scheduler oneshot tasks for cooking dispatching.
Dec 12 2017, 3:51 PM · Vault
seirl added a parent task for T886: Vault: use swh.scheduler oneshot tasks for cooking dispatching: T744: Vault: test installation on Unibo.
Dec 12 2017, 3:51 PM · Vault
seirl added a comment to T744: Vault: test installation on Unibo.

The current install works but is very clunky, mostly because we're not using swh.scheduler oneshot tasks. I'm adding that as a dependency.

Dec 12 2017, 3:50 PM · Vault
seirl closed T532: Vault API, a subtask of T530: Software Heritage Vault, as Resolved.
Dec 12 2017, 3:47 PM · Vault
seirl closed T532: Vault API as Resolved.
Dec 12 2017, 3:47 PM · Vault
seirl added a comment to T532: Vault API.

The internal API has been done and in use for a while, so I'm closing this task.

Dec 12 2017, 3:47 PM · Vault
seirl added a subtask for T887: Vault: "snapshot" cooker: T565: embrace repository snapshot object in the data model (meta task).
Dec 12 2017, 3:45 PM · Vault
seirl created T887: Vault: "snapshot" cooker.
Dec 12 2017, 3:45 PM · Vault
seirl closed T531: Vault cookers, a subtask of T530: Software Heritage Vault, as Resolved.
Dec 12 2017, 3:45 PM · Vault
seirl closed T531: Vault cookers as Resolved.

I'm closing this and adding a new snapshot specific task.

Dec 12 2017, 3:45 PM · Vault
seirl closed T530: Software Heritage Vault, a subtask of T508: prototype: git archive from SWH, as Resolved.
Dec 12 2017, 3:44 PM · Vault, General
seirl closed T530: Software Heritage Vault as Resolved.

The Vault prototype is "done", now the other remaining tasks should be task roots with the Vault project tag. I'm closing this meta-task.

Dec 12 2017, 3:44 PM · Vault
seirl closed T508: prototype: git archive from SWH, a subtask of T67: prototype: git clone from SWH, as Resolved.
Dec 12 2017, 3:43 PM · Vault, General
seirl closed T508: prototype: git archive from SWH as Resolved.

This is already well beyond a prototype, so I'm closing this task.

Dec 12 2017, 3:43 PM · Vault, General
seirl closed T67: prototype: git clone from SWH as Resolved.

As the "prototype" part of the vault is definitely over, I'm closing this task.

Dec 12 2017, 3:42 PM · Vault, General
seirl closed T842: Add tests for the vault views in the web app as Resolved.
Dec 12 2017, 3:31 PM · Web app, Vault
seirl triaged T886: Vault: use swh.scheduler oneshot tasks for cooking dispatching as High priority.
Dec 12 2017, 3:29 PM · Vault
seirl created T886: Vault: use swh.scheduler oneshot tasks for cooking dispatching.
Dec 12 2017, 3:27 PM · Vault
seirl added a subtask for T885: Vault: use objstorage streaming to store and fetch bundles: T805: objstorage: allow use of file-like objects for streaming methods.
Dec 12 2017, 3:17 PM · Vault
seirl created T885: Vault: use objstorage streaming to store and fetch bundles.
Dec 12 2017, 3:16 PM · Vault

Dec 8 2017

seirl added a comment to T706: establish openvpn link with unibo for the vault.

It works perfectly, thank you!

Dec 8 2017, 1:25 PM · Vault, System administration

Dec 7 2017

cmezzett added a comment to T706: establish openvpn link with unibo for the vault.

we had to change a setting on the storage device, it should now work as expected.

Dec 7 2017, 6:31 PM · Vault, System administration

Dec 6 2017

seirl added a comment to T706: establish openvpn link with unibo for the vault.

I have a problem with the NFS mountpoint: I'm using a non-root user for the service that reads and write on the vault, but I can't write on the NFS mountpoint. When I create a directory with the root user on the mountpoint, it gets created with uid/gid = 4294967294 (2^32-2), and I can't chown them to an uid/gid that my user would be able to write to:

Dec 6 2017, 5:55 PM · Vault, System administration

Dec 5 2017

seirl created P199 Vault task in scheduler.
Dec 5 2017, 11:35 AM · Vault

Nov 13 2017

seirl updated the task description for T845: Failure mode for Vault tasks.
Nov 13 2017, 2:03 PM · Restricted Project, Vault
seirl triaged T845: Failure mode for Vault tasks as Normal priority.
Nov 13 2017, 1:32 PM · Restricted Project, Vault
seirl created T845: Failure mode for Vault tasks.
Nov 13 2017, 1:31 PM · Restricted Project, Vault

Nov 8 2017

seirl added a comment to T843: Vault: Add a "git bare" tarball cooker.

That was the plan, yes.

Nov 8 2017, 6:36 PM · Vault
zack added a comment to T843: Vault: Add a "git bare" tarball cooker.

And what would be the actual on-the-wire serialization format used to send the bare repo to users? Some sort of archive of the .git dir or what?

Nov 8 2017, 10:29 AM · Vault

Nov 7 2017

seirl triaged T843: Vault: Add a "git bare" tarball cooker as Wishlist priority.
Nov 7 2017, 7:06 PM · Vault
seirl created T843: Vault: Add a "git bare" tarball cooker.
Nov 7 2017, 7:06 PM · Vault
seirl closed T745: Vault: Add cache expiration feature as Resolved.
Nov 7 2017, 6:59 PM · Vault
seirl added a comment to T743: Vault: polish web-ui API.

What is needed now is mainly to change the way formats are given in parameter (revision_gitfast/<hash> -> revision/<hash>/gitfast maybe?)

Nov 7 2017, 6:58 PM · Restricted Project, Web app, Vault
seirl closed T747: Vault: add cache tests as Resolved.
Nov 7 2017, 6:56 PM · Vault