This has actually been in production for quite a while. Some related tasks are still pending, but there is no need to keep the general "we need a content archiver" task open anymore.
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed Advanced Search
Advanced Search
Advanced Search
Feb 12 2017
Feb 12 2017
zack added a project to T75: Check integrity of directories, revisions, and releases: Restricted Project.
Feb 10 2017
Feb 10 2017
zack renamed T399: (Re-)Compute data checksums before insertion from Compute checksums of data before insertion to (Re-)Compute data checksums before insertion.
Feb 9 2017
Feb 9 2017
The initial count has been done:
Feb 7 2017
Feb 7 2017
olasd changed the status of T672: Keep an up to date count of the number of objects in each archive from Open to Work in Progress.
The counting strategy has been implemented in rDSTO598114c5da.
Jan 26 2017
Jan 26 2017
ardumont removed a project from T645: Type swh-storage endpoints with swh.model objects: Restricted Project.
ardumont renamed T646: Unify duplicated code about client/server code from Remove duplicated code about client/server code to Unify duplicated code about client/server code.
ardumont closed T646: Unify duplicated code about client/server code as Resolved by committing rDSTOfed8afb66ffb: d/control: Update dependencies.
Jan 13 2017
Jan 13 2017
ardumont closed T627: /api/1/revision/origin/log/ - Add pagination as Resolved by committing rDWAPPS7631b3c40792: /api/1/revision/origin/log/: Add pagination explanation.
ardumont closed T626: /api/1/revision/log/ - Add pagination as Resolved by committing rDWAPPS5c92533e2731: /api/1/revision/log/: Add pagination explanation.
As explained on irc, there is actually a pagination.
The last element is used to create a new /revision/log/ link to follow in the 'next_urls_rev' key.
Jan 12 2017
Jan 12 2017
zack moved T627: /api/1/revision/origin/log/ - Add pagination from Restricted Project Column to Restricted Project Column on the Restricted Project board.
zack edited projects for T627: /api/1/revision/origin/log/ - Add pagination, added: Restricted Project; removed Release managers.
See comment in T626, about code reuse and whether or not remove the underlying storage manager limit.
ardumont renamed T627: /api/1/revision/origin/log/ - Add pagination from /api/1/revision/origin/log - Add pagination to /api/1/revision/origin/log/ - Add pagination.
zack moved T626: /api/1/revision/log/ - Add pagination from Restricted Project Column to Restricted Project Column on the Restricted Project board.
To this day, there is only a limit (propery configuration) to what's returned.
Remove such limit.
Jan 10 2017
Jan 10 2017
Dec 17 2016
Dec 17 2016
Deployment finished by olasd.
Dec 16 2016
Dec 16 2016
Deployment finished by olasd.
Dec 15 2016
Dec 15 2016
Nov 8 2016
Nov 8 2016
So, after discussion with @olasd:
- PartitionKey: content ID
- RowKey: {revision_timestamp}_{revision ID} (without trailing sequential no)
Nov 7 2016
Nov 7 2016
We need to generate RowKeys that are:
olasd updated the task description for T598: Store content -> revision cache in azure table storage.
Sep 29 2016
Sep 29 2016
ardumont updated the title for P104 provenance - populate caches (dev) - compatible schema v085 from provenance - populate caches (only for dev) to provenance - populate caches (dev) - compatible schema v085.
Sep 26 2016
Sep 26 2016
ardumont updated the title for P110 remains some noise left in archiver db from maybe some noise left in archiver db to remains some noise left in archiver db.
Sep 23 2016
Sep 23 2016
Sep 22 2016
Sep 22 2016
Sep 20 2016
Sep 20 2016
qcampos added a revision to T531: Vault cookers: D115: Refactor the vault cooker to add new bundle types.
Sep 15 2016
Sep 15 2016
we need to add the new table for repository snapshot objects, but also rework origin_visit & friends to cope
Sep 5 2016
Sep 5 2016
Sep 2 2016
Sep 2 2016
ardumont added a project to T559: Open /api/1/origin/<origin_id>/visits/<visit_id>/ endpoint: Storage manager.
Sep 1 2016
Sep 1 2016
Fully implemented in rDSTO963634dca6ba
Aug 30 2016
Aug 30 2016
ardumont removed parent tasks for T552: List and populate contents per revision in cache_content_revision (api endpoint): Unknown Object (Maniphest Task), T547: Azure prototype: Content provenance information API.
ardumont added a parent task for T552: List and populate contents per revision in cache_content_revision (api endpoint): Unknown Object (Maniphest Task).
ardumont renamed T552: List and populate contents per revision in cache_content_revision (api endpoint) from List and populate contents per revision in cache_content_revision to List and populate contents per revision in cache_content_revision (api endpoint).
olasd closed T550: Add cache tables for provenance information API as Resolved by committing rDSTO4ea78453aeb5: Add content provenance information cache tables.
ardumont renamed T552: List and populate contents per revision in cache_content_revision (api endpoint) from List and population contents per revision in cache_content_revision to List and populate contents per revision in cache_content_revision.
ardumont renamed T552: List and populate contents per revision in cache_content_revision (api endpoint) from List contents per revision to List and population contents per revision in cache_content_revision.
The branch information on cache_revision_origin makes the complexity and redundancy of this task explode.
zack added a parent task for T552: List and populate contents per revision in cache_content_revision (api endpoint): Unknown Object (Maniphest Task).
zack added a parent task for T550: Add cache tables for provenance information API: Unknown Object (Maniphest Task).
Aug 29 2016
Aug 29 2016
ardumont closed T539: Update loaders to register origin_visit's state, a subtask of T538: Add origin_visit API entry points to permit creation/update, as Resolved.
Aug 26 2016
Aug 26 2016
That's accurate, @ardumont. Nothing to add. Thanks for the puppet stuff!
An update on this, there are 3 checker implementations (all of them are puppetized independently):
ardumont closed T545: Create puppet manifests for the content integrity checkers, a subtask of T304: content integrity checker, as Resolved.
ardumont changed the status of T545: Create puppet manifests for the content integrity checkers, a subtask of T304: content integrity checker, from Open to Work in Progress.
Aug 24 2016
Aug 24 2016
Aug 23 2016
Aug 23 2016
ardumont updated the title for P103 [Fixed] packaging swh-storage: problem with transitive runtime dependency from swh.core on python3-yaml from packaging swh-storage: problem with transitive runtime dependency from swh.core on python3-yaml to [Fixed] packaging swh-storage: problem with transitive runtime dependency from swh.core on python3-yaml.
Aug 22 2016
Aug 22 2016
ardumont closed T538: Add origin_visit API entry points to permit creation/update, a subtask of T534: Add completion information to softwareheritage.origin_visit table, as Resolved.
Aug 19 2016
Aug 19 2016
The listening side of this (rDSTOd74b3e51f320) has now been deployed (rSPPROF2be94777ce21)
ardumont updated the task description for T538: Add origin_visit API entry points to permit creation/update.
Aug 18 2016
Aug 18 2016
ardumont updated the task description for T534: Add completion information to softwareheritage.origin_visit table.
ardumont edited parent tasks for T539: Update loaders to register origin_visit's state, added: T538: Add origin_visit API entry points to permit creation/update; removed: T533: Allow loaders to register partial state (meta task).