In D2084#48356, @ardumont wrote:I don't think this is useful anymore. The postgresql storage does the filtering itself now (see D2034). Are you planning on removing this feature from the postgresql storage?
It always did that.
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed Advanced Search
Advanced Search
Advanced Search
Oct 8 2019
Oct 8 2019
swh-public-ci added a comment to D2085: swh.storage.buffer: Add buffering proxy storage implementation.
Build has FAILED
Also, I understand how that's currently useful, but FYI, batching content has absolutely no effect for Cassandra (the Cassandra backend breaks batches into individual records)
Also, I understand how that's currently useful, but FYI, batching content has absolutely no effect for Cassandra (the Cassandra backend breaks batches into individual records)
BUILD has failed
vlorentz requested changes to D2085: swh.storage.buffer: Add buffering proxy storage implementation.
s/Sequence/Iterable/ because we don't do random access (see definition of sequences here: https://docs.python.org/3/glossary.html#term-sequence )
Build has FAILED
I don't think this is useful anymore. The postgresql storage does the filtering itself now (see D2034). Are you planning on removing this feature from the postgresql storage?
I don't think this is useful anymore. The postgresql storage does the filtering itself now (see D2034). Are you planning on removing this feature from the postgresql storage?
swh-public-ci added a comment to D2085: swh.storage.buffer: Add buffering proxy storage implementation.
Build has FAILED
Harbormaster failed remote builds in B8173: Diff 7003 for D2085: swh.storage.buffer: Add buffering proxy storage implementation!
Build has FAILED
Build has FAILED
Harbormaster failed remote builds in B8172: Diff 7002 for D2084: swh.storage.filter: Add filtering storage implementation!
Oct 1 2019
Oct 1 2019
ardumont added a comment to T2019: race condition during concurrent loading of the same objects from multiple origins.
tagged and deployed (loaders are mostly restarted or in progress)
zack renamed T2019: race condition during concurrent loading of the same objects from multiple origins from Investigate hash collision error to race condition during concurrent loading of the same objects from multiple origins.
Sep 30 2019
Sep 30 2019
ardumont added a comment to T2019: race condition during concurrent loading of the same objects from multiple origins.
Thanks!
olasd added a comment to T2019: race condition during concurrent loading of the same objects from multiple origins.
This is a race condition that happens when two different workers are loading the exact same content in parallel transactions.
olasd shifted T2019: race condition during concurrent loading of the same objects from multiple origins from the Restricted Space space to the S1 Public space.
Should have been closed by rDSTOe2393243e07f ?
ardumont triaged T2019: race condition during concurrent loading of the same objects from multiple origins as High priority.
vlorentz added a project to T2018: origin reference in skipped_content is still an id: Storage manager.
Sep 3 2019
Sep 3 2019
Aug 21 2019
Aug 21 2019
vlorentz added a parent task for T1910: Redesign origin search using a dedicated component (swh-search): T1523: Search tools on metadata.
Aug 20 2019
Aug 20 2019
Aug 5 2019
Aug 5 2019
Jul 29 2019
Jul 29 2019
Jul 26 2019
Jul 26 2019
anlambert added a comment to T1934: vault timeout on cooking revision_gitfast for repositories with numerous number of revisions.
The issue came from the fact that the vault tries to retrieve the whole revisions log in a single call to the storage API.
Jul 22 2019
Jul 22 2019
twitu closed T1633: skipped_content_missing is not implemented by the in-memory storage as Resolved.
Jul 21 2019
Jul 21 2019
Jul 12 2019
Jul 12 2019
Jul 11 2019
Jul 11 2019
vlorentz added a parent task for T1912: Support origin pagination without origin ids: T1805: Public API v2.
vlorentz closed T1816: Make all clients of swh-storage use origin URL as identifier and use visit types instead of origin types as Resolved.
Indeed, this task just inherited the tag from its parent task.
Jul 10 2019
Jul 10 2019
vlorentz reopened T1731: Intrinsic identifiers for origins, a subtask of T1892: Cassandra as a storage backend, as Open.
vlorentz closed T1731: Intrinsic identifiers for origins, a subtask of T1892: Cassandra as a storage backend, as Resolved.
vlorentz triaged T1910: Redesign origin search using a dedicated component (swh-search) as Normal priority.
Jul 8 2019
Jul 8 2019
vlorentz updated the task description for T1816: Make all clients of swh-storage use origin URL as identifier and use visit types instead of origin types.
vlorentz renamed T1892: Cassandra as a storage backend from Cassandra storage backend (meta-task) to Cassandra as a storage backend (meta-task).
Jul 5 2019
Jul 5 2019
zack added a project to T1816: Make all clients of swh-storage use origin URL as identifier and use visit types instead of origin types: Storage manager.
(not strictly speaking a big *in* swh-storage, but close enough…)
Jun 25 2019
Jun 25 2019
Jun 24 2019
Jun 24 2019
Jun 20 2019
Jun 20 2019
D1582 has been pushed the task can be closed
Jun 19 2019
Jun 19 2019
olasd closed T1823: make DB/FS transactions nest properly as Resolved by committing rDOBJS67197802d5aa: pathslicing: Make sure data is flushed to disk before renaming the tempfile.
Jun 18 2019
Jun 18 2019
Jun 17 2019
Jun 17 2019
Jun 15 2019
Jun 15 2019
All columns commented in swh-scheduler, waiting review.
Some columns for swh-storage required a small discussion to frame appropriate comments.
Build is green
See https://jenkins.softwareheritage.org/job/DSTO/job/tox/493/ for more details.
ardumont added projects to D1589: storage-db: Fixing migration scripts 135-136: Storage manager, Data Model.
Jun 14 2019
Jun 14 2019
All columns are already commented in swh-indexer
Have added a few comments in D1582
Jun 13 2019
Jun 13 2019
The latest upgrade is 136.sql while the version in 30-swh-schema.sql is 133. Should I name the next upgrade 137?
there seems to be an inconsistency between sql/upgrades and latest sql version in swh-storage. The latest upgrade is 136.sql while the version in 30-swh-schema.sql is 133. Should I name the next upgrade 137?
is there anything left to be done to close the task?
Jun 12 2019
Jun 12 2019
modules swh-scheduler, swh-indexer, swh-storage, all seem to have column comments written in 30-swh-schema.sql
Can you provide a few more details so I can work on this? Maybe which packages will be affected and what is expected in the comments.
May 25 2019
May 25 2019
zack renamed T1377: in-memory storage: compute all counters from in-memory storage: compute all counters. to in-memory storage: compute all counters.
swh-storage-testdata is gone, closing
zack closed T523: Figure out what to do with corrupted copies detected by the archiver, a subtask of T240: content archiver, as Invalid.
the archiver is gone, closing
May 24 2019
May 24 2019
Still open?
May 21 2019
May 21 2019
May 6 2019
May 6 2019
Apr 23 2019
Apr 23 2019
Apr 19 2019
Apr 19 2019
The fix has been deployed to moma.
Apr 18 2019
Apr 18 2019
anlambert closed T1675: Fix message and error for artifact history view bug as Resolved by committing rDSTO6510b5ec4d83: algos.revisions_walker: Handle truncated/shallow histories.
anlambert triaged T1677: Revisions walker: Add notification for truncated/shallow histories as Normal priority.
anlambert added a project to T1675: Fix message and error for artifact history view bug: Storage manager.
Apr 16 2019
Apr 16 2019
Apr 12 2019
Apr 12 2019
ardumont retitled D1345: swh.journal: Add backfiller implementation from swh.journal: Bootstrap backfiller to swh.journal: Add backfiller implementation.
Build is green
See https://jenkins.softwareheritage.org/job/DJNL/job/tox/145/ for more details.
Rebase to latest master and branch diff to master