Page MenuHomeSoftware Heritage
Feed Advanced Search

Advanced Search
Use Results
Hide Query

Aug 25 2020

ardumont created P747 160-161-bis.sql.
Aug 25 2020, 2:11 PM · Storage manager
douardda closed T2525: Prevent the storage server from starting in case of db version mismatch , a subtask of T2524: Storage database migration tooling, as Resolved.
Aug 25 2020, 11:52 AM · Storage manager
douardda closed T2525: Prevent the storage server from starting in case of db version mismatch as Resolved by committing rDSTO4532a4dc64e7: Add a --check-config option for cli commands.
Aug 25 2020, 11:52 AM · Storage manager

Aug 24 2020

ardumont added a revision to T2524: Storage database migration tooling: D3833: 161: Fix sql upgrade script.
Aug 24 2020, 4:45 PM · Storage manager
douardda added a revision to T2525: Prevent the storage server from starting in case of db version mismatch : D3832: Add support for a new "check_config" config option in get_storage().
Aug 24 2020, 2:51 PM · Storage manager

Aug 20 2020

anlambert added a project to T2524: Storage database migration tooling: Storage manager.
Aug 20 2020, 5:34 PM · Storage manager
douardda added a revision to T2525: Prevent the storage server from starting in case of db version mismatch : D3814: Add a check_dbversion() method to the Db class.
Aug 20 2020, 12:10 PM · Storage manager
douardda added a comment to T2525: Prevent the storage server from starting in case of db version mismatch .

Just to make is clear, my main objective here is to have a seatbelt for the mirror scenario: prevent a mirror with updated code but not-yet-updated database from even running until the DB migration script has been executed.

Aug 20 2020, 11:16 AM · Storage manager
douardda updated the task description for T2525: Prevent the storage server from starting in case of db version mismatch .
Aug 20 2020, 11:04 AM · Storage manager
douardda updated the task description for T2525: Prevent the storage server from starting in case of db version mismatch .
Aug 20 2020, 11:02 AM · Storage manager
douardda triaged T2525: Prevent the storage server from starting in case of db version mismatch as Normal priority.
Aug 20 2020, 10:57 AM · Storage manager
vlorentz renamed T2033: Run Cassandra storage backend with production data from Run Cassandra storage draft with production data to Run Cassandra storage backend with production data.
Aug 20 2020, 8:36 AM · Storage manager

Aug 14 2020

vlorentz closed T2519: Rewrite the in-mem storage as a backend for the cassandra storage as Resolved.
Aug 14 2020, 9:00 PM · Storage manager

Aug 12 2020

vlorentz claimed T2519: Rewrite the in-mem storage as a backend for the cassandra storage.
Aug 12 2020, 10:41 PM · Storage manager
vlorentz added revisions to T2519: Rewrite the in-mem storage as a backend for the cassandra storage: D3766: in_memory: Add class Table, which emulates a Cassandra table., D3767: Make InMemoryStorage inherit from CassandraStorage., D3768: in_memory: Add InMemoryCqlRunner, a class that emulates cassandra.cql.CqlRunner without Cassandra., D3769: in_memory: make object_find_by_sha1_git merge results from the CassandraStorage., D3770: in_memory: Remove InMemoryStorage.content_* and implement InMemoryCqlRunner.content_*, D3771: in_memory: Remove InMemoryStorage.skipped_content_* and implement InMemoryCqlRunner.skipped_content_*, D3772: in_memory: Remove InMemoryStorage.directory_* and implement InMemoryCqlRunner.directory_*, D3773: in_memory: Remove InMemoryStorage.revision_* and implement InMemoryCqlRunner.revision_*, D3774: in_memory: Remove InMemoryStorage.release_* and implement InMemoryCqlRunner.release_*, D3775: Remove endpoint snapshot_get_by_origin_visit., D3776: in_memory: Remove InMemoryStorage.snapshot_* and implement InMemoryCqlRunner.snapshot_*, D3777: in_memory: Remove InMemoryStorage.origin_* and implement InMemoryCqlRunner.origin_*, D3780: in_memory: Remove InMemoryStorage.origin_visit_* and implement InMemoryCqlRunner.origin_visit_*, D3781: in_memory: Remove InMemoryStorage.*metadata_* and implement InMemoryCqlRunner.*metadata_*.
Aug 12 2020, 6:07 PM · Storage manager
vlorentz added a project to T2519: Rewrite the in-mem storage as a backend for the cassandra storage: Storage manager.
Aug 12 2020, 6:06 PM · Storage manager

Aug 7 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3738: model: Add Sha1 alias.
Aug 7 2020, 9:54 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3737: Adapt type and rename content_get_metadata calls to content_get.
Aug 7 2020, 6:40 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3735: test_npm: Adapt content_get_metadata call to content_get.
Aug 7 2020, 12:59 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3734: indexer.rehash: Adapt content_get_metadata call to content_get.
Aug 7 2020, 12:53 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3733: storage*: Rename and type content_get(List[Sha1]) -> List[Optional[Content]].
Aug 7 2020, 12:45 AM · Data Model, Storage manager

Aug 6 2020

ardumont closed T2518: storage.cassandra: Improve content_get_partition pagination implementation as Resolved.
Aug 6 2020, 8:20 PM · Storage manager
ardumont added a revision to T2518: storage.cassandra: Improve content_get_partition pagination implementation: D3723: cassandra.storage: Use next token for pagination instead of computing it.
Aug 6 2020, 1:10 PM · Storage manager
ardumont updated the task description for T2518: storage.cassandra: Improve content_get_partition pagination implementation.
Aug 6 2020, 12:46 PM · Storage manager
ardumont added a revision to T2517: Add remaining missing types to swh.storage.interface: D3719: Adapt code according to storage signature.
Aug 6 2020, 9:58 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3718: "text"-indexers: Migrate to partition index instead of range.
Aug 6 2020, 9:50 AM · Data Model, Storage manager
ardumont added a revision to T2517: Add remaining missing types to swh.storage.interface: D3717: Adapt code according to storage signature.
Aug 6 2020, 9:46 AM · Data Model, Storage manager
ardumont added a revision to T2517: Add remaining missing types to swh.storage.interface: D3716: Adapt code according to storage signature.
Aug 6 2020, 9:40 AM · Data Model, Storage manager

Aug 5 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3715: in_memory: Drop dead code.
Aug 5 2020, 8:02 PM · Data Model, Storage manager
ardumont added a project to T2518: storage.cassandra: Improve content_get_partition pagination implementation: Storage manager.
Aug 5 2020, 5:59 PM · Storage manager
ardumont added a comment to T645: Type swh-storage endpoints with swh.model objects.
  1. {object}_missing for object in {content, directory, revision, release, snapshot}
Aug 5 2020, 5:45 PM · Data Model, Storage manager
ardumont added a comment to T645: Type swh-storage endpoints with swh.model objects.

I'll type with what we have right now, that will simplify the next diffs which introduce type changes.
But also demonstrates the inconsistencies we have right now.

Aug 5 2020, 5:44 PM · Data Model, Storage manager
ardumont closed T2517: Add remaining missing types to swh.storage.interface, a subtask of T645: Type swh-storage endpoints with swh.model objects, as Resolved.
Aug 5 2020, 5:29 PM · Data Model, Storage manager
ardumont closed T2517: Add remaining missing types to swh.storage.interface as Resolved.
Aug 5 2020, 5:29 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3713: storage*: content_get_partition(...) -> PagedResult[Content].
Aug 5 2020, 4:11 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3712: storage*: Drop deprecated content_get_range endpoint.
Aug 5 2020, 3:18 PM · Data Model, Storage manager
ardumont renamed T645: Type swh-storage endpoints with swh.model objects from Type swh-storage endpoints to Type swh-storage endpoints with swh.model objects.
Aug 5 2020, 2:06 PM · Data Model, Storage manager
ardumont removed revisions from T645: Type swh-storage endpoints with swh.model objects: D3708: storage*: origin_get_by_sha1: Drop generator from pgstorage, D3707: storage*: revision_*: Type remaining existing endpoints, D3706: storage*: directory_*: Type remaining existing endpoints, D3705: storage*: skipped_content_missing: Type remaining existing endpoints, D3704: storage*: content_missing_per_sha1(_git): Type remaining existing endpoints, D3703: storage*: content_missing: Unify and type remaining existing endpoints, D3702: storage*: content_get_partition: Type remaining existing endpoints, D3701: storage*: content_get_range: Type remaining existing endpoints, D3700: storage*: content_get: Type remaining existing endpoints, D3699: storage*: content_update: Type remaining existing endpoints, D3698: storage*: origin_get_by_sha1: Type remaining existing endpoints, D3697: storage*: check_config: Type remaining existing endpoints.
Aug 5 2020, 1:12 PM · Data Model, Storage manager
ardumont added revisions to T2517: Add remaining missing types to swh.storage.interface: D3708: storage*: origin_get_by_sha1: Drop generator from pgstorage, D3707: storage*: revision_*: Type remaining existing endpoints, D3706: storage*: directory_*: Type remaining existing endpoints, D3705: storage*: skipped_content_missing: Type remaining existing endpoints, D3704: storage*: content_missing_per_sha1(_git): Type remaining existing endpoints, D3703: storage*: content_missing: Unify and type remaining existing endpoints, D3702: storage*: content_get_partition: Type remaining existing endpoints, D3701: storage*: content_get_range: Type remaining existing endpoints, D3700: storage*: content_get: Type remaining existing endpoints, D3699: storage*: content_update: Type remaining existing endpoints, D3698: storage*: origin_get_by_sha1: Type remaining existing endpoints, D3697: storage*: check_config: Type remaining existing endpoints.
Aug 5 2020, 1:12 PM · Data Model, Storage manager
ardumont renamed T2517: Add remaining missing types to swh.storage.interface from Add remaining missing types to the interface to Add remaining missing types to swh.storage.interface.
Aug 5 2020, 1:10 PM · Data Model, Storage manager
ardumont added a revision to T2517: Add remaining missing types to swh.storage.interface: D3711: storage*: object_find_by_sha1_git: Type remaining existing endpoints.
Aug 5 2020, 1:06 PM · Data Model, Storage manager
ardumont added a revision to T2517: Add remaining missing types to swh.storage.interface: D3710: storage*: snapshot_*: Type remaining existing endpoints.
Aug 5 2020, 12:50 PM · Data Model, Storage manager
ardumont added a revision to T2517: Add remaining missing types to swh.storage.interface: D3709: storage*: release_*: Type remaining existing endpoints.
Aug 5 2020, 12:30 PM · Data Model, Storage manager
ardumont triaged T2517: Add remaining missing types to swh.storage.interface as Normal priority.
Aug 5 2020, 12:21 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3708: storage*: origin_get_by_sha1: Drop generator from pgstorage.
Aug 5 2020, 9:55 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3707: storage*: revision_*: Type remaining existing endpoints.
Aug 5 2020, 8:38 AM · Data Model, Storage manager

Aug 4 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3706: storage*: directory_*: Type remaining existing endpoints.
Aug 4 2020, 11:12 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3705: storage*: skipped_content_missing: Type remaining existing endpoints.
Aug 4 2020, 11:11 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3704: storage*: content_missing_per_sha1(_git): Type remaining existing endpoints.
Aug 4 2020, 6:58 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3703: storage*: content_missing: Unify and type remaining existing endpoints.
Aug 4 2020, 6:48 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3702: storage*: content_get_partition: Type remaining existing endpoints.
Aug 4 2020, 6:47 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3701: storage*: content_get_range: Type remaining existing endpoints.
Aug 4 2020, 6:47 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3700: storage*: content_get: Type remaining existing endpoints.
Aug 4 2020, 6:40 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3699: storage*: content_update: Type remaining existing endpoints.
Aug 4 2020, 6:40 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3698: storage*: origin_get_by_sha1: Type remaining existing endpoints.
Aug 4 2020, 6:02 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3697: storage*: check_config: Type remaining existing endpoints.
Aug 4 2020, 6:00 PM · Data Model, Storage manager
ardumont added a comment to T645: Type swh-storage endpoints with swh.model objects.

I'll type with what we have right now, that will simplify the next diffs which introduce type changes.
But also demonstrates the inconsistencies we have right now.

Aug 4 2020, 5:26 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3693: service: Adapt according to the latest storage.content_find changes.
Aug 4 2020, 11:15 AM · Data Model, Storage manager
ardumont changed the status of T645: Type swh-storage endpoints with swh.model objects from Open to Work in Progress.

Current status, related endpoints to origin, origin-visit and origin-visit-status are done now both read/write.
Remains dag model objects (content, directory, revision, release, snapshot) reading endpoints to align and type.

Aug 4 2020, 10:17 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3692: storage*: Type content_find(...) -> List[Content].
Aug 4 2020, 10:11 AM · Data Model, Storage manager

Aug 3 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3687: storage*: Type {cnt,dir,rev,rel,snp}_get_random(...) -> Sha1Git.
Aug 3 2020, 1:27 PM · Data Model, Storage manager

Aug 2 2020

ardumont added a revision to T2214: Scale-out graph and database storage in production: D3675: origin: Migrate use to storage.origin_list instead of origin_get_range.
Aug 2 2020, 11:17 AM · meta-task, Roadmap 2022, Roadmap 2021, Storage manager
ardumont removed a revision from T645: Type swh-storage endpoints with swh.model objects: D3684: swh.search.get_search: Simplify instantiation.
Aug 2 2020, 11:15 AM · Data Model, Storage manager
ardumont removed a revision from T645: Type swh-storage endpoints with swh.model objects: D3685: swh.search: Define an interface for search backends and use it.
Aug 2 2020, 11:15 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3685: swh.search: Define an interface for search backends and use it.
Aug 2 2020, 11:11 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3684: swh.search.get_search: Simplify instantiation.
Aug 2 2020, 11:11 AM · Data Model, Storage manager

Aug 1 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3683: storage.in_memory: Fix origin_list implementation.
Aug 1 2020, 11:16 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3682: cli.task: Migrate scheduler cli to latest storage change on iter_origins.
Aug 1 2020, 10:04 AM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3681: storage*: Drop origin-get-range in favor of origin-list.
Aug 1 2020, 9:30 AM · Data Model, Storage manager

Jul 31 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3675: origin: Migrate use to storage.origin_list instead of origin_get_range.
Jul 31 2020, 4:19 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3671: storage*: Add type annotation to origin_count.
Jul 31 2020, 2:51 PM · Data Model, Storage manager
vlorentz closed T2514: Add raw_extrinsic_metadata to the journal backfiller as Resolved.
Jul 31 2020, 2:25 PM · Journal, Storage manager, Roadmap 2020
vlorentz added a revision to T2514: Add raw_extrinsic_metadata to the journal backfiller: D3659: Add support for metadata-related object types to the backfiller and replayer..
Jul 31 2020, 2:25 PM · Journal, Storage manager, Roadmap 2020
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3669: Reuse swh.core stream_results function.
Jul 31 2020, 1:58 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3664: api.classes: Open swh.core.api.classes.stream_results.
Jul 31 2020, 12:58 PM · Data Model, Storage manager

Jul 30 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3661: common/service: Migrate origin_search to latest apis change.
Jul 30 2020, 10:56 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3657: search*: Type origin_search(...) -> PagedResult[Dict].
Jul 30 2020, 7:33 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3651: storage*: Type origin_search(...) -> PagedResult[Origin].
Jul 30 2020, 4:10 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3650: storage*: Adapt origin_list(...) -> PagedResult[Origin].
Jul 30 2020, 2:32 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3648: algos.snapshot: Open snapshot_get_from_revision algorithm.
Jul 30 2020, 10:02 AM · Data Model, Storage manager

Jul 29 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3647: service: Migrate to latest origin_visit_get api change.
Jul 29 2020, 8:17 PM · Data Model, Storage manager
vlorentz closed T2074: Publish extrinsic metadata to swh-journal/Kafka, a subtask of T2514: Add raw_extrinsic_metadata to the journal backfiller, as Resolved.
Jul 29 2020, 7:36 PM · Journal, Storage manager, Roadmap 2020
vlorentz closed T2074: Publish extrinsic metadata to swh-journal/Kafka as Resolved.
Jul 29 2020, 7:36 PM · Storage manager, Journal, Metadata workflow
vlorentz added a revision to T2074: Publish extrinsic metadata to swh-journal/Kafka: D3633: Write metadata + metadata authorities/fetchers to the journal..
Jul 29 2020, 7:36 PM · Storage manager, Journal, Metadata workflow
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3645: deposit.migrations: Migrate to latest storage api change.
Jul 29 2020, 7:35 PM · Data Model, Storage manager
vlorentz added a parent task for T2074: Publish extrinsic metadata to swh-journal/Kafka: T2514: Add raw_extrinsic_metadata to the journal backfiller.
Jul 29 2020, 7:35 PM · Storage manager, Journal, Metadata workflow
vlorentz added a subtask for T2514: Add raw_extrinsic_metadata to the journal backfiller: T2074: Publish extrinsic metadata to swh-journal/Kafka.
Jul 29 2020, 7:35 PM · Journal, Storage manager, Roadmap 2020
vlorentz triaged T2514: Add raw_extrinsic_metadata to the journal backfiller as Normal priority.
Jul 29 2020, 7:35 PM · Journal, Storage manager, Roadmap 2020
vlorentz closed T2306: Generic storage for extrinsic, qualified metadata related to any node of the swh archive, a subtask of T2074: Publish extrinsic metadata to swh-journal/Kafka, as Resolved.
Jul 29 2020, 7:33 PM · Storage manager, Journal, Metadata workflow
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3643: storage*: Simplify next-page-token computation.
Jul 29 2020, 4:59 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3641: storage*: add origin_visit_status_get(...) -> PagedResult[OriginVisitStatus].
Jul 29 2020, 4:35 PM · Data Model, Storage manager

Jul 28 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3632: swh.core.api: Expose a serializable PagedResult object.
Jul 28 2020, 3:42 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3629: storage*: use an enum to explicit the order in origin_visit_get.
Jul 28 2020, 1:14 PM · Data Model, Storage manager

Jul 27 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3627: storage*: origin_visit_get(...) -> PagedResult[OriginVisit].
Jul 27 2020, 10:10 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3626: Update swh.storage.origin_visit_get_by calls to latest api change.
Jul 27 2020, 4:01 PM · Data Model, Storage manager
ardumont added a parent task for T645: Type swh-storage endpoints with swh.model objects: T2223: Type checking.
Jul 27 2020, 2:55 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3625: storage*: origin_visit_get_by -> Optional[OriginVisit].
Jul 27 2020, 2:19 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3622: storage*: origin_visit_find_by_date -> Optional[OriginVisit].
Jul 27 2020, 12:49 PM · Data Model, Storage manager
ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3620: storage*: type origin_visit_get_latest endpoint result.
Jul 27 2020, 8:16 AM · Data Model, Storage manager

Jul 25 2020

ardumont added a revision to T645: Type swh-storage endpoints with swh.model objects: D3619: metadata: Update swh.storage.origin_get call to latest api change.
Jul 25 2020, 8:02 AM · Data Model, Storage manager
First
Prev
Next
About · Developer information · Legal