Page MenuHomeSoftware Heritage
Feed Advanced Search

Jul 1 2020

ardumont changed the status of T2306: Generic storage for extrinsic, qualified metadata related to any node of the swh archive, a subtask of T2074: Publish extrinsic metadata to swh-journal/Kafka, from Open to Work in Progress.
Jul 1 2020, 3:50 PM · Storage manager, Journal, Metadata workflow
ardumont updated the task description for T2310: Make origin visits immutable.
Jul 1 2020, 3:38 PM · Storage manager, Data Model

Jun 30 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 30 2020, 4:09 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 30 2020, 4:02 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3380: storage*: Drop intermediary conversion step into OriginVisit.
Jun 30 2020, 3:30 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3376: journal_data: Drop obsolete origin_visit fields.
Jun 30 2020, 2:18 PM · Storage manager, Data Model

Jun 29 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 29 2020, 6:35 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 29 2020, 6:25 PM · Storage manager, Data Model
vlorentz added a subtask for T2074: Publish extrinsic metadata to swh-journal/Kafka: T2306: Generic storage for extrinsic, qualified metadata related to any node of the swh archive.
Jun 29 2020, 2:58 PM · Storage manager, Journal, Metadata workflow
vlorentz closed T2075: Implement metadata authority specification as Resolved.
Jun 29 2020, 2:58 PM · Storage manager, Metadata workflow
vlorentz closed T2075: Implement metadata authority specification, a subtask of T2074: Publish extrinsic metadata to swh-journal/Kafka, as Resolved.
Jun 29 2020, 2:58 PM · Storage manager, Journal, Metadata workflow
vlorentz closed T1912: Support origin pagination without origin ids, a subtask of T1892: Cassandra as a storage backend, as Resolved.
Jun 29 2020, 2:13 PM · meta-task, Storage manager
vlorentz closed T1912: Support origin pagination without origin ids as Resolved.
Jun 29 2020, 2:13 PM · Web app, Storage manager
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 29 2020, 1:19 PM · Storage manager, Data Model

Jun 26 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 26 2020, 6:09 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3365: tests*: Drop obsolete origin visit fields.
Jun 26 2020, 6:08 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3350: Iterate over paginated visits in batches to retrieve latest visit/snapshot.
Jun 26 2020, 5:45 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 26 2020, 3:36 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 26 2020, 12:33 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3363: tests*: Drop obsolete origin visit fields.
Jun 26 2020, 12:28 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 26 2020, 10:26 AM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3362: indexer*: Drop obsolete origin visit fields.
Jun 26 2020, 10:25 AM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3361: loader*: Drop obsolete origin visit fields.
Jun 26 2020, 10:12 AM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 26 2020, 10:00 AM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3360: replayer: Drop obsolete fields from origin-visit.
Jun 26 2020, 9:58 AM · Storage manager, Data Model

Jun 25 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 25 2020, 7:21 PM · Storage manager, Data Model

Jun 24 2020

ardumont added a revision to T2310: Make origin visits immutable: D3344: journal_data: Make origin-visit optional fields to None.
Jun 24 2020, 1:40 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3342: storage*: Drop obsolete fields from origin_visit.
Jun 24 2020, 11:00 AM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3341: OriginVisitStatus: Allow "created" status.
Jun 24 2020, 9:14 AM · Storage manager, Data Model

Jun 23 2020

ardumont added a revision to T2310: Make origin visits immutable: D3340: model.OriginVisit: Make obsolete fields optional.
Jun 23 2020, 5:32 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 23 2020, 4:00 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 23 2020, 3:48 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 23 2020, 2:37 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 23 2020, 1:13 PM · Storage manager, Data Model

Jun 22 2020

ardumont added a revision to T2310: Make origin visits immutable: D3337: model.OriginVisit: Drop obsolete fields.
Jun 22 2020, 6:16 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3333: tests: Use assert_last_visit_matches.
Jun 22 2020, 2:54 PM · Storage manager, Data Model

Jun 18 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 18 2020, 6:44 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3316: service: Use latest origin visit status from an origin.
Jun 18 2020, 6:43 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3314: Move deprecated endpoint snapshot_get_latest from api endpoint to algos.
Jun 18 2020, 3:24 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3313: algos.origin: Open origin-get-latest-visit-status function.
Jun 18 2020, 1:45 PM · Storage manager, Data Model

Jun 17 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 17 2020, 5:31 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3308: loader: Read snapshot out of the last visit status.
Jun 17 2020, 5:30 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3307: loader: Read snapshot out of the last origin visit status.
Jun 17 2020, 4:59 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3306: loader: Read snapshot out of the last origin visit status.
Jun 17 2020, 4:26 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3305: package.loader: Retrieve latest snapshot out of the latest visit status.
Jun 17 2020, 4:24 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 17 2020, 12:37 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3301: origin_head: Retrieve snapshot out of the last visit status.
Jun 17 2020, 11:56 AM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 17 2020, 9:48 AM · Storage manager, Data Model

Jun 16 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 16 2020, 8:15 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3296: storage*: Add origin-visit-status-get-latest endpoint.
Jun 16 2020, 8:15 PM · Storage manager, Data Model
douardda closed T2455: backfiller: properly support null authors as Resolved by committing rDSTOc2b673b49b88: Fix db_to_author() converter to return None is all fields are None.
Jun 16 2020, 1:59 PM · Storage manager
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 16 2020, 12:50 PM · Storage manager, Data Model
douardda added a revision to T2455: backfiller: properly support null authors: D3290: Fix db_to_author() converter to return None is all fields are None.
Jun 16 2020, 12:22 PM · Storage manager
olasd triaged T2455: backfiller: properly support null authors as High priority.
Jun 16 2020, 12:16 PM · Storage manager
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 16 2020, 9:56 AM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 16 2020, 9:36 AM · Storage manager, Data Model

Jun 15 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 15 2020, 3:11 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 15 2020, 2:35 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 15 2020, 2:31 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3281: storage*: Drop origin_visit_upsert endpoint.
Jun 15 2020, 2:28 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3279: test_storage: Add journal behavior coverage for origin-visit-*add.
Jun 15 2020, 11:28 AM · Storage manager, Data Model

Jun 14 2020

zack added a comment to T2449: Consider switching timestamp offset storage to strings/byte arrays.

Yeah, for having played with it quite a bit in recent times, the current state of timestamp offsets isn't great. I'm fine with the idea of switching them to bytestrings as proposed.

Jun 14 2020, 8:49 AM · Storage manager, Data Model

Jun 13 2020

ardumont added a revision to T2310: Make origin visits immutable: D3278: in_memory: Make origin-visit-status-add respect "on conflict ignore" policy.
Jun 13 2020, 8:42 AM · Storage manager, Data Model

Jun 12 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 12 2020, 7:15 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3276: storage*: Remove origin-visit-update endpoint.
Jun 12 2020, 6:55 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3275: cassandra.storage: Write origin-visit-status once in journal.
Jun 12 2020, 6:53 PM · Storage manager, Data Model
olasd updated subscribers of T2449: Consider switching timestamp offset storage to strings/byte arrays.

(ping @zack who has done some actual analysis on the timezone-related data in the archive)

Jun 12 2020, 1:03 PM · Storage manager, Data Model
olasd triaged T2449: Consider switching timestamp offset storage to strings/byte arrays as Low priority.
Jun 12 2020, 1:00 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 12 2020, 12:04 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3273: replay: Replay origin-visit and origin-visit-status.
Jun 12 2020, 12:02 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3272: Adapt visit_date type from string to datetime.
Jun 12 2020, 11:30 AM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 12 2020, 11:05 AM · Storage manager, Data Model

Jun 11 2020

ardumont added a revision to T2310: Make origin visits immutable: D3267: swh-web: Migrate to new storage.origin_visit_add endpoint.
Jun 11 2020, 7:11 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3265: tests: Migrate to new storage.origin_visit_add endpoint.
Jun 11 2020, 6:57 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 11 2020, 5:46 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3262: storage*: Align origin-visit-add to take iterable of OriginVisit objects as input.
Jun 11 2020, 3:26 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 11 2020, 10:20 AM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3260: tests: Migrate to use origin visit status add storage endpoint.
Jun 11 2020, 10:15 AM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 11 2020, 10:07 AM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3259: tests: Migrate to use origin visit status add storage endpoint.
Jun 11 2020, 10:05 AM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 11 2020, 9:51 AM · Storage manager, Data Model

Jun 10 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 10 2020, 3:09 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3253: loader: Migrate to use origin visit status add storage endpoint.
Jun 10 2020, 12:36 PM · Storage manager, Data Model
ardumont added a revision to T2310: Make origin visits immutable: D3251: origin-visit-upsert: Write origin-visit-status to journal.
Jun 10 2020, 10:39 AM · Storage manager, Data Model
ardumont closed D3238: origin-visit-add: Write origin-visit-status objects to the journal.
Jun 10 2020, 9:06 AM · Storage manager

Jun 9 2020

ardumont updated the task description for T2310: Make origin visits immutable.
Jun 9 2020, 5:43 PM · Storage manager, Data Model
ardumont updated the task description for T2310: Make origin visits immutable.
Jun 9 2020, 5:43 PM · Storage manager, Data Model
ardumont updated the test plan for D3238: origin-visit-add: Write origin-visit-status objects to the journal.
Jun 9 2020, 2:38 PM · Storage manager
swh-public-ci added a comment to D3238: origin-visit-add: Write origin-visit-status objects to the journal.

Build is green

Jun 9 2020, 2:38 PM · Storage manager
ardumont retitled D3238: origin-visit-add: Write origin-visit-status objects to the journal from origin-visit-add: Write origin-visit-status to the journal to origin-visit-add: Write origin-visit-status objects to the journal.
Jun 9 2020, 2:35 PM · Storage manager
ardumont updated the diff for D3238: origin-visit-add: Write origin-visit-status objects to the journal.
  • Rebase on latest master
  • Fix journal related tests to take into accounts origin-visit-status objects
Jun 9 2020, 2:30 PM · Storage manager
ardumont updated the test plan for D3238: origin-visit-add: Write origin-visit-status objects to the journal.
Jun 9 2020, 12:22 PM · Storage manager
swh-public-ci added a comment to D3238: origin-visit-add: Write origin-visit-status objects to the journal.

Build has FAILED

Jun 9 2020, 11:19 AM · Storage manager

Jun 8 2020

ardumont added inline comments to D3238: origin-visit-add: Write origin-visit-status objects to the journal.
Jun 8 2020, 5:12 PM · Storage manager
ardumont added a revision to T2310: Make origin visits immutable: D3244: origin-visit-update: Write visit-status objects to the journal.
Jun 8 2020, 4:28 PM · Storage manager, Data Model
vlorentz accepted D3238: origin-visit-add: Write origin-visit-status objects to the journal.
Jun 8 2020, 2:52 PM · Storage manager
ardumont added inline comments to D3238: origin-visit-add: Write origin-visit-status objects to the journal.
Jun 8 2020, 2:50 PM · Storage manager
vlorentz added inline comments to D3238: origin-visit-add: Write origin-visit-status objects to the journal.
Jun 8 2020, 2:46 PM · Storage manager
Harbormaster failed remote builds in B12694: Diff 11489 for D3238: origin-visit-add: Write origin-visit-status objects to the journal!
Jun 8 2020, 2:41 PM · Storage manager
swh-public-ci added a comment to D3238: origin-visit-add: Write origin-visit-status objects to the journal.

Build has FAILED

Jun 8 2020, 2:41 PM · Storage manager