deserialize_message() now takes an optional 'object_type' argument.
Depends on D7379.
Details
Details
- Reviewers
ardumont seirl - Group Reviewers
Reviewers - Commits
- rDDATASETd7c332e4e7e1: Update JournalClientOffsetRanges for swh.journal 0.9
Diff Detail
Diff Detail
- Repository
- rDDATASET Datasets
- Branch
- orc
- Lint
Lint Skipped - Unit
Unit Tests Skipped - Build Status
Buildable 27716 Build 43378: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 43377: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D7380 (id=26682)
Could not rebase; Attempt merge onto 68f9bd2028...
Updating 68f9bd2..8cae6ad Fast-forward requirements-swh.txt | 6 +++--- swh/dataset/exporters/orc.py | 31 +++++++++++-------------------- swh/dataset/journalprocessor.py | 2 +- swh/dataset/relational.py | 15 +++++++++------ swh/dataset/test/test_orc.py | 14 ++++++-------- 5 files changed, 30 insertions(+), 38 deletions(-)
Changes applied before test
commit 8cae6adb5c63af22bc798ebe7072a33978f3037e
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Dec 15 16:44:49 2021 +0100
Update JournalClientOffsetRanges for swh.journal 0.9
deserialize_message() now takes an optional 'object_type' argument.
commit 8c2b5e951c1a1195c9ec3e700cb9da60711a96ab
Author: David Douard <david.douard@sdfa3.org>
Date: Fri Mar 18 11:46:37 2022 +0100
Encode TimestampWithTimezone as (sec, usec, offset) in ORC file
instead of using the ORC Timestamp format, since we cannot always encode
them in this format.
The offset is encoded as binary (byte string), following recent evolutions
of swh-model.
This makes swh-dataset compatible with swh-model 5.See https://jenkins.softwareheritage.org/job/DDATASET/job/tests-on-diff/30/ for more details.
Comment Actions
Build is green
Patch application report for D7380 (id=26703)
Could not rebase; Attempt merge onto 68f9bd2028...
Updating 68f9bd2..316d51b Fast-forward requirements-swh.txt | 6 +++--- swh/dataset/exporters/orc.py | 36 ++++++++++++++++------------------- swh/dataset/journalprocessor.py | 2 +- swh/dataset/relational.py | 15 +++++++++------ swh/dataset/test/test_orc.py | 42 +++++++++++++++++++++++++++++++++-------- 5 files changed, 63 insertions(+), 38 deletions(-)
Changes applied before test
commit 316d51b6da36719bca767c78ad04402c609d5abe
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Dec 15 16:44:49 2021 +0100
Update JournalClientOffsetRanges for swh.journal 0.9
deserialize_message() now takes an optional 'object_type' argument.
commit ae440431049470ecac6aca0e8cbed4a51cde0c09
Author: David Douard <david.douard@sdfa3.org>
Date: Fri Mar 18 11:46:37 2022 +0100
Encode TimestampWithTimezone as (sec, usec, offset) in ORC file
instead of using the ORC Timestamp format, since we cannot always encode
them in this format.
The offset is encoded as binary (byte string), following recent evolutions
of swh-model.
This makes swh-dataset compatible with swh-model 5.See https://jenkins.softwareheritage.org/job/DDATASET/job/tests-on-diff/42/ for more details.
Comment Actions
Build has FAILED
Patch application report for D7380 (id=26773)
Could not rebase; Attempt merge onto 68f9bd2028...
Updating 68f9bd2..d49db10 Fast-forward requirements-swh.txt | 6 ++-- swh/dataset/exporters/orc.py | 79 +++++++++++++++++++++++++++++++---------- swh/dataset/journalprocessor.py | 2 +- swh/dataset/relational.py | 15 ++++---- swh/dataset/test/test_orc.py | 57 ++++++++++++++++++++++------- 5 files changed, 119 insertions(+), 40 deletions(-)
Changes applied before test
commit d49db10f0bf7174ea4f2742d5d5ac8c8e25b707a
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Dec 15 16:44:49 2021 +0100
Update JournalClientOffsetRanges for swh.journal 0.9
deserialize_message() now takes an optional 'object_type' argument.
commit 69e806698bbb6df42bfa3520681e0203f91d8a65
Author: David Douard <david.douard@sdfa3.org>
Date: Fri Mar 18 11:46:37 2022 +0100
Encode TimestampWithTimezone as (timestamp, offset, raw_offset_bytes) in ORC file
ie. use the standard ORC Timestamp format (aka a couple
(seconds, nanoseconds)) with 2 extra fields for the offset.
The offset is stored as an integer (in minutes), but the raw offset
value is also present as a binary string representation, following
recent evolutions of swh-model.
This makes swh-dataset compatible with swh-model 5.Link to build: https://jenkins.softwareheritage.org/job/DDATASET/job/tests-on-diff/53/
See console output for more information: https://jenkins.softwareheritage.org/job/DDATASET/job/tests-on-diff/53/console
Comment Actions
Build is green
Patch application report for D7380 (id=26795)
Could not rebase; Attempt merge onto 68f9bd2028...
Updating 68f9bd2..70d9d31 Fast-forward requirements-swh.txt | 6 +-- swh/dataset/exporters/orc.py | 81 ++++++++++++++++++++++++++++++++--------- swh/dataset/journalprocessor.py | 2 +- swh/dataset/relational.py | 15 +++++--- swh/dataset/test/test_orc.py | 59 ++++++++++++++++++++++++------ 5 files changed, 124 insertions(+), 39 deletions(-)
Changes applied before test
commit 70d9d3182de1420ba545f2f507ade8f59b2c2f33
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Dec 15 16:44:49 2021 +0100
Update JournalClientOffsetRanges for swh.journal 0.9
deserialize_message() now takes an optional 'object_type' argument.
commit 09d2840dbd4db6e1a3dd976c44b3c628b9174741
Author: David Douard <david.douard@sdfa3.org>
Date: Fri Mar 18 11:46:37 2022 +0100
Encode TimestampWithTimezone as (timestamp, offset, raw_offset_bytes) in ORC file
ie. use the standard ORC Timestamp format (aka a couple
(seconds, nanoseconds)) with 2 extra fields for the offset.
The offset is stored as an integer (in minutes), but the raw offset
value is also present as a binary string representation, following
recent evolutions of swh-model.
This makes swh-dataset compatible with swh-model 5.See https://jenkins.softwareheritage.org/job/DDATASET/job/tests-on-diff/65/ for more details.
Comment Actions
Build is green
Patch application report for D7380 (id=26991)
Could not rebase; Attempt merge onto 68f9bd2028...
Updating 68f9bd2..a8442bc Fast-forward requirements-swh.txt | 6 +-- swh/dataset/exporters/orc.py | 81 ++++++++++++++++++++++++++++++++--------- swh/dataset/journalprocessor.py | 2 +- swh/dataset/relational.py | 3 ++ swh/dataset/test/test_orc.py | 59 ++++++++++++++++++++++++------ 5 files changed, 118 insertions(+), 33 deletions(-)
Changes applied before test
commit a8442bcf7c4311a28bea0898a01dc9475889efc7
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Dec 15 16:44:49 2021 +0100
Update JournalClientOffsetRanges for swh.journal 0.9
deserialize_message() now takes an optional 'object_type' argument.
commit f588e20a41af4b1b8042f9b5f0e88a1f1dc91e59
Author: David Douard <david.douard@sdfa3.org>
Date: Fri Mar 18 11:46:37 2022 +0100
Encode TimestampWithTimezone as (timestamp, offset, raw_offset_bytes) in ORC file
ie. use the standard ORC Timestamp format (aka a couple
(seconds, nanoseconds)) with 2 extra fields for the offset.
The offset is stored as an integer (in minutes), but the raw offset
value is also present as a binary string representation, following
recent evolutions of swh-model.
This makes swh-dataset compatible with swh-model 5.See https://jenkins.softwareheritage.org/job/DDATASET/job/tests-on-diff/77/ for more details.
Comment Actions
Build is green
Patch application report for D7380 (id=27028)
Rebasing onto 31081e4121...
Current branch diff-target is up to date.
Changes applied before test
commit d7c332e4e7e1d5ee531a914b302f98c11503663e
Author: David Douard <david.douard@sdfa3.org>
Date: Wed Dec 15 16:44:49 2021 +0100
Update JournalClientOffsetRanges for swh.journal 0.9
deserialize_message() now takes an optional 'object_type' argument.See https://jenkins.softwareheritage.org/job/DDATASET/job/tests-on-diff/88/ for more details.