Details

Reviewers

Group Reviewers

Commits

rDSTO45687d8c6829: Add test_revision_add_fractional_timezone
rDSTOfb1b3a066ff0: postgresql: Fix one-by-one error in db_to_date on negative dates

Summary

Replaces D6819

Diff Detail

Repository

rDSTO Storage manager

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 25614
Build 40053: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 40052: arc lint + arc unit

Event Timeline

vlorentz created this revision.Dec 13 2021, 1:23 PM

vlorentz mentioned this in D6821: Add tests checking round-tripping of dir/rev/rel/snp objects generated by Hypothesis.Dec 13 2021, 1:23 PM

vlorentz added a child revision: D6821: Add tests checking round-tripping of dir/rev/rel/snp objects generated by Hypothesis.

Build is green

Patch application report for D6825 (id=24749)

Could not rebase; Attempt merge onto 7cb4128e40...

Updating 7cb4128e..824d823e
Fast-forward
 requirements.txt                                |   2 +-
 swh/storage/postgresql/converters.py            |   7 +-
 swh/storage/tests/storage_tests.py              |  31 ++++
 swh/storage/tests/test_postgresql_converters.py | 187 +++++++++++++++++-------
 4 files changed, 174 insertions(+), 53 deletions(-)

Changes applied before test

commit 824d823e6f7d4b5aaca3c7ac78aa0dd63b3190f7
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Dec 9 17:04:40 2021 +0100

    Add test_revision_add_fractional_timezone

commit fb1b3a066ff0898318ea28c901761c5c3562f0f6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Dec 9 16:25:20 2021 +0100

    postgresql: Fix one-by-one error in db_to_date on negative dates
    
    Using `int()` on `date.timestamp()` rounded it up (toward zero),
    but the semantics of `model.Timestamp` is that the actual time is
    `ts.seconds + ts.microseconds/1000000`, so all negative dates were
    shifted one second up.
    
    In particular, this causes dates from
    `1969-12-31T23:59:59.000001` to `1969-12-31T23:59:59.999999`
    (inclusive) to smash into dates from
    `1970-01-01T00:00:00.000001` to `1970-01-01T00:00:00.999999`,
    which is how I discovered the issue.

commit 34ca67e21f68de920f603223f629070683018c2a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Dec 9 16:23:44 2021 +0100

    postgresql: Add tests for db_to_date.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1505/ for more details.

Harbormaster completed remote builds in B25600: Diff 24749.Dec 13 2021, 1:30 PM

vlorentz requested review of this revision.Dec 13 2021, 1:30 PM

Add workaround to support fractional timezones on psycopg2 < 2.9.0.

This revision was not accepted when it landed; it landed in state Needs Review.Dec 14 2021, 11:22 AM

This revision was landed with ongoing or failed builds.

Closed by commit rDSTOfb1b3a066ff0: postgresql: Fix one-by-one error in db_to_date on negative dates (authored by vlorentz). · Explain Why

This revision was automatically updated to reflect the committed changes.

vlorentz added a commit: rDSTOfb1b3a066ff0: postgresql: Fix one-by-one error in db_to_date on negative dates.

Build is green

Patch application report for D6825 (id=24761)

Could not rebase; Attempt merge onto 7cb4128e40...

Updating 7cb4128e..08e1adee
Fast-forward
 swh/storage/postgresql/converters.py            |   7 +-
 swh/storage/postgresql/storage.py               |  10 +-
 swh/storage/tests/storage_tests.py              |  39 +++++
 swh/storage/tests/test_postgresql_converters.py | 187 +++++++++++++++++-------
 4 files changed, 190 insertions(+), 53 deletions(-)

Changes applied before test

commit 08e1adee3f65bd3a519a745be3848a44c6dd147c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Dec 9 17:04:40 2021 +0100

    Add test_revision_add_fractional_timezone

commit fb1b3a066ff0898318ea28c901761c5c3562f0f6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Dec 9 16:25:20 2021 +0100

    postgresql: Fix one-by-one error in db_to_date on negative dates
    
    Using `int()` on `date.timestamp()` rounded it up (toward zero),
    but the semantics of `model.Timestamp` is that the actual time is
    `ts.seconds + ts.microseconds/1000000`, so all negative dates were
    shifted one second up.
    
    In particular, this causes dates from
    `1969-12-31T23:59:59.000001` to `1969-12-31T23:59:59.999999`
    (inclusive) to smash into dates from
    `1970-01-01T00:00:00.000001` to `1970-01-01T00:00:00.999999`,
    which is how I discovered the issue.

commit 34ca67e21f68de920f603223f629070683018c2a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Dec 9 16:23:44 2021 +0100

    postgresql: Add tests for db_to_date.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1506/ for more details.

Harbormaster completed remote builds in B25614: Diff 24761.Dec 14 2021, 11:30 AM

oops, I messed up git again, this isn't actually landed

remove commit that was already landed

Build is green

Patch application report for D6825 (id=24767)

Rebasing onto fb1b3a066f...

Current branch diff-target is up to date.

Changes applied before test

commit 08e1adee3f65bd3a519a745be3848a44c6dd147c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Dec 9 17:04:40 2021 +0100

    Add test_revision_add_fractional_timezone

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1508/ for more details.

Harbormaster completed remote builds in B25620: Diff 24767.Dec 14 2021, 2:01 PM

olasd accepted this revision.Dec 15 2021, 12:59 PM

olasd added inline comments.

swh/storage/postgresql/storage.py
120	Should probably happen here too?

This revision is now accepted and ready to land.Dec 15 2021, 12:59 PM

indeed

Build is green

Patch application report for D6825 (id=24796)

Rebasing onto fb1b3a066f...

Current branch diff-target is up to date.

Changes applied before test

commit 45687d8c68298abb69172386eca04e318eefc29c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Dec 9 17:04:40 2021 +0100

    Add test_revision_add_fractional_timezone

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1509/ for more details.

Harbormaster completed remote builds in B25645: Diff 24796.Dec 15 2021, 1:19 PM

Closed by commit rDSTO45687d8c6829: Add test_revision_add_fractional_timezone (authored by vlorentz). · Explain WhyDec 15 2021, 1:23 PM

This revision was automatically updated to reflect the committed changes.

vlorentz added a commit: rDSTO45687d8c6829: Add test_revision_add_fractional_timezone.

Add test_revision_add_fractional_timezone
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Patch application report for D6825 (id=24749)

Changes applied before test

Patch application report for D6825 (id=24761)

Changes applied before test

Patch application report for D6825 (id=24767)

Changes applied before test

Patch application report for D6825 (id=24796)

Changes applied before test

Revision Contents
Changeset List

Diff 24761

swh/storage/postgresql/converters.py

swh/storage/postgresql/storage.py

swh/storage/tests/storage_tests.py

swh/storage/tests/test_postgresql_converters.py

Add test_revision_add_fractional_timezoneClosedPublicActions

Details

Diff Detail

Event Timeline

Patch application report for D6825 (id=24749)

Changes applied before test

Patch application report for D6825 (id=24761)

Changes applied before test

Patch application report for D6825 (id=24767)

Changes applied before test

Patch application report for D6825 (id=24796)

Changes applied before test

Revision ContentsChangeset List

Diff 24761

swh/storage/postgresql/converters.py

swh/storage/postgresql/storage.py

swh/storage/tests/storage_tests.py

swh/storage/tests/test_postgresql_converters.py

Add test_revision_add_fractional_timezone
ClosedPublic
Actions

Revision Contents
Changeset List