Page MenuHomeSoftware Heritage
Differential D6939

Stop passing 'offset' and 'negative_utc' to TimestampWithTimezone()
ClosedPublic
Actions

Authored by vlorentz on Jan 13 2022, 12:22 PM.

Details

Reviewers
olasd
Group Reviewers
Reviewers
Maniphest Tasks
T3752: Store/represent time offsets as strings
Commits
rDLDG4b77e567f80b: Stop passing 'offset' and 'negative_utc' to TimestampWithTimezone()
Summary

swh-model 5.0.0 removes these arguments from the constructor.

Depends on D6923.

Test Plan

Will fail because this depends on D6936, which isn't released yet.

Diff Detail

Repository
rDLDG Git loader
Branch
weird-git
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 26035
Build 40685: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 40684: arc lint + arc unit

Event Timeline

vlorentz created this revision.Jan 13 2022, 12:22 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptJan 13 2022, 12:22 PM
swh-public-ci added a comment.Jan 13 2022, 12:22 PM

Build has FAILED

Patch application report for D6939 (id=25129)

Could not rebase; Attempt merge onto 64050aa08b...

Merge made by the 'recursive' strategy.
 requirements-swh.txt                    |   2 +-
 swh/loader/git/converters.py            | 114 +++++++++++++--
 swh/loader/git/tests/test_converters.py | 246 ++++++++++++++++++++++++++++----
 3 files changed, 319 insertions(+), 43 deletions(-)
Changes applied before test
commit f993bc53a5b44e6930300dcb1b7a7a22cfec2d2d
Merge: 64050aa 5d2549a
Author: Jenkins user <jenkins@localhost>
Date:   Thu Jan 13 11:22:17 2022 +0000

    Merge branch 'diff-target' into HEAD

commit 5d2549aceb51315d08d0208fb9e970379ea9e5d7
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jan 12 17:08:04 2022 +0100

    Stop passing 'offset' and 'negative_utc' to TimestampWithTimezone()
    
    swh-model 5.0.0 removes these arguments from the constructor.

commit c2947c14bd7d7ae9209555caf4455f52fe109ab2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jan 12 11:58:29 2022 +0100

    converters: Write raw_manifest of Directory objects
    
    This allows representing git trees with disordered entries,
    as the "normal" data model requires them to be sorted.

commit 6808ddef4ff9a5f330ae384f3654d3acc38a5fb8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 7 17:27:10 2022 +0100

    converters: Write object_bytes and raw_manifest on revisions and releases
    
    This allows representing all git objects instead of rejecting
    objects that do not fit in our "normal" data model.
    
    This commit is restricted to revisions and releases for now,
    a future commit will add directories.

Link to build: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/176/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/176/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2022, 12:22 PM
Harbormaster failed remote builds in B25999: Diff 25129!
vlorentz added a task: T3752: Store/represent time offsets as strings.Jan 13 2022, 12:26 PM
vlorentz requested review of this revision.Jan 13 2022, 12:29 PM
vlorentz updated this revision to Diff 25171.Jan 14 2022, 12:40 PM
  • Fix fallback timestamp conversion in dulwich_tsinfo_to_timestamp.
swh-public-ci added a comment.Jan 14 2022, 12:40 PM

Build has FAILED

Patch application report for D6939 (id=25171)

Could not rebase; Attempt merge onto 64050aa08b...

Merge made by the 'recursive' strategy.
 requirements-swh.txt                    |   2 +-
 swh/loader/git/converters.py            | 114 +++++++++++--
 swh/loader/git/tests/test_converters.py | 279 ++++++++++++++++++++++++++++----
 3 files changed, 352 insertions(+), 43 deletions(-)
Changes applied before test
commit e0b5e4d9132a3fe9c674d7fe82f11c16f66471bd
Merge: 64050aa c8703db
Author: Jenkins user <jenkins@localhost>
Date:   Fri Jan 14 11:40:38 2022 +0000

    Merge branch 'diff-target' into HEAD

commit c8703db672566658bcf3a1145c892492254a5793
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 14 12:40:14 2022 +0100

    Fix fallback timestamp conversion in dulwich_tsinfo_to_timestamp.

commit 5d2549aceb51315d08d0208fb9e970379ea9e5d7
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jan 12 17:08:04 2022 +0100

    Stop passing 'offset' and 'negative_utc' to TimestampWithTimezone()
    
    swh-model 5.0.0 removes these arguments from the constructor.

commit c2947c14bd7d7ae9209555caf4455f52fe109ab2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jan 12 11:58:29 2022 +0100

    converters: Write raw_manifest of Directory objects
    
    This allows representing git trees with disordered entries,
    as the "normal" data model requires them to be sorted.

commit 6808ddef4ff9a5f330ae384f3654d3acc38a5fb8
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 7 17:27:10 2022 +0100

    converters: Write object_bytes and raw_manifest on revisions and releases
    
    This allows representing all git objects instead of rejecting
    objects that do not fit in our "normal" data model.
    
    This commit is restricted to revisions and releases for now,
    a future commit will add directories.

Link to build: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/177/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/177/console

Harbormaster failed remote builds in B26035: Diff 25171!Jan 14 2022, 12:40 PM
vlorentz updated this revision to Diff 25187.Jan 14 2022, 3:21 PM

lower swh-model version requirement

vlorentz updated this revision to Diff 25188.Jan 14 2022, 3:21 PM

lower swh-model version requirement

swh-public-ci added a comment.Jan 14 2022, 3:23 PM

Build is green

Patch application report for D6939 (id=25187)

Rebasing onto 64050aa08b...

Current branch diff-target is up to date.
Changes applied before test
commit 4b77e567f80b2105b517e245ad9aaaf4a58a51dc
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jan 12 17:08:04 2022 +0100

    Stop passing 'offset' and 'negative_utc' to TimestampWithTimezone()
    
    swh-model 5.0.0 removes these arguments from the constructor.

commit dc63db4c0c43e012167bd1d17eade5f6839e16fc
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jan 12 11:58:29 2022 +0100

    converters: Write raw_manifest of Directory objects
    
    This allows representing git trees with disordered entries,
    as the "normal" data model requires them to be sorted.

commit 44cc7d083327449594e93e8ad2d01abb6982be82
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 7 17:27:10 2022 +0100

    converters: Write object_bytes and raw_manifest on revisions and releases
    
    This allows representing all git objects instead of rejecting
    objects that do not fit in our "normal" data model.
    
    This commit is restricted to revisions and releases for now,
    a future commit will add directories.

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/178/ for more details.

Harbormaster completed remote builds in B26051: Diff 25187.Jan 14 2022, 3:23 PM
swh-public-ci added a comment.Jan 14 2022, 3:23 PM

Build is green

Patch application report for D6939 (id=25188)

Could not rebase; Attempt merge onto 64050aa08b...

Updating 64050aa..4b77e56
Fast-forward
 requirements-swh.txt                    |   2 +-
 swh/loader/git/converters.py            | 114 +++++++++++--
 swh/loader/git/tests/test_converters.py | 279 ++++++++++++++++++++++++++++----
 3 files changed, 352 insertions(+), 43 deletions(-)
Changes applied before test
commit 4b77e567f80b2105b517e245ad9aaaf4a58a51dc
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jan 12 17:08:04 2022 +0100

    Stop passing 'offset' and 'negative_utc' to TimestampWithTimezone()
    
    swh-model 5.0.0 removes these arguments from the constructor.

commit dc63db4c0c43e012167bd1d17eade5f6839e16fc
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jan 12 11:58:29 2022 +0100

    converters: Write raw_manifest of Directory objects
    
    This allows representing git trees with disordered entries,
    as the "normal" data model requires them to be sorted.

commit 44cc7d083327449594e93e8ad2d01abb6982be82
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Jan 7 17:27:10 2022 +0100

    converters: Write object_bytes and raw_manifest on revisions and releases
    
    This allows representing all git objects instead of rejecting
    objects that do not fit in our "normal" data model.
    
    This commit is restricted to revisions and releases for now,
    a future commit will add directories.

See https://jenkins.softwareheritage.org/job/DLDG/job/tests-on-diff/179/ for more details.

Harbormaster completed remote builds in B26052: Diff 25188.Jan 14 2022, 3:23 PM
olasd added a subscriber: olasd.Jan 14 2022, 5:50 PM
olasd added inline comments.
swh/loader/git/converters.py
152–153

Should this be a warning?

olasd added inline comments.Jan 17 2022, 4:30 PM
swh/loader/git/tests/test_converters.py
319–320

It's not clear what makes this manifest "not understood" by the custom parser?

vlorentz added inline comments.Jan 17 2022, 4:42 PM
swh/loader/git/converters.py
152–153

I don't know; it will most likely trigger the Expected revision to have id %s, but got %s. Recording raw_manifest warning anyway.

swh/loader/git/tests/test_converters.py
319–320

I didn't set it, so the manifest is just b""

olasd accepted this revision.Jan 17 2022, 4:44 PM

Thanks!

swh/loader/git/converters.py
152–153

Yeah, that's true.

swh/loader/git/tests/test_converters.py
319–320

Ah, I guess it's worth a comment then, because from a casual look this is a "well-formed" commit object.

This revision is now accepted and ready to land.Jan 17 2022, 4:44 PM
Closed by commit rDLDG4b77e567f80b: Stop passing 'offset' and 'negative_utc' to TimestampWithTimezone() (authored by vlorentz). · Explain WhyJan 18 2022, 11:02 AM
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDLDG4b77e567f80b: Stop passing 'offset' and 'negative_utc' to TimestampWithTimezone().

Revision Contents
Changeset List

PathSize
2 lines
swh/
loader/
git/
22 lines
tests/
91 lines

Diff 25171

View Options

requirements-swh.txt

Loading...
View Options

swh/loader/git/converters.py

Loading...
View Options

swh/loader/git/tests/test_converters.py

Loading...
About · Developer information · Legal