Page MenuHomeSoftware Heritage
Differential D2905

hypothesis: split hypothesis strategies as a dict + entity instance
ClosedPublic
Actions

Authored by douardda on Mar 27 2020, 3:04 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Summary

for each entity model Model, provide a models_d strategy that
produces dicts suitable for using as argument for the Model.from_dict
factory method, and reimplement the models generator using this
former hypothesis generator.

This is needed to help writing low level tests for model entities.

Depends on D2824.

Diff Detail

Repository
rDMOD Data model
Branch
typecheck
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 11417
Build 17288: tox-on-jenkinsJenkins
Build 17287: arc lint + arc unit

Event Timeline

douardda created this revision.Mar 27 2020, 3:04 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptMar 27 2020, 3:04 PM
Harbormaster failed remote builds in B11417: Diff 10338!Mar 27 2020, 3:04 PM
swh-public-ci added a comment.Mar 27 2020, 3:04 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DMOD/job/tox/229/
See console output for more information: https://jenkins.softwareheritage.org/job/DMOD/job/tox/229/console

douardda mentioned this in D2884: model: fix SkippedContent origin to be a str.Mar 27 2020, 3:05 PM
douardda added a child revision: D2884: model: fix SkippedContent origin to be a str.Mar 27 2020, 3:06 PM
vlorentz requested changes to this revision.Mar 27 2020, 4:39 PM
vlorentz added a subscriber: vlorentz.

You should rewrite all the object-generating strategies:

def releases():
    return builds(
        Release.from_dict,
        releases_d())

becomes:

releases = releases_d().map(Release.from_dict)

build() is only useful if you want hypothesis to read the constructor's signature.

This revision now requires changes to proceed.Mar 27 2020, 4:39 PM
douardda added a comment.Mar 30 2020, 10:47 AM
In D2905#70213, @vlorentz wrote:

You should rewrite all the object-generating strategies:

def releases():
    return builds(
        Release.from_dict,
        releases_d())

becomes:

releases = releases_d().map(Release.from_dict)

build() is only useful if you want hypothesis to read the constructor's signature.

But then one would need not to call the MappedSearchStrategy, so instead of writing:

@given(releases())
def so_something():
   pass

you have to write

@given(releases)
def do_something():
   pass

making these strategies usage pretty inconsistent and difficult to use (unless I missed something).
So for now I propose to keep the function but without using builds() when not needed.

douardda updated this revision to Diff 10372.Mar 30 2020, 10:59 AM

Get rid of unnecessary calls to builds() as reported by vlorentz

swh-public-ci added a comment.Mar 30 2020, 10:59 AM

Build has FAILED

Patch application report for D2905 (id=10372)

Could not rebase; Attempt merge onto 4a2233c5f7...

Updating 4a2233c..369e0b6
Fast-forward
 mypy.ini                                      |   4 +-
 requirements.txt                              |   1 +
 swh/model/hypothesis_strategies.py            | 257 +++++++++++++++-------
 swh/model/model.py                            | 300 ++++++++++++++++++--------
 swh/model/tests/test_hypothesis_strategies.py |  28 ++-
 swh/model/tests/test_model.py                 | 118 ++++++++++
 6 files changed, 540 insertions(+), 168 deletions(-)
Changes applied before test
commit 369e0b6892bf6d4633cb0234b7371d4b76055a55
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Mar 23 10:32:39 2020 +0100

    hypothesis: split hypothesis strategies as a dict + entity instance
    
    for each entity model `Model`, provide a `models_d` strategy that
    produces dicts suitable for using as argument for the `Model.from_dict`
    factory method, and reimplement the `models` generator using this
    former hypothesis generator.
    
    This is needed to help writing low level tests for model entities.

commit 28834a6c37b6dfb381831aef3809f407a7c0f787
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Mar 12 16:01:55 2020 +0100

    model: improve a bit the TimestampWithTimezone model
    
    - add a validator for negative_utc (can be True iff offset is 0),
    - update the timestamps_with_timezone hypothesis strategy,
    - add low-level tests for it.

commit ad41de7323ed63f458f3734ba95c4dceb045accc
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Mar 12 14:27:23 2020 +0100

    tests: add low level tests for the Timestamp model entity

commit 96d279e3596a70187bc80fe3bc8aa9ea0a7d94e6
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Mar 20 12:59:56 2020 +0100

    model: use attrs_static to enforce type validation of model objects
    
    This ensures all instanciated model entities have valid types for attributes.
    
    Related to T2308.

Link to build: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/5/
See console output for more information: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/5/console

Harbormaster failed remote builds in B11448: Diff 10372!Mar 30 2020, 10:59 AM
swh-public-ci added a comment.Mar 30 2020, 11:44 AM

Build is green

Patch application report for D2905 (id=10372)

Could not rebase; Attempt merge onto 4a2233c5f7...

Updating 4a2233c..369e0b6
Fast-forward
 mypy.ini                                      |   4 +-
 requirements.txt                              |   1 +
 swh/model/hypothesis_strategies.py            | 257 +++++++++++++++-------
 swh/model/model.py                            | 300 ++++++++++++++++++--------
 swh/model/tests/test_hypothesis_strategies.py |  28 ++-
 swh/model/tests/test_model.py                 | 118 ++++++++++
 6 files changed, 540 insertions(+), 168 deletions(-)
Changes applied before test
commit 369e0b6892bf6d4633cb0234b7371d4b76055a55
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Mar 23 10:32:39 2020 +0100

    hypothesis: split hypothesis strategies as a dict + entity instance
    
    for each entity model `Model`, provide a `models_d` strategy that
    produces dicts suitable for using as argument for the `Model.from_dict`
    factory method, and reimplement the `models` generator using this
    former hypothesis generator.
    
    This is needed to help writing low level tests for model entities.

commit 28834a6c37b6dfb381831aef3809f407a7c0f787
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Mar 12 16:01:55 2020 +0100

    model: improve a bit the TimestampWithTimezone model
    
    - add a validator for negative_utc (can be True iff offset is 0),
    - update the timestamps_with_timezone hypothesis strategy,
    - add low-level tests for it.

commit ad41de7323ed63f458f3734ba95c4dceb045accc
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Mar 12 14:27:23 2020 +0100

    tests: add low level tests for the Timestamp model entity

commit 96d279e3596a70187bc80fe3bc8aa9ea0a7d94e6
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Mar 20 12:59:56 2020 +0100

    model: use attrs_static to enforce type validation of model objects
    
    This ensures all instanciated model entities have valid types for attributes.
    
    Related to T2308.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/11/ for more details.

Harbormaster completed remote builds in B11448: Diff 10372.Mar 30 2020, 11:44 AM
vlorentz accepted this revision.Mar 30 2020, 1:01 PM

Indeed, I didn't consider the inconsistency.

This revision is now accepted and ready to land.Mar 30 2020, 1:01 PM
douardda updated this revision to Diff 10443.Apr 1 2020, 10:59 AM

rebase + update for new origin visit updates entity

swh-public-ci added a comment.Apr 1 2020, 11:01 AM

Build is green

Patch application report for D2905 (id=10443)

Could not rebase; Attempt merge onto e9a4c7519e...

Updating e9a4c75..ada507f
Fast-forward
 mypy.ini                                      |   4 +-
 requirements.txt                              |   1 +
 swh/model/hypothesis_strategies.py            | 271 ++++++++++++++-------
 swh/model/model.py                            | 324 ++++++++++++++++++--------
 swh/model/tests/test_hypothesis_strategies.py |  28 ++-
 swh/model/tests/test_model.py                 | 121 +++++++++-
 6 files changed, 568 insertions(+), 181 deletions(-)
Changes applied before test
commit ada507fddd4b0b51d6412240232ed6f3eab2d207
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Mar 23 10:32:39 2020 +0100

    hypothesis: split hypothesis strategies as a dict + entity instance
    
    for each entity model `Model`, provide a `models_d` strategy that
    produces dicts suitable for using as argument for the `Model.from_dict`
    factory method, and reimplement the `models` generator using this
    former hypothesis generator.
    
    This is needed to help writing low level tests for model entities.

commit 10b069921e74f0d0411fb105349471e0f9a79f29
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Mar 12 16:01:55 2020 +0100

    model: improve a bit the TimestampWithTimezone model
    
    - add a validator for negative_utc (can be True iff offset is 0),
    - update the timestamps_with_timezone hypothesis strategy,
    - add low-level tests for it.

commit ac9d4c84a115c6c8d3aea04ed55635e8e743ea1d
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Mar 12 14:27:23 2020 +0100

    tests: add low level tests for the Timestamp model entity

commit 85ca7d7848008951f2e26c55c1c72ed9fa92cefb
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Mar 20 12:59:56 2020 +0100

    model: use attrs_static to enforce type validation of model objects
    
    This ensures all instanciated model entities have valid types for attributes.
    
    Related to T2308.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/21/ for more details.

Harbormaster completed remote builds in B11526: Diff 10443.Apr 1 2020, 11:01 AM
douardda closed this revision.Apr 1 2020, 3:10 PM

closed by f51327138fa14c5b285ffad2b2844afbb48e8462

Revision Contents
Changeset List

PathSize
swh/
model/
264 lines
tests/
28 lines

Diff 10338

View Options

swh/model/hypothesis_strategies.py

Loading...
View Options

swh/model/tests/test_hypothesis_strategies.py

Loading...
About · Developer information · Legal