Page MenuHomeSoftware Heritage
Feed Advanced Search

Apr 7 2020

douardda created D2963: test: fix test_content_add_validation() for swh.model 0.0.64.
Apr 7 2020, 12:16 PM
douardda added a comment to T2310: Make origin visits immutable.

Thanks for the questions. I'm unsure about some questions and i replied as best
i could.

do we allow an OriginVisitUpdate(status='ongoing', snaphost=None)? what would
be the meaning of this?

Yes. It means "loading started, so no snapshot yet".
That sounds sensible ;)

Apr 7 2020, 10:56 AM · Storage manager, Data Model
douardda added a comment to T2310: Make origin visits immutable.

We currently don't have "created" (so no "start" either), but it would make sense to create it.

Regarding this model, a few questions come to my mind:

  • do we allow an OriginVisitUpdate(status='ongoing', snaphost=None)? what would be the meaning of this? or do we enforce one just after the created step to model the start transition?

This could mean these things:

  1. on a first update, to mean the visit was created (but we don't need it if we have a "created" state)
Apr 7 2020, 10:47 AM · Storage manager, Data Model

Apr 6 2020

douardda added a comment to T2310: Make origin visits immutable.

As I understand this, an origin visit, consisting in one OriginVisit object plus a list of OriginVisitUpdate represent the process of visiting an origin to load its content in the archive.

Apr 6 2020, 11:00 AM · Storage manager, Data Model

Apr 3 2020

douardda committed rDJNL7d6ead58a430: writer: make the writer use swh.model objects instead of (possibly) dicts (authored by douardda).
writer: make the writer use swh.model objects instead of (possibly) dicts
Apr 3 2020, 4:09 PM
douardda committed rDJNLf3112dbf7cce: Enforce current dev version of pyflakes in tox and pre-commit (authored by douardda).
Enforce current dev version of pyflakes in tox and pre-commit
Apr 3 2020, 4:09 PM
douardda closed D2956: writer: make the writer use swh.model objects instead of (possibly) dicts.
Apr 3 2020, 4:09 PM
douardda closed D2955: Update pre-commit config to use a 'local' configuration.
Apr 3 2020, 4:09 PM
douardda committed rDJNLa25e1222aed6: Add a setup.cfg to make flake8 compatible with blackified files (authored by douardda).
Add a setup.cfg to make flake8 compatible with blackified files
Apr 3 2020, 4:09 PM
douardda closed D2954: Add a setup.cfg to make flake8 compatible with blackified files.
Apr 3 2020, 4:09 PM
douardda updated the diff for D2956: writer: make the writer use swh.model objects instead of (possibly) dicts.

rebase + add comment on @overload usage

Apr 3 2020, 3:48 PM
douardda updated the diff for D2955: Update pre-commit config to use a 'local' configuration.

rebase + modify tox as well

Apr 3 2020, 3:48 PM
douardda updated the diff for D2954: Add a setup.cfg to make flake8 compatible with blackified files.

rebased

Apr 3 2020, 3:46 PM
douardda added inline comments to D2956: writer: make the writer use swh.model objects instead of (possibly) dicts.
Apr 3 2020, 3:45 PM
douardda closed D2957: Code cleanup to make flake8 happy.

closed by f6a398f99a5fa57e590fc4dfaf4fdc880eefba9d

Apr 3 2020, 3:40 PM
douardda updated the diff for D2957: Code cleanup to make flake8 happy.

rebased

Apr 3 2020, 3:38 PM
douardda committed rDJNLf6a398f99a5f: Code cleanup to make flake8 happy (authored by douardda).
Code cleanup to make flake8 happy
Apr 3 2020, 3:32 PM
douardda closed D2953: test: use Multihash to create a valid dict for a Content object.
Apr 3 2020, 3:22 PM
douardda committed rDJNL9c697f681289: test: use Multihash to create a valid dict for a Content object (authored by douardda).
test: use Multihash to create a valid dict for a Content object
Apr 3 2020, 3:22 PM
douardda created D2957: Code cleanup to make flake8 happy.
Apr 3 2020, 3:14 PM
douardda created D2956: writer: make the writer use swh.model objects instead of (possibly) dicts.
Apr 3 2020, 3:14 PM
douardda created D2955: Update pre-commit config to use a 'local' configuration.
Apr 3 2020, 3:12 PM
douardda created D2954: Add a setup.cfg to make flake8 compatible with blackified files.
Apr 3 2020, 3:11 PM
douardda created D2953: test: use Multihash to create a valid dict for a Content object.
Apr 3 2020, 3:07 PM
douardda accepted D2947: Remove magic CassObject class, use dicts instead..
Apr 3 2020, 11:32 AM

Apr 2 2020

douardda created P636 (An Untitled Masterwork).
Apr 2 2020, 4:22 PM
douardda created P635 (An Untitled Masterwork).
Apr 2 2020, 4:03 PM

Apr 1 2020

douardda closed T2308: Better Validation in swh.model as Resolved.

Let's consider this done with the landing of D2819 which adds runtime type validation.

Apr 1 2020, 3:43 PM · Data Model
douardda abandoned D2918: GitLoader: instantiate SnapshotBranch object using the from_dict factory.

see D2940

Apr 1 2020, 3:38 PM
douardda accepted D2940: Consistently use TargetType, instead of mixing it with strings..
Apr 1 2020, 3:38 PM
douardda accepted D2937: in_memory storage: Adapt internal implementations to use origin visit status model representation.

LGTM

Apr 1 2020, 3:21 PM
douardda closed D2906: model: add support for ctime in [Skipped]Content.from_[data,dict]().

closed by ca0f6a1eb7e6079ee0b8a649ff40353409c86560

Apr 1 2020, 3:12 PM
douardda closed D2884: model: fix SkippedContent origin to be a str.

closed by 414a6552143a1e2406bb980ed905969f4b778ad7

Apr 1 2020, 3:11 PM
douardda closed D2905: hypothesis: split hypothesis strategies as a dict + entity instance.

closed by f51327138fa14c5b285ffad2b2844afbb48e8462

Apr 1 2020, 3:10 PM
douardda closed D2927: test: convert test_converts.py to pytest style.

closed by 20baa1b0bb3359e3a91a249fa1bae4d78cf1ddf2

Apr 1 2020, 3:09 PM
douardda closed D2916: Adapt cassandra backend to validating model types.

closed by df3207a6b753b254e945db492227b3bb9bbac7d6

Apr 1 2020, 3:08 PM
douardda closed D2915: test: get rid of normalized_xxx in tests.

closed by fa4a043887e960731da5373f92e3f951ba411bc5

Apr 1 2020, 3:08 PM
douardda closed D2914: test: ensure timestamp in test data are properly typed.

closed by fcca905a95a262dc5596ff2afd71c657a1bcb522

Apr 1 2020, 3:07 PM
douardda committed rDMODca0f6a1eb7e6: model: add support for ctime in [Skipped]Content.from_[data,dict]() (authored by douardda).
model: add support for ctime in [Skipped]Content.from_[data,dict]()
Apr 1 2020, 2:54 PM
douardda committed rDMOD414a6552143a: model: small code improvement of SkippedContent.from_dict (authored by douardda).
model: small code improvement of SkippedContent.from_dict
Apr 1 2020, 2:54 PM
douardda committed rDMOD6ce0f71458d4: model: fix SkippedContent origin to be a str (authored by douardda).
model: fix SkippedContent origin to be a str
Apr 1 2020, 2:54 PM
douardda committed rDMODf51327138fa1: hypothesis: split hypothesis strategies as a dict + entity instance (authored by douardda).
hypothesis: split hypothesis strategies as a dict + entity instance
Apr 1 2020, 2:54 PM
douardda committed rDMOD10b069921e74: model: improve a bit the TimestampWithTimezone model (authored by douardda).
model: improve a bit the TimestampWithTimezone model
Apr 1 2020, 2:54 PM
douardda closed D2824: model: improve a bit the TimestampWithTimezone model.
Apr 1 2020, 2:54 PM
douardda committed rDMODac9d4c84a115: tests: add low level tests for the Timestamp model entity (authored by douardda).
tests: add low level tests for the Timestamp model entity
Apr 1 2020, 2:54 PM
douardda closed D2823: tests: add low level tests for the Timestamp model entity.
Apr 1 2020, 2:54 PM
douardda committed rDMOD85ca7d784800: model: use attrs_static to enforce type validation of model objects (authored by douardda).
model: use attrs_static to enforce type validation of model objects
Apr 1 2020, 2:54 PM
douardda closed D2819: model: use attrs_static to enforce type validation of model objects.
Apr 1 2020, 2:54 PM
douardda committed rDSTOdf3207a6b753: Adapt cassandra backend to validating model types (authored by douardda).
Adapt cassandra backend to validating model types
Apr 1 2020, 2:53 PM
douardda committed rDSTO20baa1b0bb33: test: convert test_converts.py to pytest style (authored by douardda).
test: convert test_converts.py to pytest style
Apr 1 2020, 2:53 PM
douardda committed rDSTOfa4a043887e9: test: get rid of normalized_xxx in tests (authored by douardda).
test: get rid of normalized_xxx in tests
Apr 1 2020, 2:53 PM
douardda committed rDSTOfcca905a95a2: test: ensure timestamp in test data are properly typed (authored by douardda).
test: ensure timestamp in test data are properly typed
Apr 1 2020, 2:53 PM
douardda added a comment to D2914: test: ensure timestamp in test data are properly typed.

it also misses tests to check date_to_db is used in release_to_db and revision_to_db

ok (but plz give all the infos/comments at once!)

It was a comment on your new version on the diff.

Apr 1 2020, 12:14 PM
douardda updated the diff for D2906: model: add support for ctime in [Skipped]Content.from_[data,dict]().

rebase

Apr 1 2020, 11:01 AM
douardda updated the diff for D2884: model: fix SkippedContent origin to be a str.

rebase

Apr 1 2020, 11:01 AM
douardda updated the diff for D2905: hypothesis: split hypothesis strategies as a dict + entity instance.

rebase + update for new origin visit updates entity

Apr 1 2020, 10:59 AM
douardda updated the diff for D2824: model: improve a bit the TimestampWithTimezone model.

rebas

Apr 1 2020, 10:59 AM
douardda updated the diff for D2823: tests: add low level tests for the Timestamp model entity.

rebase

Apr 1 2020, 10:58 AM
douardda updated the diff for D2819: model: use attrs_static to enforce type validation of model objects.

rebased

Apr 1 2020, 10:58 AM
douardda added a comment to D2914: test: ensure timestamp in test data are properly typed.

it also misses tests to check date_to_db is used in release_to_db and revision_to_db

ok (but plz give all the infos/comments at once!)

Apr 1 2020, 10:14 AM
douardda added a comment to D2914: test: ensure timestamp in test data are properly typed.

it also misses tests to check date_to_db is used in release_to_db and revision_to_db

Apr 1 2020, 10:07 AM
douardda committed rDJNLca7e9d20c5c8: test: reimplement a local version of 'object_dicts' with 'skipped_content' (authored by douardda).
test: reimplement a local version of 'object_dicts' with 'skipped_content'
Apr 1 2020, 10:04 AM
douardda committed rDJNLcd0485050688: test: ensure test data is properly typed (authored by douardda).
test: ensure test data is properly typed
Apr 1 2020, 10:04 AM
douardda closed D2928: test: ensure tests are compatible with next swh.model release.
Apr 1 2020, 10:04 AM
douardda accepted D2880: model: Add OriginVisitUpdate model object.
Apr 1 2020, 9:59 AM

Mar 31 2020

douardda accepted D2931: docker tests: Handle tasks in `scheduled` state.
Mar 31 2020, 12:08 PM
douardda updated the diff for D2906: model: add support for ctime in [Skipped]Content.from_[data,dict]().

add a test for ctime as string in Content.from_dict()

Mar 31 2020, 10:46 AM
douardda claimed T2308: Better Validation in swh.model .
Mar 31 2020, 9:16 AM · Data Model

Mar 30 2020

douardda committed rDLDGa600855a1234: converters: fix types of TimestampWithTimezone arguments (authored by douardda).
converters: fix types of TimestampWithTimezone arguments
Mar 30 2020, 5:27 PM
douardda closed D2917: converters: fix types of TimestampWithTimezone arguments.
Mar 30 2020, 5:27 PM
douardda added a comment to D2918: GitLoader: instantiate SnapshotBranch object using the from_dict factory.

DULWICH_TARGET_TYPES are already the right enum values.

Mar 30 2020, 5:24 PM
douardda updated the diff for D2928: test: ensure tests are compatible with next swh.model release.

Typo in the first commit message

Mar 30 2020, 5:18 PM
douardda created D2928: test: ensure tests are compatible with next swh.model release.
Mar 30 2020, 5:16 PM
douardda updated the diff for D2906: model: add support for ctime in [Skipped]Content.from_[data,dict]().

with the actual revision merged in...

Mar 30 2020, 3:45 PM
douardda committed rDCIDXc95269d3597f: test: make test data properly typed (authored by douardda).
test: make test data properly typed
Mar 30 2020, 3:39 PM
douardda closed D2919: test: make test data properly typed.
Mar 30 2020, 3:39 PM
douardda created D2927: test: convert test_converts.py to pytest style.
Mar 30 2020, 3:36 PM
douardda added a comment to D2916: Adapt cassandra backend to validating model types.

This is going in the right direction, but I don't like the CassObject thing. I think you made it because you are conflating two uses of my "Frankenstein's model objects":

  1. Reads, where you can use the row directly (it's a namedtuple, so attr access)
  2. Writes, where a dict would do (item access)
Mar 30 2020, 3:27 PM
douardda updated the diff for D2914: test: ensure timestamp in test data are properly typed.

add a test for date_to_db so the test coverage of this later stays the same

Mar 30 2020, 3:21 PM
douardda committed rDSTO623a1b75cb97: test: add a small test to check for type validation (using release_add) (authored by douardda).
test: add a small test to check for type validation (using release_add)
Mar 30 2020, 3:20 PM
douardda committed rDSTO90c411266897: validate: fix type annotation for origin_visit_add (authored by douardda).
validate: fix type annotation for origin_visit_add
Mar 30 2020, 3:20 PM
douardda closed D2911: validate: fix type annotation for origin_visit_add.
Mar 30 2020, 3:20 PM
douardda closed D2912: test: add a small test to check for type validation (using release_add).
Mar 30 2020, 3:20 PM
douardda committed rDSTO1916fd71f26f: validate: ensure StorageArgumentException is always encodable (authored by douardda).
validate: ensure StorageArgumentException is always encodable
Mar 30 2020, 3:20 PM
douardda closed D2910: validate: ensure StorageArgumentException is always encodable.
Mar 30 2020, 3:20 PM
douardda committed rDSTOc67fe217f006: writer: fix skipped_content_add type declaration to use SkippedContent (authored by douardda).
writer: fix skipped_content_add type declaration to use SkippedContent
Mar 30 2020, 3:20 PM
douardda closed D2909: writer: fix skipped_content_add type declaration to use SkippedContent.
Mar 30 2020, 3:20 PM
douardda committed rDSTO19be96fa306a: tests: fix types of several test data sets (authored by douardda).
tests: fix types of several test data sets
Mar 30 2020, 3:20 PM
douardda closed D2908: tests: fix types of several test data sets.
Mar 30 2020, 3:20 PM
douardda updated the diff for D2906: model: add support for ctime in [Skipped]Content.from_[data,dict]().

Rebase + comments from vlorentz

Mar 30 2020, 11:28 AM
douardda updated the diff for D2884: model: fix SkippedContent origin to be a str.

Rebase

Mar 30 2020, 11:27 AM
douardda updated the diff for D2905: hypothesis: split hypothesis strategies as a dict + entity instance.

Get rid of unnecessary calls to builds() as reported by vlorentz

Mar 30 2020, 10:59 AM
douardda added a comment to D2905: hypothesis: split hypothesis strategies as a dict + entity instance.

You should rewrite all the object-generating strategies:

def releases():
    return builds(
        Release.from_dict,
        releases_d())

becomes:

releases = releases_d().map(Release.from_dict)

build() is only useful if you want hypothesis to read the constructor's signature.

Mar 30 2020, 10:47 AM
douardda accepted D2923: bin/update add support for (re)installing packages after update.
Mar 30 2020, 10:17 AM

Mar 27 2020

douardda created D2919: test: make test data properly typed.
Mar 27 2020, 5:05 PM
douardda created D2918: GitLoader: instantiate SnapshotBranch object using the from_dict factory.
Mar 27 2020, 4:36 PM
douardda created D2917: converters: fix types of TimestampWithTimezone arguments.
Mar 27 2020, 4:36 PM
douardda created D2916: Adapt cassandra backend to validating model types.
Mar 27 2020, 4:12 PM
douardda created D2915: test: get rid of normalized_xxx in tests.
Mar 27 2020, 4:12 PM
douardda created D2914: test: ensure timestamp in test data are properly typed.
Mar 27 2020, 4:11 PM