Page MenuHomeSoftware Heritage

vlorentz (Valentin Lorentz)
User

Projects

User Details

User Since
Oct 1 2018, 11:23 AM (19 w, 6 d)

Recent Activity

Fri, Feb 15

vlorentz added inline comments to D1146: cli: add a --runs option to 'swh-scheduler task list' tool.
Fri, Feb 15, 6:06 PM
vlorentz requested changes to D1146: cli: add a --runs option to 'swh-scheduler task list' tool.

The new API endpoints don't have tests.

Fri, Feb 15, 6:06 PM
vlorentz added a comment to D1148: config: use already loaded swh config, if any, when instantiating an Indexer.

I don't see any variable named CONFIG in swh.scheduler outside of swh/scheduler/celery_backend/config.py.

Fri, Feb 15, 5:54 PM
vlorentz accepted D1145: celery_backend: remove debug statements and reduce info level verbosity.
Fri, Feb 15, 5:12 PM
vlorentz committed rDSTO152cb528234f: Move origin_get_range's right after origin_get. (authored by vlorentz).
Move origin_get_range's right after origin_get.
Fri, Feb 15, 4:30 PM
vlorentz closed D1144: Move `origin_get_range`'s right after `origin_get`..
Fri, Feb 15, 4:30 PM
vlorentz committed rDCIDX064f1eb047c5: Add missing documentation of 'mappings' arguments in the in-mem storage. (authored by vlorentz).
Add missing documentation of 'mappings' arguments in the in-mem storage.
Fri, Feb 15, 4:30 PM
vlorentz closed D1143: Add missing documentation of 'mappings' arguments in the in-mem storage..
Fri, Feb 15, 4:30 PM
vlorentz updated the diff for D1143: Add missing documentation of 'mappings' arguments in the in-mem storage..
  • Fix indent
  • Squash
Fri, Feb 15, 4:30 PM
Herald added a reviewer for D1144: Move `origin_get_range`'s right after `origin_get`.: Reviewers.
Fri, Feb 15, 3:10 PM
Herald added a reviewer for D1143: Add missing documentation of 'mappings' arguments in the in-mem storage.: Reviewers.
Fri, Feb 15, 2:58 PM
Herald added a reviewer for D1142: Fix documentation of CLI option --verbose.: Reviewers.
Fri, Feb 15, 11:24 AM

Thu, Feb 14

vlorentz committed rDCIDXd003f0b2a042: Don't send 'None' as a revision id to storage.revision_get. (authored by vlorentz).
Don't send 'None' as a revision id to storage.revision_get.
Thu, Feb 14, 5:15 PM
vlorentz closed D1141: Don't send 'None' as a revision id to storage.revision_get..
Thu, Feb 14, 5:15 PM
Herald added a reviewer for D1141: Don't send 'None' as a revision id to storage.revision_get.: Reviewers.
Thu, Feb 14, 5:09 PM
vlorentz committed rDCIDXa2da900a465e: Fix deduplication of origins when persisting origin intrinsic metadata. (authored by vlorentz).
Fix deduplication of origins when persisting origin intrinsic metadata.
Thu, Feb 14, 3:18 PM
vlorentz committed rDCIDXd4238d332f38: Fix duplication checking in the in-mem implementation of IndexerStorage. (authored by vlorentz).
Fix duplication checking in the in-mem implementation of IndexerStorage.
Thu, Feb 14, 3:18 PM
vlorentz closed D1138: Fix duplication checking in the in-mem implementation of IndexerStorage. origin_intrinsic_metadata_add..
Thu, Feb 14, 3:18 PM
vlorentz updated the diff for D1138: Fix duplication checking in the in-mem implementation of IndexerStorage. origin_intrinsic_metadata_add..
  • rebase
Thu, Feb 14, 3:16 PM
vlorentz committed rDCIDXeca69ce7543b: Fix mismatch between origins and head_revs. (authored by vlorentz).
Fix mismatch between origins and head_revs.
Thu, Feb 14, 3:16 PM
vlorentz closed D1139: Fix mismatch between origins and head_revs..
Thu, Feb 14, 3:16 PM
vlorentz updated the diff for D1139: Fix mismatch between origins and head_revs..
  • rebase
Thu, Feb 14, 3:16 PM
vlorentz committed rDCIDX65a31e93fcc4: Remove files that weren't supposed to be committed yet. (authored by vlorentz).
Remove files that weren't supposed to be committed yet.
Thu, Feb 14, 3:15 PM
vlorentz closed D1140: Remove files that weren't supposed to be committed yet..
Thu, Feb 14, 3:15 PM
vlorentz added a comment to D1140: Remove files that weren't supposed to be committed yet..

i missed those add instructions in the previous diffs, sorry.

Thu, Feb 14, 3:15 PM
Herald added a reviewer for D1140: Remove files that weren't supposed to be committed yet.: Reviewers.
Thu, Feb 14, 3:12 PM
Herald added a reviewer for D1139: Fix mismatch between origins and head_revs.: Reviewers.
Thu, Feb 14, 3:11 PM
vlorentz committed rDCIDX795bc1daca3e: Fix value of origin_id when more than one origin is indexed (authored by vlorentz).
Fix value of origin_id when more than one origin is indexed
Thu, Feb 14, 2:47 PM
vlorentz closed D1137: Fix value of origin_id when more than one origin is indexed.
Thu, Feb 14, 2:47 PM
vlorentz updated the diff for D1137: Fix value of origin_id when more than one origin is indexed.
  • rebase
Thu, Feb 14, 2:47 PM
vlorentz committed rDCIDX2354d9bb313b: Fix crash when an origin has no HEAD. (authored by vlorentz).
Fix crash when an origin has no HEAD.
Thu, Feb 14, 2:46 PM
vlorentz closed D1136: Fix crash when an origin has no HEAD..
Thu, Feb 14, 2:46 PM
vlorentz updated the diff for D1138: Fix duplication checking in the in-mem implementation of IndexerStorage. origin_intrinsic_metadata_add..
  • Fix deduplication of origins when persisting origin intrinsic metadata.
Thu, Feb 14, 2:45 PM
Herald added a reviewer for D1138: Fix duplication checking in the in-mem implementation of IndexerStorage. origin_intrinsic_metadata_add.: Reviewers.
Thu, Feb 14, 2:43 PM
vlorentz updated the diff for D1137: Fix value of origin_id when more than one origin is indexed.
  • Fix commit message
Thu, Feb 14, 2:23 PM
Herald added a reviewer for D1137: Fix value of origin_id when more than one origin is indexed: Reviewers.
Thu, Feb 14, 2:23 PM
Herald added a reviewer for D1136: Fix crash when an origin has no HEAD.: Reviewers.
Thu, Feb 14, 2:07 PM
vlorentz updated the diff for D1121: Generate pom.xml files ourselves instead of using xmltodict.unparse..
  • Add test for tree_to_xml.
Thu, Feb 14, 1:46 PM
vlorentz added a comment to T1528: Efficient reindex when adding a metadata mapping.

Or, as the OriginMetadataIndexer already fetches this data anyway, it could write it to the indexer db when it's done.

Thu, Feb 14, 12:01 PM · Indexer
vlorentz added a comment to D1121: Generate pom.xml files ourselves instead of using xmltodict.unparse..

It'd be good to have tests around the xml_document_strategy since it's used for tests though.

Thu, Feb 14, 11:32 AM
vlorentz added a comment to T1529: Efficient reindex when updating a metadata mapping.

Should be easy to do, we already have this info in the indexer db (in origin_intrinsic_metadata.mappings). Then it's just a matter of creating oneshot tasks.

Thu, Feb 14, 11:16 AM · Indexer
vlorentz added a comment to T1528: Efficient reindex when adding a metadata mapping.

The naive solution to do this is adding a new indexer that pre-fetches snapshot+revision+root dir of an origin and writes its list of root files in the indexer db. Then we can read that to find which origins have a given file name pattern.

Thu, Feb 14, 11:15 AM · Indexer
vlorentz triaged T1529: Efficient reindex when updating a metadata mapping as Normal priority.
Thu, Feb 14, 11:12 AM · Indexer
vlorentz triaged T1528: Efficient reindex when adding a metadata mapping as Low priority.
Thu, Feb 14, 11:11 AM · Indexer
vlorentz committed rDCIDXb43f534ec9e3: Document the 'mappings' columns of revision_metadata and… (authored by vlorentz).
Document the 'mappings' columns of revision_metadata and…
Thu, Feb 14, 11:09 AM
vlorentz closed D1133: Document the 'mappings' columns of revision_metadata and origin_intrinsic_metadata..
Thu, Feb 14, 11:09 AM
vlorentz updated the diff for D1133: Document the 'mappings' columns of revision_metadata and origin_intrinsic_metadata..
  • rebase
Thu, Feb 14, 11:09 AM
vlorentz added a comment to D1133: Document the 'mappings' columns of revision_metadata and origin_intrinsic_metadata..
In D1133#24087, @zack wrote:

Speaking of which, it would be nice to have comments on all columns on all tables of all databases…

Fancy having a look at that or, alternatively, file a bug to track the need of doing so?

Thu, Feb 14, 11:09 AM
vlorentz triaged T1527: Have comments on all columns of all databases as Normal priority.
Thu, Feb 14, 11:08 AM · Storage manager, Scheduling utilities, Indexer
Herald added a reviewer for D1133: Document the 'mappings' columns of revision_metadata and origin_intrinsic_metadata.: Reviewers.
Thu, Feb 14, 10:58 AM
vlorentz updated the diff for D1121: Generate pom.xml files ourselves instead of using xmltodict.unparse..

Reword commits:

Thu, Feb 14, 10:49 AM
vlorentz added a comment to D1122: Use xmlschema instead of xmltodict for parsing pom.xml..
  • Commit missing stuff.
  • Fix commit again.
  • Fix requirements.txt
  • Add missing tests.

    I don't buy it! Please use 'git rebase -i'!
Thu, Feb 14, 10:44 AM

Wed, Feb 13

vlorentz updated the summary of D1122: Use xmlschema instead of xmltodict for parsing pom.xml..
Wed, Feb 13, 3:22 PM
Herald added a reviewer for D1126: Catch exceptions raised by elementpath.: Reviewers.
Wed, Feb 13, 1:23 PM
vlorentz added a child revision for D1123: Use schema validation for pom.xml: D1126: Catch exceptions raised by elementpath..
Wed, Feb 13, 1:23 PM
vlorentz retitled D1123: Use schema validation for pom.xml from Use schema validation. to Use schema validation for pom.xml.
Wed, Feb 13, 12:06 PM
vlorentz committed rDCIDX801d43e44a5d: Fix gemspec test. (authored by vlorentz).
Fix gemspec test.
Wed, Feb 13, 11:23 AM
vlorentz closed D1120: Fix gemspec test..
Wed, Feb 13, 11:23 AM
vlorentz updated the diff for D1120: Fix gemspec test..
  • rebase
Wed, Feb 13, 11:22 AM
vlorentz committed rDCIDX29e200839267: Break dependency of swh.indexer.storage on swh.indexer. (authored by vlorentz).
Break dependency of swh.indexer.storage on swh.indexer.
Wed, Feb 13, 11:22 AM
vlorentz closed D1125: Break dependency of swh.indexer.storage on swh.indexer..
Wed, Feb 13, 11:22 AM
vlorentz requested changes to D1119: indexer_storage: Work around deployment issue.

Proper resolution: D1125

Wed, Feb 13, 11:17 AM
Herald added a reviewer for D1125: Break dependency of swh.indexer.storage on swh.indexer.: Reviewers.
Wed, Feb 13, 11:15 AM
vlorentz accepted D1119: indexer_storage: Work around deployment issue.
Wed, Feb 13, 10:27 AM

Tue, Feb 12

vlorentz updated the diff for D1122: Use xmlschema instead of xmltodict for parsing pom.xml..
  • Rebase
Tue, Feb 12, 7:03 PM
vlorentz abandoned D1124: Add an hypothesis strategy to generate pom.xml documents..

Duplicate of D1121.

Tue, Feb 12, 7:01 PM
vlorentz updated the diff for D1124: Add an hypothesis strategy to generate pom.xml documents..
  • rebase
Tue, Feb 12, 6:59 PM
Herald added a reviewer for D1124: Add an hypothesis strategy to generate pom.xml documents.: Reviewers.
Tue, Feb 12, 6:58 PM
vlorentz updated the diff for D1122: Use xmlschema instead of xmltodict for parsing pom.xml..
  • Fix base commit
Tue, Feb 12, 6:54 PM
Herald added a reviewer for D1123: Use schema validation for pom.xml: Reviewers.
Tue, Feb 12, 6:54 PM
vlorentz added a child revision for D1122: Use xmlschema instead of xmltodict for parsing pom.xml.: D1123: Use schema validation for pom.xml.
Tue, Feb 12, 6:54 PM
Herald added a reviewer for D1122: Use xmlschema instead of xmltodict for parsing pom.xml.: Reviewers.
Tue, Feb 12, 6:53 PM
vlorentz added a child revision for D1121: Generate pom.xml files ourselves instead of using xmltodict.unparse.: D1122: Use xmlschema instead of xmltodict for parsing pom.xml..
Tue, Feb 12, 6:53 PM
vlorentz updated the diff for D1121: Generate pom.xml files ourselves instead of using xmltodict.unparse..
  • More keys.
Tue, Feb 12, 6:31 PM
Herald added a reviewer for D1121: Generate pom.xml files ourselves instead of using xmltodict.unparse.: Reviewers.
Tue, Feb 12, 6:29 PM
Herald added a reviewer for D1120: Fix gemspec test.: Reviewers.
Tue, Feb 12, 6:23 PM
vlorentz committed rDARCee85027276dd: Use the new BaseDb from swh.core, drop dependency on swh.storage. (authored by vlorentz).
Use the new BaseDb from swh.core, drop dependency on swh.storage.
Tue, Feb 12, 2:24 PM
vlorentz closed D1116: Use the new BaseDb from swh.core, drop dependency on swh.storage..
Tue, Feb 12, 2:24 PM
vlorentz committed rDSTO1efbe5c16b27: Fix test to work with the latest version of swh.core. (authored by vlorentz).
Fix test to work with the latest version of swh.core.
Tue, Feb 12, 2:21 PM
vlorentz closed D1118: Fix test to work with the latest version of swh.core..
Tue, Feb 12, 2:21 PM
vlorentz added a comment to D1075: Storage: Add origin_count method.

D1118

Tue, Feb 12, 2:19 PM
Herald added a reviewer for D1118: Fix test to work with the latest version of swh.core.: Reviewers.
Tue, Feb 12, 2:18 PM
vlorentz added a comment to D1075: Storage: Add origin_count method.

Oh wait, my mistake. I tagged a new version of swh.core that broke that particular test.

Tue, Feb 12, 2:12 PM
vlorentz added a comment to D1075: Storage: Add origin_count method.

You need to rebase on master to fix the build failure.

Tue, Feb 12, 2:11 PM
Herald added a reviewer for D1116: Use the new BaseDb from swh.core, drop dependency on swh.storage.: Reviewers.
Tue, Feb 12, 1:44 PM
vlorentz committed rDCIDXfdf904ecb5e6: Fix rebase. (authored by vlorentz).
Fix rebase.
Tue, Feb 12, 12:24 PM
vlorentz committed rDCIDX9657abaf20a2: Prevent OriginMetadataIndexer from sending duplicate revisions to… (authored by vlorentz).
Prevent OriginMetadataIndexer from sending duplicate revisions to…
Tue, Feb 12, 12:23 PM
vlorentz closed D1081: Prevent OriginMetadataIndexer from sending duplicate revisions to revision_metadata_add..
Tue, Feb 12, 12:23 PM
vlorentz updated the diff for D1081: Prevent OriginMetadataIndexer from sending duplicate revisions to revision_metadata_add..
  • rebase
Tue, Feb 12, 12:22 PM
vlorentz committed rDCIDX0f75ae1e7346: Use hypothesis to generate adversarial inputs. (authored by vlorentz).
Use hypothesis to generate adversarial inputs.
Tue, Feb 12, 12:20 PM
vlorentz committed rDCIDXa0f6fe3ea02d: Fix bugs found by hypothesis. (authored by vlorentz).
Fix bugs found by hypothesis.
Tue, Feb 12, 12:20 PM
vlorentz committed rDCIDXd3a6ae1930ff: Add more type checks in metadata dictionary. (authored by vlorentz).
Add more type checks in metadata dictionary.
Tue, Feb 12, 12:20 PM
vlorentz closed D1077: Add more type checks in metadata dictionary..
Tue, Feb 12, 12:20 PM
vlorentz updated the diff for D1077: Add more type checks in metadata dictionary..
  • Rebase + squash
Tue, Feb 12, 12:19 PM
vlorentz committed rDCORE021746d08212: Add tests for swh.core.db. (authored by vlorentz).
Add tests for swh.core.db.
Tue, Feb 12, 12:16 PM
vlorentz committed rDCOREc80a4afffdeb: Add test for BaseDb.connect. (authored by vlorentz).
Add test for BaseDb.connect.
Tue, Feb 12, 12:16 PM
vlorentz closed D1111: Add tests for swh.core.db..
Tue, Feb 12, 12:16 PM
vlorentz updated the diff for D1081: Prevent OriginMetadataIndexer from sending duplicate revisions to revision_metadata_add..
  • rebase
Tue, Feb 12, 12:15 PM
vlorentz committed rDCIDXfd4c400d9299: Move boilerplate out of test_origin_metadata_indexer so it's reusable by other… (authored by vlorentz).
Move boilerplate out of test_origin_metadata_indexer so it's reusable by other…
Tue, Feb 12, 12:15 PM
vlorentz closed D1106: Move boilerplate out of test_origin_metadata_indexer so it's reusable by other tests..
Tue, Feb 12, 12:15 PM