Page MenuHomeSoftware Heritage

vlorentz (Valentin Lorentz)
User

Projects

User Details

User Since
Oct 1 2018, 11:23 AM (86 w, 4 d)

Recent Activity

Yesterday

vlorentz added a comment to D3203: Replace MockedJournalClient and MockedKafkaWriter by proper kafka test scaffolding.

This also kills test_write_replay.py file since it does not test anything
more than what is currently tested in test_replay.py.

Fri, May 29, 4:15 PM
vlorentz accepted D3202: Fix InMemoryStorage.origin_visit_upsert() method.
Fri, May 29, 4:04 PM
vlorentz added a comment to D3197: swh.deposit.models: Upload deposit archives to dedicated folder.

Or you could just use a random value every time.

Fri, May 29, 3:56 PM
vlorentz accepted D3201: Do not reference unloaded snapshots in origin_visit_update.
Fri, May 29, 3:49 PM
vlorentz added a comment to D3197: swh.deposit.models: Upload deposit archives to dedicated folder.

The proposal is good imo. Maybe add milliseconds just to be sure though, in case the client is scripting multiple uploads at the same time

Fri, May 29, 3:48 PM
vlorentz abandoned D3192: Disambiguate uploaded files with a prefix instead of a suffix..
Fri, May 29, 3:46 PM
vlorentz requested changes to D3197: swh.deposit.models: Upload deposit archives to dedicated folder.

I think you should change check_archive to detect Django's name changes (see D3192)

Fri, May 29, 3:44 PM
vlorentz accepted D3197: swh.deposit.models: Upload deposit archives to dedicated folder.

No test?

Fri, May 29, 3:42 PM
vlorentz added a comment to D3201: Do not reference unloaded snapshots in origin_visit_update.

I wonder if there should be a test checking the snapshot isn't referenced if Storage.snapshot_add errors

Fri, May 29, 3:06 PM
vlorentz accepted D3200: Refactor DVCS loader tests to actually run the DVCSLoader store_data method.

this should be two commits, though

Fri, May 29, 3:02 PM
vlorentz accepted D3199: Make the test_loader origin a global variable to be able to reuse it later.
Fri, May 29, 3:01 PM
vlorentz accepted D3198: Drop useless fs mark on test that does not use any test fixtures.
Fri, May 29, 3:00 PM
vlorentz renamed T2393: Two deposits of files the same name and a "compound" extension fails the checker from Two deposits of files the same name and a "double" extension fails the checker to Two deposits of files the same name and a "compound" extension fails the checker.
Fri, May 29, 9:58 AM · SWORD deposit

Thu, May 28

vlorentz committed rDSTO6c6080b4d70c: Fix type annotation. (authored by vlorentz).
Fix type annotation.
Thu, May 28, 5:03 PM
vlorentz committed rDSTO9332547c7410: Remove function drops from the migration. (authored by vlorentz).
Remove function drops from the migration.
Thu, May 28, 4:38 PM
vlorentz closed D3195: Remove function drops from the migration..
Thu, May 28, 4:38 PM
vlorentz created D3195: Remove function drops from the migration..
Thu, May 28, 4:35 PM
vlorentz accepted D3188: Add a default target for symbolic reference branches.
Thu, May 28, 4:14 PM
vlorentz added a comment to D3188: Add a default target for symbolic reference branches.

"ok then"

Thu, May 28, 4:14 PM
vlorentz planned changes to D3194: [WIP] Start rewriting the in-mem storage as a backend for cassandra..
Thu, May 28, 4:12 PM
vlorentz created D3194: [WIP] Start rewriting the in-mem storage as a backend for cassandra..
Thu, May 28, 4:12 PM
vlorentz added a comment to D3191: Add check-copyright-date pre-commit hook..

What will be the output of that pre-commit check (when it fails)?

Thu, May 28, 1:39 PM
vlorentz added a comment to D3191: Add check-copyright-date pre-commit hook..
In D3191#77776, @olasd wrote:

Good idea, but unfortunately I think this will complain when running pre-commit -a, if a file hasn't seen an update recently. This would need to check whether the last year matches the last commit in git log for that file.

Thu, May 28, 1:38 PM
vlorentz updated the diff for D3192: Disambiguate uploaded files with a prefix instead of a suffix..

improve comments

Thu, May 28, 1:32 PM
vlorentz created D3192: Disambiguate uploaded files with a prefix instead of a suffix..
Thu, May 28, 1:31 PM
vlorentz added a revision to T2393: Two deposits of files the same name and a "compound" extension fails the checker: D3192: Disambiguate uploaded files with a prefix instead of a suffix..
Thu, May 28, 1:31 PM · SWORD deposit
vlorentz claimed T2393: Two deposits of files the same name and a "compound" extension fails the checker.
Thu, May 28, 12:31 PM · SWORD deposit
vlorentz added a comment to T2393: Two deposits of files the same name and a "compound" extension fails the checker.

It should be easy to fix

Thu, May 28, 12:30 PM · SWORD deposit
vlorentz updated the diff for D3191: Add check-copyright-date pre-commit hook..

fix typo

Thu, May 28, 12:10 PM
vlorentz created D3191: Add check-copyright-date pre-commit hook..
Thu, May 28, 12:10 PM
vlorentz raised the priority of T2393: Two deposits of files the same name and a "compound" extension fails the checker from Normal to High.

Upping priority, as we are hitting this issue in production

Thu, May 28, 11:55 AM · SWORD deposit
vlorentz updated the task description for T2393: Two deposits of files the same name and a "compound" extension fails the checker.
Thu, May 28, 11:54 AM · SWORD deposit
vlorentz committed rDSTOfe877ce3a04c: Make content.blake2s256 not null. (authored by vlorentz).
Make content.blake2s256 not null.
Thu, May 28, 11:52 AM
vlorentz closed D3190: Make content.blake2s256 not null..
Thu, May 28, 11:52 AM
vlorentz committed rDSTOb0bff97191ba: Remove unused SQL functions. (authored by vlorentz).
Remove unused SQL functions.
Thu, May 28, 11:52 AM
vlorentz closed D3187: Remove unused SQL functions..
Thu, May 28, 11:52 AM
vlorentz requested changes to D3188: Add a default target for symbolic reference branches.

LGTM, but I think it's missing some tests:

Thu, May 28, 11:27 AM
vlorentz created D3190: Make content.blake2s256 not null..
Thu, May 28, 11:14 AM
vlorentz updated the diff for D3187: Remove unused SQL functions..

fix migration, it was dropping a function that is still in use.

Thu, May 28, 10:50 AM
vlorentz updated the summary of D3187: Remove unused SQL functions..
Thu, May 28, 10:47 AM
vlorentz updated the summary of D3187: Remove unused SQL functions..
Thu, May 28, 10:46 AM
vlorentz created D3187: Remove unused SQL functions..
Thu, May 28, 10:42 AM
vlorentz committed rDSTO1aff3c6d155b: Add a pre-commit hook to check there are version bumps in sql/upgrades/*.sql (authored by vlorentz).
Add a pre-commit hook to check there are version bumps in sql/upgrades/*.sql
Thu, May 28, 10:40 AM
vlorentz closed D3186: Add a pre-commit hook to check there are version bumps in sql/upgrades/*.sql.
Thu, May 28, 10:40 AM
vlorentz created D3186: Add a pre-commit hook to check there are version bumps in sql/upgrades/*.sql.
Thu, May 28, 10:22 AM
vlorentz committed rDSTO3613438f36bd: Add missing dbversion bump in 150.sql. (authored by vlorentz).
Add missing dbversion bump in 150.sql.
Thu, May 28, 10:01 AM
vlorentz closed D3185: Add missing dbversion bump in 150.sql..
Thu, May 28, 10:01 AM
vlorentz created D3185: Add missing dbversion bump in 150.sql..
Thu, May 28, 9:54 AM
vlorentz accepted D3180: pg: Write origin visit updates & status, read from origin_visit_status.
Thu, May 28, 9:31 AM · Storage manager

Wed, May 27

vlorentz triaged T2424: recaptcha javascript is loaded on every page as Normal priority.
Wed, May 27, 12:24 AM · Website

Tue, May 26

vlorentz accepted D3176: deposit-admin: Filtering out deposits matching an excluding pattern.
Tue, May 26, 2:58 PM · Web app
vlorentz committed rDSTO213f1b1239a8: Add artifact metadata to the extrinsic metadata storage specification. (authored by vlorentz).
Add artifact metadata to the extrinsic metadata storage specification.
Tue, May 26, 1:02 PM
vlorentz closed D3154: Add artifact metadata to the extrinsic metadata storage specification..
Tue, May 26, 1:02 PM
vlorentz updated the diff for D3154: Add artifact metadata to the extrinsic metadata storage specification..

rebase

Tue, May 26, 1:02 PM
vlorentz updated the diff for D3154: Add artifact metadata to the extrinsic metadata storage specification..
  • fix typo
  • rebase
Tue, May 26, 1:01 PM
vlorentz accepted D2949: nixguix: catch and log artifact resolution failures.
Tue, May 26, 1:00 PM · Package Loader

Mon, May 25

vlorentz updated the diff for D3154: Add artifact metadata to the extrinsic metadata storage specification..

fix mismatch

Mon, May 25, 4:57 PM
vlorentz requested changes to D3177: Make all model objects hashable.

We wouldn't need that if pytest had an equivalent for assertCountEqual :/

Mon, May 25, 3:05 PM
vlorentz added a comment to D3177: Make all model objects hashable.

What is the motivation for this?

Mon, May 25, 2:28 PM

Wed, May 20

vlorentz added a comment to D3171: Add support for model object anonymization.

Maybe, but that would require we keep the info "this is an anonymized object" somewhere, which is not the case for now. This idea can be dealt later, maybe?

Wed, May 20, 2:45 PM
vlorentz updated the diff for D3154: Add artifact metadata to the extrinsic metadata storage specification..

fix typo + use SWHID as first argument

Wed, May 20, 1:01 PM
vlorentz added a comment to D3171: Add support for model object anonymization.

Shouldn't we make anonymized objects error when their compute_hash() method is called?

Wed, May 20, 12:47 PM
vlorentz accepted D3170: storage: Add not null constraints to metadata_authority/origin_metadata tables.
Wed, May 20, 12:14 PM
vlorentz requested changes to D3170: storage: Add not null constraints to metadata_authority/origin_metadata tables.
Wed, May 20, 11:48 AM

Tue, May 19

vlorentz accepted D3165: Deploy loader_nixguix service on workers.
Tue, May 19, 8:00 PM
vlorentz accepted D3167: webapp: admin-deposit-page: Drop swh_anchor_id* references from deposit entries.
Tue, May 19, 8:00 PM
vlorentz accepted D3166: swh-deposit: Drop swh_anchor_id* columns from model Deposit.
Tue, May 19, 7:59 PM
vlorentz committed rDSTO8c2ee709495b: origin_metadata_add: Reject non-bytes types for 'metadata'. (authored by vlorentz).
origin_metadata_add: Reject non-bytes types for 'metadata'.
Tue, May 19, 6:41 PM
vlorentz closed D3168: origin_metadata_add: Reject non-bytes types for 'metadata'..
Tue, May 19, 6:41 PM
vlorentz closed D3169: Pass bytes instead a dict to origin_metadata_add..
Tue, May 19, 6:41 PM
vlorentz committed rDLDBASE6e20b23a593e: Pass bytes instead a dict to origin_metadata_add. (authored by vlorentz).
Pass bytes instead a dict to origin_metadata_add.
Tue, May 19, 6:41 PM
vlorentz added a reviewer for D3169: Pass bytes instead a dict to origin_metadata_add.: ardumont.
Tue, May 19, 6:36 PM
vlorentz created D3169: Pass bytes instead a dict to origin_metadata_add..
Tue, May 19, 6:35 PM
vlorentz created D3168: origin_metadata_add: Reject non-bytes types for 'metadata'..
Tue, May 19, 6:33 PM
vlorentz committed rDLDBASE1e9b83cdf895: Use the new API for origin_metadata. (authored by vlorentz).
Use the new API for origin_metadata.
Tue, May 19, 2:43 PM
vlorentz closed D3091: Use the new API for origin_metadata..
Tue, May 19, 2:43 PM
vlorentz updated the diff for D3091: Use the new API for origin_metadata..

rebase

Tue, May 19, 2:31 PM
vlorentz added inline comments to D3156: browse: Add snapshot related query parameters in numerous object views.
Tue, May 19, 2:28 PM
vlorentz committed rDSTOe645e6370951: Implement extrinsic origin metadata specification. (authored by vlorentz).
Implement extrinsic origin metadata specification.
Tue, May 19, 12:51 PM
vlorentz closed D2988: Implement extrinsic origin metadata specification..
Tue, May 19, 12:51 PM

Mon, May 18

vlorentz updated the diff for D2988: Implement extrinsic origin metadata specification..

fixup of the rebase: alter index metadata_provider_type_url to change its name, instead of creating metadata_authority_type_url which is a duplicate.

Mon, May 18, 6:26 PM
vlorentz updated the diff for D3154: Add artifact metadata to the extrinsic metadata storage specification..

Make 'visit' depend on 'origin' being provided.

Mon, May 18, 6:13 PM
vlorentz updated the diff for D2988: Implement extrinsic origin metadata specification..

apply @ardumont's changes to the migration.

Mon, May 18, 4:40 PM
vlorentz added a comment to D3154: Add artifact metadata to the extrinsic metadata storage specification..

I applied most of your comments, except:

Mon, May 18, 3:53 PM
vlorentz updated the diff for D3154: Add artifact metadata to the extrinsic metadata storage specification..

apply most comments

Mon, May 18, 3:52 PM
vlorentz updated the diff for D3154: Add artifact metadata to the extrinsic metadata storage specification..

undo push on the wrong diff (for real this time)

Mon, May 18, 3:40 PM
vlorentz updated the diff for D3154: Add artifact metadata to the extrinsic metadata storage specification..

undo push on the wrong diff

Mon, May 18, 3:39 PM
vlorentz updated the diff for D2988: Implement extrinsic origin metadata specification..

big rebase

Mon, May 18, 3:38 PM
vlorentz updated the diff for D3154: Add artifact metadata to the extrinsic metadata storage specification..

big rebase

Mon, May 18, 3:12 PM
vlorentz accepted D3159: storage: metadata_provider: Ensure idempotency when creating provider.
Mon, May 18, 1:57 PM

Sat, May 16

vlorentz added inline comments to D3154: Add artifact metadata to the extrinsic metadata storage specification..
Sat, May 16, 4:16 PM
vlorentz added a comment to D3154: Add artifact metadata to the extrinsic metadata storage specification..

for the context, we need to use the SHWIDs themselves, not the sha1_git that is bound to version 1 of SWHIDS

Sat, May 16, 4:15 PM

Fri, May 15

vlorentz accepted D3142: deposit: Adapt loader to use the latest deposit update api.
Fri, May 15, 12:32 PM
vlorentz added a comment to D3154: Add artifact metadata to the extrinsic metadata storage specification..

I don't think it makes sense to make this difference, because if a metadata only has origin in its context, it means it's context-free on all "axis" but the origin already. Having also no origin context is just one more axis it would be context-free on.

Fri, May 15, 10:04 AM

Thu, May 14

vlorentz added a revision to T2306: Generic storage for extrinsic, qualified metadata related to any node of the swh archive: D3154: Add artifact metadata to the extrinsic metadata storage specification..
Thu, May 14, 2:53 PM · Metadata workflow, Restricted Project
vlorentz added a task to D3154: Add artifact metadata to the extrinsic metadata storage specification.: T2306: Generic storage for extrinsic, qualified metadata related to any node of the swh archive.
Thu, May 14, 2:53 PM
vlorentz created D3154: Add artifact metadata to the extrinsic metadata storage specification..
Thu, May 14, 2:53 PM
vlorentz added a comment to D3152: Tag model entities with their "type".

Hmm actually, "artifact" isn't great either, because we'll want to use it to mean extrinsic metadata , which aren't artifacts. Ugh...

Thu, May 14, 11:43 AM
vlorentz added a comment to D3152: Tag model entities with their "type".

And I'm not a big fan of the name. What about object_type, as we already use this name in other places?

yes I also agree, but not a big fan of object_type either. However I have no better candidate to propose, I'm afraid. entity_type maybe ?

Thu, May 14, 11:38 AM

Wed, May 13

vlorentz added a comment to D3152: Tag model entities with their "type".

Great! You know how little I like isinstance ;)

Wed, May 13, 5:21 PM