Page MenuHomeSoftware Heritage
Feed All Stories

Oct 25 2022

swh-public-ci added a comment to D8756: azure: Add tests based on Azurite in addition to mocks.

Build has FAILED

Oct 25 2022, 2:46 PM
vlorentz updated the diff for D8756: azure: Add tests based on Azurite in addition to mocks.

Begone, markers

Oct 25 2022, 2:44 PM
swh-public-ci added a comment to D8756: azure: Add tests based on Azurite in addition to mocks.

Build is green

Oct 25 2022, 2:44 PM
olasd added inline comments to D8756: azure: Add tests based on Azurite in addition to mocks.
Oct 25 2022, 2:43 PM
ardumont accepted D8764: metadata: Make default tool configuration follow swh.indexer versions.

Something like this needs to happen on content indexer too (regarding their respective tools file and nomossa).

Oct 25 2022, 2:43 PM
vlorentz updated the diff for D8756: azure: Add tests based on Azurite in addition to mocks.

Use 'azurite-blob' instead of 'azurite'/'azurite.js'

Oct 25 2022, 2:41 PM
vlorentz added inline comments to D8756: azure: Add tests based on Azurite in addition to mocks.
Oct 25 2022, 2:41 PM
olasd added inline comments to D8756: azure: Add tests based on Azurite in addition to mocks.
Oct 25 2022, 2:40 PM
olasd accepted D8765: Install Azurite in base docker image.
Oct 25 2022, 2:33 PM
olasd added inline comments to D8756: azure: Add tests based on Azurite in addition to mocks.
Oct 25 2022, 2:32 PM
vlorentz updated the summary of D8765: Install Azurite in base docker image.
Oct 25 2022, 2:31 PM
vlorentz updated the summary of D8765: Install Azurite in base docker image.
Oct 25 2022, 2:31 PM
vlorentz requested review of D8765: Install Azurite in base docker image.
Oct 25 2022, 2:27 PM
swh-public-ci added a comment to D8756: azure: Add tests based on Azurite in addition to mocks.

Build is green

Oct 25 2022, 2:16 PM
vlorentz added a comment to D8756: azure: Add tests based on Azurite in addition to mocks.
In D8756#227875, @olasd wrote:

Then we should just npm install azurite container-wide on the CI system so these tests actually run on jenkins. (Alternatively, we can add a package.json / yarn.lock locally to this package, which should make jenkins automagically install it)

Oct 25 2022, 2:14 PM
vlorentz updated the test plan for D8756: azure: Add tests based on Azurite in addition to mocks.
Oct 25 2022, 2:13 PM
vlorentz updated the diff for D8756: azure: Add tests based on Azurite in addition to mocks.

use AZURITE_PATH as a dir name, and switch to skipif

Oct 25 2022, 2:13 PM
vlorentz requested review of D8764: metadata: Make default tool configuration follow swh.indexer versions.
Oct 25 2022, 2:07 PM
vlorentz added a revision to T4654: swh-indexer produces dates not supported by swh-search/ElasticSearch: D8764: metadata: Make default tool configuration follow swh.indexer versions.
Oct 25 2022, 2:03 PM · Archive search, Indexer
olasd accepted D8756: azure: Add tests based on Azurite in addition to mocks.

This seems like a great idea.

Oct 25 2022, 1:45 PM
ardumont closed D8763: nixguix: Allow lister to ignore specific extensions.
Oct 25 2022, 1:40 PM
ardumont committed rDLSca4ab7f277dc: nixguix: Allow lister to ignore specific extensions (authored by ardumont).
nixguix: Allow lister to ignore specific extensions
Oct 25 2022, 1:40 PM
ardumont closed D8761: nixguix/test: Add all supported tarball extensions to test manifest.
Oct 25 2022, 1:40 PM
ardumont committed rDLSd96a39d5b056: nixguix/test: Add all supported tarball extensions to test manifest (authored by ardumont).
nixguix/test: Add all supported tarball extensions to test manifest
Oct 25 2022, 1:40 PM
olasd added a comment to D8759: model: Add payload to ExtID class.

Anyway, I'm guessing it will fix itself when you send any update to the diff so let's ignore that issue for now.

This looks fine to me, though we might want to use something other than SHA1 for this.

We are currently in the process of moving the objstorage to multi-hashes, so eventually we will want to use multi-hashes here too. I wonder if we should do that before landing this diff (and D8760). It would be less effort overall, but will delay your work a bit.

eg. here it could be done by making payload a dictionary (with algos as keys and digests as values), and serialize the manifest as:

[payload_type $StrWithoutSpaces]
[payload blake2s256 $Hexdigest]
[payload sha1 $Hexdigest]
[payload sha1_git $Hexdigest]
[payload sha256 $Hexdigest]

@samplet @olasd What do you think?

Oct 25 2022, 1:22 PM
olasd added a comment to D8759: model: Add payload to ExtID class.

Hi, thanks for the diffs

It looks like using the --draft flag when sending this with arc has confused the CI system. The build failed because it couldn't access the updated code:

fatal: Couldn't find remote ref refs/tags/phabricator/diff/31582

It was because you hadn't signed the CLA when submitting the diff

Oct 25 2022, 1:14 PM
swh-public-ci added a comment to D8763: nixguix: Allow lister to ignore specific extensions.

Build is green

Oct 25 2022, 12:14 PM
ardumont updated the diff for D8763: nixguix: Allow lister to ignore specific extensions.

Adapt according to last review (merge variable comment, drop spurious .copy)

Oct 25 2022, 12:09 PM
ardumont added inline comments to D8763: nixguix: Allow lister to ignore specific extensions.
Oct 25 2022, 12:08 PM
anlambert added inline comments to D8763: nixguix: Allow lister to ignore specific extensions.
Oct 25 2022, 12:05 PM
swh-public-ci added a comment to D8763: nixguix: Allow lister to ignore specific extensions.

Build is green

Oct 25 2022, 11:59 AM
ardumont closed D8758: swh.core.tarball: Wire support to existing tarball.
Oct 25 2022, 11:57 AM
ardumont committed rDCORE306a74d01a99: swh.core.tarball: Add support for war files as well (authored by ardumont).
swh.core.tarball: Add support for war files as well
Oct 25 2022, 11:57 AM
ardumont updated the diff for D8763: nixguix: Allow lister to ignore specific extensions.
  • Add missing test
  • Fix another test
Oct 25 2022, 11:54 AM
Harbormaster failed remote builds in B32551: Diff 31597 for D8077: Add a static query cost calculator to reject malicious quries!
Oct 25 2022, 11:49 AM
swh-public-ci added a comment to D8077: Add a static query cost calculator to reject malicious quries.

Build has FAILED

Oct 25 2022, 11:49 AM
jayeshv updated the diff for D8077: Add a static query cost calculator to reject malicious quries.

Schema changes

Oct 25 2022, 11:47 AM
Harbormaster failed remote builds in B32550: Diff 31596 for D8763: nixguix: Allow lister to ignore specific extensions!
Oct 25 2022, 11:47 AM
swh-public-ci added a comment to D8763: nixguix: Allow lister to ignore specific extensions.

Build has FAILED

Oct 25 2022, 11:47 AM
Harbormaster failed remote builds in B32549: Diff 31595 for D8763: nixguix: Allow lister to ignore specific extensions!
Oct 25 2022, 11:42 AM
swh-public-ci added a comment to D8763: nixguix: Allow lister to ignore specific extensions.

Build was aborted

Oct 25 2022, 11:42 AM
ardumont updated the diff for D8763: nixguix: Allow lister to ignore specific extensions.

Rename variable parameter with a better name

Oct 25 2022, 11:41 AM
ardumont added inline comments to D8763: nixguix: Allow lister to ignore specific extensions.
Oct 25 2022, 11:41 AM
ardumont updated the diff for D8763: nixguix: Allow lister to ignore specific extensions.
  • Rebase
  • Refactor common code on extension detection
  • Adapt docstring and commit message
  • Default extensions to ignore
  • Adapt constructor to extend extension to ignore if any is passed around
Oct 25 2022, 11:38 AM
ardumont retitled D8763: nixguix: Allow lister to ignore specific extensions from nixguix: Allow lister configuration to ignore specific extensions to nixguix: Allow lister to ignore specific extensions.
Oct 25 2022, 11:37 AM
swh-public-ci added a comment to D8761: nixguix/test: Add all supported tarball extensions to test manifest.

Build is green

Oct 25 2022, 11:34 AM
ardumont updated the diff for D8761: nixguix/test: Add all supported tarball extensions to test manifest.

Rebase

Oct 25 2022, 11:29 AM
ardumont added inline comments to D8763: nixguix: Allow lister to ignore specific extensions.
Oct 25 2022, 11:11 AM
lunar updated the summary of D8671: Add a job running swh-mirror tests.
Oct 25 2022, 11:06 AM
franckbret added a comment to D8762: Puppet: Switch artifacts from dict to list.

Looks good to me, I guess you need to update the loader too.

Oct 25 2022, 11:05 AM
anlambert accepted D8758: swh.core.tarball: Wire support to existing tarball.
Oct 25 2022, 11:04 AM
anlambert accepted D8761: nixguix/test: Add all supported tarball extensions to test manifest.
Oct 25 2022, 11:02 AM
anlambert accepted D8762: Puppet: Switch artifacts from dict to list.

Looks good to me, I guess you need to update the loader too.

Oct 25 2022, 11:01 AM
anlambert closed D8747: conda: Yield listed origins after all artifacts in a page are processed.
Oct 25 2022, 10:58 AM
anlambert committed rDLS4f6b3f3f09b8: conda: Yield listed origins after all artifacts in a page are processed (authored by anlambert).
conda: Yield listed origins after all artifacts in a page are processed
Oct 25 2022, 10:58 AM
anlambert accepted D8763: nixguix: Allow lister to ignore specific extensions.

LGTM, one thought posted as inline comment.

Oct 25 2022, 10:58 AM
swh-public-ci added a comment to D8747: conda: Yield listed origins after all artifacts in a page are processed.

Build is green

Oct 25 2022, 10:57 AM
anlambert updated the diff for D8747: conda: Yield listed origins after all artifacts in a page are processed.

Rebase

Oct 25 2022, 10:52 AM
ardumont requested review of D8763: nixguix: Allow lister to ignore specific extensions.
Oct 25 2022, 10:46 AM
swh-public-ci added a comment to D8761: nixguix/test: Add all supported tarball extensions to test manifest.

Build is green

Oct 25 2022, 10:46 AM
franckbret updated the summary of D8762: Puppet: Switch artifacts from dict to list.
Oct 25 2022, 10:42 AM
ardumont added a revision to T3781: Replace the Nixguix loader with a lister: D8763: nixguix: Allow lister to ignore specific extensions.
Oct 25 2022, 10:41 AM · Data Model, Nixguix loader
ardumont updated the diff for D8761: nixguix/test: Add all supported tarball extensions to test manifest.

Rework commit message

Oct 25 2022, 10:40 AM
ardumont retitled D8761: nixguix/test: Add all supported tarball extensions to test manifest from nixguix/test: Add all supported extensions in test manifest to nixguix/test: Add all supported tarball extensions to test manifest.
Oct 25 2022, 10:40 AM
vlorentz created P1509 API useful output.
Oct 25 2022, 10:26 AM
vlorentz created P1508 CLI useful output.
Oct 25 2022, 10:25 AM
vlorentz created P1507 get_licenses("blobs/ff/ff/ffff1f95821e88fa5e1a2c6e21918962aab3a9d1").
Oct 25 2022, 10:22 AM
vlorentz created P1506 scancode --license --timeout 120 --license-score 0 blobs/ff/ff/ffff1f95821e88fa5e1a2c6e21918962aab3a9d1 --json-pp /dev/stdout.
Oct 25 2022, 10:21 AM
ardumont closed D8757: Add support for more tarball/zip extensions.
Oct 25 2022, 10:16 AM
ardumont committed rDLS31eb5f637f69: Add support for more tarball recognition based on extensions (authored by ardumont).
Add support for more tarball recognition based on extensions
Oct 25 2022, 10:16 AM
franckbret requested review of D8762: Puppet: Switch artifacts from dict to list.
Oct 25 2022, 10:16 AM
ardumont requested review of D8761: nixguix/test: Add all supported tarball extensions to test manifest.
Oct 25 2022, 10:14 AM
ardumont requested review of D8758: swh.core.tarball: Wire support to existing tarball.
Oct 25 2022, 10:12 AM
swh-public-ci added a comment to D8757: Add support for more tarball/zip extensions.

Build is green

Oct 25 2022, 10:12 AM
ardumont added a revision to T3781: Replace the Nixguix loader with a lister: D8761: nixguix/test: Add all supported tarball extensions to test manifest.
Oct 25 2022, 10:09 AM · Data Model, Nixguix loader
ardumont updated the diff for D8757: Add support for more tarball/zip extensions.

Add missing new archive format in test manifest

Oct 25 2022, 10:06 AM
swh-public-ci added a comment to D8757: Add support for more tarball/zip extensions.

Build is green

Oct 25 2022, 9:55 AM
ardumont updated the diff for D8757: Add support for more tarball/zip extensions.

Add dataset around newly supported tarball formats

Oct 25 2022, 9:49 AM
swh-public-ci added a comment to D8425: Return more information with revision and release dates.

Build is green

Oct 25 2022, 9:29 AM
jayeshv updated the diff for D8425: Return more information with revision and release dates.

test fixes

Oct 25 2022, 9:27 AM
vlorentz updated subscribers of D8759: model: Add payload to ExtID class.

Anyway, I'm guessing it will fix itself when you send any update to the diff so let's ignore that issue for now.

Oct 25 2022, 9:07 AM
vlorentz added a comment to D8759: model: Add payload to ExtID class.
Oct 25 2022, 8:53 AM
vlorentz added a comment to D8759: model: Add payload to ExtID class.

Hi, thanks for the diffs

Oct 25 2022, 8:52 AM
swh-public-ci added a comment to D8759: model: Add payload to ExtID class.

Build has FAILED

Oct 25 2022, 8:50 AM
vlorentz updated the summary of D8760: Add payloads to ExtIDs.
Oct 25 2022, 8:41 AM
franckbret added inline comments to D8379: Hackage: Loads Hackage Listed origins.
Oct 25 2022, 8:25 AM
samplet requested review of D8760: Add payloads to ExtIDs.
Oct 25 2022, 5:34 AM
samplet added a comment to D8759: model: Add payload to ExtID class.

It looks like using the --draft flag when sending this with arc has confused the CI system. The build failed because it couldn't access the updated code:

Oct 25 2022, 5:04 AM
samplet published D8759: model: Add payload to ExtID class for review.
Oct 25 2022, 4:58 AM

Oct 24 2022

zack edited P1505 (An Untitled Masterwork).
Oct 24 2022, 9:49 PM
vlorentz requested review of D8756: azure: Add tests based on Azurite in addition to mocks.
Oct 24 2022, 6:27 PM
vlorentz accepted D8757: Add support for more tarball/zip extensions.
Oct 24 2022, 6:26 PM
vlorentz accepted D8747: conda: Yield listed origins after all artifacts in a page are processed.

perfect, thanks

Oct 24 2022, 6:26 PM
vlorentz closed D8755: Reset Sentry tags when leaving an object's context.
Oct 24 2022, 6:25 PM
vlorentz committed rDCIDXdc48e9ebaae6: Reset Sentry tags when leaving an object's context (authored by vlorentz).
Reset Sentry tags when leaving an object's context
Oct 24 2022, 6:25 PM
ardumont added inline comments to D8757: Add support for more tarball/zip extensions.
Oct 24 2022, 5:19 PM
anlambert accepted D8755: Reset Sentry tags when leaving an object's context.
Oct 24 2022, 4:59 PM
zack edited P1505 (An Untitled Masterwork).
Oct 24 2022, 4:48 PM
anlambert added inline comments to D8757: Add support for more tarball/zip extensions.
Oct 24 2022, 4:47 PM
swh-public-ci added a comment to D8757: Add support for more tarball/zip extensions.

Build is green

Oct 24 2022, 4:47 PM