Page MenuHomeSoftware Heritage
Feed Advanced Search

Jul 10 2020

ardumont added a revision to T2484: Move sharable fixtures out of conftest into a dedicated pytest plugin: D3492: setup: Do no expose the pytest-plugin, let modules require it.
Jul 10 2020, 12:11 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

Jul 9 2020

ardumont added a revision to T2483: tests: Make check-snapshot utility test function recursively check targetted object exists: D3478: loader.core: Make check_snapshot ensure snapshot object references exist.
Jul 9 2020, 4:17 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a revision to T2484: Move sharable fixtures out of conftest into a dedicated pytest plugin: D3475: setup: Do no expose the pytest-plugin any longer.
Jul 9 2020, 10:22 AM · SVN Loader, Mercurial loader, Git loader, Core Loader

Jul 8 2020

ardumont added a revision to T2483: tests: Make check-snapshot utility test function recursively check targetted object exists: D3473: loader.core: check_snapshot: Allow to check using both Snapshot and dict objects.
Jul 8 2020, 6:40 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a revision to T2481: Migrate dvcs loader tests code to pytest: D3468: test_init: Use swh_storage fixture.
Jul 8 2020, 4:52 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont closed T2484: Move sharable fixtures out of conftest into a dedicated pytest plugin, a subtask of T2481: Migrate dvcs loader tests code to pytest, as Resolved.
Jul 8 2020, 4:51 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont closed T2484: Move sharable fixtures out of conftest into a dedicated pytest plugin as Resolved.
Jul 8 2020, 4:51 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

Jul 7 2020

ardumont added a revision to T2484: Move sharable fixtures out of conftest into a dedicated pytest plugin: D3453: tests: Use dedicated storage and scheduler fixtures.
Jul 7 2020, 6:05 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a revision to T2484: Move sharable fixtures out of conftest into a dedicated pytest plugin: D3452: Move sharable fixtures out of conftest into a dedicated pytest plugin.
Jul 7 2020, 5:55 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a revision to T2481: Migrate dvcs loader tests code to pytest: D3449: tests: Rename resources directory to data.
Jul 7 2020, 4:44 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont triaged T2488: Drop loader.core BaseLoaderTest and BaseLoaderStorageTest as Normal priority.
Jul 7 2020, 4:21 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont closed T2486: Add missing coverage to the check-snapshot utility check function (and drop dead code), a subtask of T2481: Migrate dvcs loader tests code to pytest, as Resolved.
Jul 7 2020, 2:04 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont closed T2486: Add missing coverage to the check-snapshot utility check function (and drop dead code) as Resolved.
Jul 7 2020, 2:04 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont renamed T2486: Add missing coverage to the check-snapshot utility check function (and drop dead code) from Add missing coverage to the check-snapshot utility check function (or drop dead code) to Add missing coverage to the check-snapshot utility check function (and drop dead code).
Jul 7 2020, 2:04 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a revision to T2486: Add missing coverage to the check-snapshot utility check function (and drop dead code): D3444: loaders.tests: Simplify and add coverage to check-snapshot.
Jul 7 2020, 1:16 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont renamed T2486: Add missing coverage to the check-snapshot utility check function (and drop dead code) from Add missing coverage to the check-snapshot utility check function to Add missing coverage to the check-snapshot utility check function (or drop dead code).
Jul 7 2020, 1:09 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a comment to T2486: Add missing coverage to the check-snapshot utility check function (and drop dead code).

In the end, it's more dead code since it's only code we pass into when the storage used is an in-memory instance.
This is no longer the case, tests are now using pg-storage instance.

Jul 7 2020, 1:09 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont removed a revision from T2481: Migrate dvcs loader tests code to pytest: D3440: loader.hg: Initialize and refactor deprecated tests to use pytest fixture.
Jul 7 2020, 1:08 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont closed T2482: Migrate git loader tests code to pytest, a subtask of T2481: Migrate dvcs loader tests code to pytest, as Resolved.
Jul 7 2020, 1:08 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont triaged T2486: Add missing coverage to the check-snapshot utility check function (and drop dead code) as Normal priority.
Jul 7 2020, 9:46 AM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a revision to T2481: Migrate dvcs loader tests code to pytest: D3440: loader.hg: Initialize and refactor deprecated tests to use pytest fixture.
Jul 7 2020, 9:32 AM · SVN Loader, Mercurial loader, Git loader, Core Loader

Jul 6 2020

ardumont triaged T2484: Move sharable fixtures out of conftest into a dedicated pytest plugin as Normal priority.
Jul 6 2020, 3:57 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont triaged T2483: tests: Make check-snapshot utility test function recursively check targetted object exists as Normal priority.
Jul 6 2020, 3:50 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a revision to T2481: Migrate dvcs loader tests code to pytest: D3433: Move remaining common test utility functions to top-level arborescence.
Jul 6 2020, 3:34 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a revision to T2481: Migrate dvcs loader tests code to pytest: D3432: Move common test utility function to the top-level arborescence.
Jul 6 2020, 3:32 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added a revision to T2481: Migrate dvcs loader tests code to pytest: D3431: Define common test helper function.
Jul 6 2020, 3:31 PM · SVN Loader, Mercurial loader, Git loader, Core Loader
ardumont added projects to T2481: Migrate dvcs loader tests code to pytest: Core Loader, Git loader, Mercurial loader, SVN Loader.
Jul 6 2020, 1:57 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

Jul 1 2020

ardumont added a revision to T2471: NPM package angular-ts-manage fails to be properly loaded: D3386: Test to reproduce issue.
Jul 1 2020, 11:25 AM · Core Loader
ardumont updated the task description for T2471: NPM package angular-ts-manage fails to be properly loaded.
Jul 1 2020, 10:12 AM · Core Loader
douardda added a comment to T2471: NPM package angular-ts-manage fails to be properly loaded.

This origin (and others) also fails to be mirrored with errors like:

Jul 1 2020, 10:10 AM · Core Loader
douardda triaged T2471: NPM package angular-ts-manage fails to be properly loaded as High priority.
Jul 1 2020, 10:05 AM · Core Loader

Jun 25 2020

olasd added a comment to T2466: Re design the lazy loading feature for Content objects.

Lazy loading for content objects (stored as files on the filesystem) gives us a choice where to put the cursor between the two ends of the eternal tradeoff between:

Jun 25 2020, 4:15 PM · Core Loader
olasd added a project to T2466: Re design the lazy loading feature for Content objects: Core Loader.
Jun 25 2020, 4:07 PM · Core Loader

May 29 2020

ardumont closed T2428: Unstuck failing loaders in staging as Resolved.
May 29 2020, 5:38 PM · Staging environment, Core Loader
ardumont added a comment to T2428: Unstuck failing loaders in staging.

Deployed.

May 29 2020, 5:38 PM · Staging environment, Core Loader
ardumont closed D3196: package.loader: Check for snapshot id prior to reifying model object.
May 29 2020, 3:59 PM · Core Loader
swh-public-ci added a comment to D3196: package.loader: Check for snapshot id prior to reifying model object.

Build is green

May 29 2020, 3:58 PM · Core Loader
ardumont updated the diff for D3196: package.loader: Check for snapshot id prior to reifying model object.

Rebase on latest master

May 29 2020, 3:57 PM · Core Loader
swh-public-ci added a comment to D3196: package.loader: Check for snapshot id prior to reifying model object.

Build is green

May 29 2020, 1:24 PM · Core Loader
ardumont retitled D3196: package.loader: Check for snapshot id prior to reifying model object from package.loader: Check for snapshot id prior to reify the model object to package.loader: Check for snapshot id prior to reifying model object.
May 29 2020, 1:23 PM · Core Loader
swh-public-ci added a comment to D3196: package.loader: Check for snapshot id prior to reifying model object.

Build is green

May 29 2020, 1:23 PM · Core Loader
ardumont updated the diff for D3196: package.loader: Check for snapshot id prior to reifying model object.

Rework commit message

May 29 2020, 1:23 PM · Core Loader
ardumont updated the summary of D3196: package.loader: Check for snapshot id prior to reifying model object.
May 29 2020, 1:22 PM · Core Loader
ardumont updated the diff for D3196: package.loader: Check for snapshot id prior to reifying model object.
  • Adapt according to review
  • Align git commit with diff description
May 29 2020, 1:21 PM · Core Loader
olasd accepted D3196: package.loader: Check for snapshot id prior to reifying model object.

Please rename the snap variable to something meaningful :)

May 29 2020, 12:52 PM · Core Loader
ardumont updated the summary of D3196: package.loader: Check for snapshot id prior to reifying model object.
May 29 2020, 12:44 PM · Core Loader
ardumont changed the status of T2428: Unstuck failing loaders in staging from Open to Work in Progress.
May 29 2020, 12:40 PM · Staging environment, Core Loader
ardumont updated the summary of D3196: package.loader: Check for snapshot id prior to reifying model object.
May 29 2020, 12:31 PM · Core Loader
ardumont added projects to T2428: Unstuck failing loaders in staging: Core Loader, Staging environment.
May 29 2020, 12:28 PM · Staging environment, Core Loader
ardumont removed a project from D3196: package.loader: Check for snapshot id prior to reifying model object: Staging environment.
May 29 2020, 12:28 PM · Core Loader
ardumont added projects to D3196: package.loader: Check for snapshot id prior to reifying model object: Core Loader, Staging environment.
May 29 2020, 12:28 PM · Core Loader

Apr 22 2020

ardumont changed the status of T2371: nixguix: fails to use previous visit snapshot from Open to Work in Progress.
Apr 22 2020, 12:17 PM · Package Loader
ardumont added a comment to T2371: nixguix: fails to use previous visit snapshot.

tl; dr: That issue must be invalid. I'm ensuring it is. I've triggered the
necessary checks and waiting for the conclusion.

Apr 22 2020, 12:16 PM · Package Loader
ardumont added a comment to T2371: nixguix: fails to use previous visit snapshot.

Note: the snapshot is a brand new one, no shared state with a prior one.

Apr 22 2020, 11:58 AM · Package Loader
ardumont added a comment to T2371: nixguix: fails to use previous visit snapshot.

Something is off though.

Apr 22 2020, 11:28 AM · Package Loader
ardumont updated the task description for T2371: nixguix: fails to use previous visit snapshot.
Apr 22 2020, 11:27 AM · Package Loader
ardumont removed a project from T1991: Implement a Guix/Nix loader: Lister.
Apr 22 2020, 10:50 AM · Package Loader
ardumont triaged T2371: nixguix: fails to use previous visit snapshot as Normal priority.
Apr 22 2020, 10:49 AM · Package Loader

Apr 20 2020

ardumont added a subtask for T1991: Implement a Guix/Nix loader: T2357: nixguix: fails to finish as downloading artifacts step hangs.
Apr 20 2020, 9:44 AM · Package Loader
ardumont added a subtask for T1991: Implement a Guix/Nix loader: T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error.
Apr 20 2020, 9:43 AM · Package Loader

Apr 14 2020

ardumont updated the task description for T2357: nixguix: fails to finish as downloading artifacts step hangs.
Apr 14 2020, 2:29 PM · Package Loader
ardumont added a revision to T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error: D3012: retry: Make content_add endpoints maximize content writes to storage.
Apr 14 2020, 10:28 AM · Package Loader, Storage manager
ardumont added a revision to T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error: D2966: storage.buffer: Add a new clear_buffers operation for the buffer proxy.
Apr 14 2020, 10:25 AM · Package Loader, Storage manager
ardumont added a revision to T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error: D2973: package.loader: Insert consistently data to storage, clear buffer proxy state in case of error when skipping artifact.
Apr 14 2020, 10:25 AM · Package Loader, Storage manager
ardumont added a revision to T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error: D3014: package.loader: Flush and clear regularly internal proxy storage states.
Apr 14 2020, 10:21 AM · Package Loader, Storage manager

Apr 11 2020

ardumont renamed T2357: nixguix: fails to finish as downloading artifacts step hangs from nixguix: fails to finish as downloading artifacts step to nixguix: fails to finish as downloading artifacts step hangs.
Apr 11 2020, 4:50 PM · Package Loader
ardumont renamed T2357: nixguix: fails to finish as downloading artifacts step hangs from nixguix: fails to finish as download hanging to nixguix: fails to finish as downloading artifacts step.
Apr 11 2020, 12:58 PM · Package Loader
ardumont updated the task description for T2357: nixguix: fails to finish as downloading artifacts step hangs.
Apr 11 2020, 12:00 PM · Package Loader
ardumont triaged T2357: nixguix: fails to finish as downloading artifacts step hangs as Normal priority.
Apr 11 2020, 11:58 AM · Package Loader
ardumont created T2357: nixguix: fails to finish as downloading artifacts step hangs.
Apr 11 2020, 11:58 AM · Package Loader
ardumont renamed T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error from nixguix: Fails to finish to nixguix: Fails to finish as it's stuck in a loop up to memory error.
Apr 11 2020, 11:51 AM · Package Loader, Storage manager

Apr 9 2020

ardumont added a comment to T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error.

In the mean time (pending reviews), a new load was triggered without the proxy storage and all went fine.

Apr 9 2020, 9:39 AM · Package Loader, Storage manager

Apr 7 2020

ardumont added a comment to T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error.

Right now, heading for 2. for now as the solution for 3. is still a pending question [2]

Apr 7 2020, 3:05 PM · Package Loader, Storage manager
ardumont updated the task description for T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error.
Apr 7 2020, 2:55 PM · Package Loader, Storage manager
ardumont added projects to T2352: nixguix: Fails to finish as it's stuck in a loop up to memory error: Core Loader, Storage manager.
Apr 7 2020, 11:56 AM · Package Loader, Storage manager

Apr 2 2020

ardumont updated the task description for T1991: Implement a Guix/Nix loader.
Apr 2 2020, 4:34 PM · Package Loader

Mar 25 2020

zack triaged T2334: Manage failing SSL requests as Normal priority.
Mar 25 2020, 11:03 AM · Core Loader

Mar 24 2020

lewo created T2334: Manage failing SSL requests.
Mar 24 2020, 3:52 PM · Core Loader

Mar 20 2020

ardumont created P620 staging: functional loader crash .
Mar 20 2020, 11:16 AM · Core Loader

Mar 19 2020

lewo updated the task description for T1991: Implement a Guix/Nix loader.
Mar 19 2020, 11:09 AM · Package Loader

Mar 18 2020

ardumont added a comment to T1991: Implement a Guix/Nix loader.

First runs triggered on staging, errors will be reported in sentry [1]

Mar 18 2020, 4:50 PM · Package Loader
ardumont added a comment to P618 functional loader through cli.

Related to T1991

Mar 18 2020, 4:49 PM · Core Loader
ardumont added a comment to P618 functional loader through cli.

well, still no message in sentry.
I'll stop that script and use the proper scheduler stanza on staging.

Mar 18 2020, 4:24 PM · Core Loader
ardumont added a comment to P618 functional loader through cli.

well the export instructions were wrong. Fixed!

Mar 18 2020, 2:49 PM · Core Loader
ardumont edited P618 functional loader through cli.
Mar 18 2020, 2:47 PM · Core Loader
ardumont added a comment to P618 functional loader through cli.

some error not reported in sentry:

DEBUG:swh.loader.package.utils:extrinsic_metadata                                                                                                                                                                                     [84/8086]
ERROR:swh.loader.package.loader:Fail to uncompress https://github.com/lihaoyi/Ammonite/releases/download/1.6.9/2.12-1.6.9
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/swh/core/tarball.py", line 72, in uncompress
    shutil.unpack_archive(tarpath, extract_dir=dest)
  File "/usr/lib/python3.7/shutil.py", line 995, in unpack_archive
    raise ReadError("Unknown archive format '{0}'".format(filename))
shutil.ReadError: Unknown archive format '/tmp/tmpmnd54wj6/2.12-1.6.9'
Mar 18 2020, 2:46 PM · Core Loader
ardumont added a comment to P618 functional loader through cli.

so sentry see the issues...

Mar 18 2020, 2:34 PM · Core Loader
ardumont created P618 functional loader through cli.
Mar 18 2020, 2:31 PM · Core Loader

Mar 10 2020

lewo updated the task description for T1991: Implement a Guix/Nix loader.
Mar 10 2020, 6:49 PM · Package Loader
lewo updated the task description for T1991: Implement a Guix/Nix loader.
Mar 10 2020, 6:42 PM · Package Loader

Mar 5 2020

ardumont assigned T1991: Implement a Guix/Nix loader to lewo.
Mar 5 2020, 7:30 PM · Package Loader
ardumont renamed T1991: Implement a Guix/Nix loader from Implement a Guix/GuixSD/Nix/NixOS loader to Implement a Guix/Nix loader.
Mar 5 2020, 7:30 PM · Package Loader
ardumont added a comment to T1991: Implement a Guix/Nix loader.

Implementation of a lister started at D2025.

Mar 5 2020, 7:30 PM · Package Loader
ardumont renamed T1991: Implement a Guix/Nix loader from Implement a Guix/GuixSD/Nix/NixOS lister to Implement a Guix/GuixSD/Nix/NixOS loader.
Mar 5 2020, 7:28 PM · Package Loader

Feb 6 2020

ardumont closed T1626: loader-core: Unify counter with the storage's results as Resolved.

It's been done and removed... so closed.

Feb 6 2020, 7:12 PM · Core Loader

Jan 22 2020

ardumont renamed T2241: lister/loader: Ingest archived artifacts from cran mirror from lister/loader: Ingest all known artifacts from cran mirror to lister/loader: Ingest archived artifacts from cran mirror.
Jan 22 2020, 10:46 AM · Core Loader, Lister, Origin-CRAN

Jan 21 2020

ardumont added a comment to T2241: lister/loader: Ingest archived artifacts from cran mirror.

This should take care of [1]

Jan 21 2020, 12:18 PM · Core Loader, Lister, Origin-CRAN
ardumont updated the task description for T2241: lister/loader: Ingest archived artifacts from cran mirror.
Jan 21 2020, 11:52 AM · Core Loader, Lister, Origin-CRAN
ardumont triaged T2241: lister/loader: Ingest archived artifacts from cran mirror as Normal priority.
Jan 21 2020, 11:52 AM · Core Loader, Lister, Origin-CRAN

Jan 6 2020

ardumont closed D2495: package.loader: Fail loading early when failing to create an origin/visit.
Jan 6 2020, 4:37 PM · Core Loader