Feed Advanced Search

Advanced Search
Use Results
Edit Query
Hide Query

Jan 8 2023

gitlab-migration closed T4201: Support empty directories as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:24 PM · Core Loader, BZR loader

gitlab-migration closed T3851: Investigate timeouts in the deposit loader in Docker as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:23 PM · Core Loader, SWORD deposit

gitlab-migration closed T3607: Document consistency guarantees of the loaders with respect to the storage as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:23 PM · Data Model, Storage manager, Package Loader, Core Loader, Documentation

gitlab-migration closed T2585: Properly handle deleted/no longer available origins as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:22 PM · Lister, Core Loader

gitlab-migration closed T2241: lister/loader: Ingest archived artifacts from cran mirror as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:22 PM · Core Loader, Lister, Origin-CRAN

gitlab-migration closed T1022: SWORD deposit requesting to save content existing on an external code hosting platform as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:21 PM · Core Loader, SWORD deposit

gitlab-migration changed the status of T3273: Use "fork" relationships to speed-up initial load of large repositories from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:02 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

gitlab-migration changed the status of T2486: Add missing coverage to the check-snapshot utility check function (and drop dead code), a subtask of T2481: Migrate dvcs loader tests code to pytest, from Resolved to Migrated.

Jan 8 2023, 10:00 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2488: Drop loader.core BaseLoaderTest and BaseLoaderStorageTest, a subtask of T2481: Migrate dvcs loader tests code to pytest, from Resolved to Migrated.

Jan 8 2023, 10:00 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2486: Add missing coverage to the check-snapshot utility check function (and drop dead code) from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:00 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2488: Drop loader.core BaseLoaderTest and BaseLoaderStorageTest from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:00 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2483: tests: Make check-snapshot utility test function recursively check targetted object exists, a subtask of T2481: Migrate dvcs loader tests code to pytest, from Resolved to Migrated.

Jan 8 2023, 10:00 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2484: Move sharable fixtures out of conftest into a dedicated pytest plugin from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:00 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2483: tests: Make check-snapshot utility test function recursively check targetted object exists from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:00 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2484: Move sharable fixtures out of conftest into a dedicated pytest plugin, a subtask of T2481: Migrate dvcs loader tests code to pytest, from Resolved to Migrated.

Jan 8 2023, 10:00 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2481: Migrate dvcs loader tests code to pytest from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:00 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T1740: fetch extrinsic origin metadata from GitHub, a subtask of T3273: Use "fork" relationships to speed-up initial load of large repositories, from Resolved to Migrated.

Jan 8 2023, 9:59 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

gitlab-migration changed the status of T1021: SWORD deposit of metadata about an existing SWH object from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:57 PM · Core Loader, SWORD deposit

gitlab-migration changed the status of T960: draft specs for deposit with incomplete tarball , a subtask of T1021: SWORD deposit of metadata about an existing SWH object, from Resolved to Migrated.

Jan 8 2023, 9:57 PM · Core Loader, SWORD deposit

gitlab-migration changed the status of T871: Migrate swh-storage api functions relative to indexers to swh-indexer from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:57 PM · SWORD deposit, Core Loader, Web app, Development environment, Storage manager, Indexer

gitlab-migration changed the status of T336: "save code now", a subtask of T1022: SWORD deposit requesting to save content existing on an external code hosting platform, from Resolved to Migrated.

Jan 8 2023, 9:56 PM · Core Loader, SWORD deposit

gitlab-migration closed T3976: opam loader: Adapt for opam > 2.1 as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 5:03 PM · Opam, Package Loader, Core Loader

gitlab-migration closed T3704: Merge `assert_last_visit_matches` and `check_snapshot` and make them diff recursively as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 5:03 PM · Core Loader

gitlab-migration closed T3030: Improve loaders to deal with new visit status events as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 5:02 PM · Core Loader

gitlab-migration closed T2466: Re design the lazy loading feature for Content objects as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 5:01 PM · Core Loader

gitlab-migration closed T2334: Manage failing SSL requests as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 5:01 PM · Core Loader

gitlab-migration changed the status of T4219: Investigate why GitHub fork detection did not bring a speed-up, a subtask of T3273: Use "fork" relationships to speed-up initial load of large repositories, from Resolved to Migrated.

Jan 8 2023, 4:36 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

gitlab-migration changed the status of T4188: Make swh-loader-core run metadata fetchers before loading an origin, a subtask of T4186: Allow loaders to fetch extrinsic metadata, from Resolved to Migrated.

Jan 8 2023, 4:36 PM · Core Loader, Metadata workflow

gitlab-migration changed the status of T4188: Make swh-loader-core run metadata fetchers before loading an origin from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:36 PM · Core Loader, Metadata workflow

gitlab-migration changed the status of T4187: Pass forge type to loaders from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:36 PM · Core Loader, Metadata workflow

gitlab-migration changed the status of T4187: Pass forge type to loaders, a subtask of T4186: Allow loaders to fetch extrinsic metadata, from Resolved to Migrated.

Jan 8 2023, 4:36 PM · Core Loader, Metadata workflow

gitlab-migration changed the status of T4186: Allow loaders to fetch extrinsic metadata from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:36 PM · Core Loader, Metadata workflow

gitlab-migration changed the status of T3347: Archive loader should have a snapshot append mode from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:34 PM · Save Code Now, Core Loader

gitlab-migration changed the status of T3142: Make loaders write to the ExtId storage from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3142: Make loaders write to the ExtId storage, a subtask of T3143: Migrate revision metadata to extid in the storage, from Resolved to Migrated.

Jan 8 2023, 4:33 PM · System administration, Storage manager, Core Loader

gitlab-migration changed the status of T3141: Make loaders pass ExtId objects between functions instead of "known_artifact", a subtask of T3140: Make loaders use ExtId to find what artifacts are already loaded, in addition to revision.metadata, from Resolved to Migrated.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3141: Make loaders pass ExtId objects between functions instead of "known_artifact" from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3141: Make loaders pass ExtId objects between functions instead of "known_artifact", a subtask of T3142: Make loaders write to the ExtId storage, from Resolved to Migrated.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3140: Make loaders use ExtId to find what artifacts are already loaded, in addition to revision.metadata from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3140: Make loaders use ExtId to find what artifacts are already loaded, in addition to revision.metadata, a subtask of T3139: Make loaders stop reading from the 'metadata' column of the 'revision' table, from Resolved to Migrated.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3139: Make loaders stop reading from the 'metadata' column of the 'revision' table, a subtask of T3138: Make loaders stop writing in the 'metadata' column of the 'revision' table, from Resolved to Migrated.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3139: Make loaders stop reading from the 'metadata' column of the 'revision' table from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3138: Make loaders stop writing in the 'metadata' column of the 'revision' table from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3138: Make loaders stop writing in the 'metadata' column of the 'revision' table, a subtask of T2471: NPM package angular-ts-manage fails to be properly loaded, from Resolved to Migrated.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3138: Make loaders stop writing in the 'metadata' column of the 'revision' table, a subtask of T3090: Make loaders not rely on the 'metadata' column of the 'revision' table, from Resolved to Migrated.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T3090: Make loaders not rely on the 'metadata' column of the 'revision' table from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:33 PM · Core Loader

gitlab-migration changed the status of T2849: Design and implement a mapping from "original VCS ids" to SWHIDs to help incremental loaders, a subtask of T3090: Make loaders not rely on the 'metadata' column of the 'revision' table, from Resolved to Migrated.

Jan 8 2023, 4:32 PM · Core Loader

gitlab-migration changed the status of T2537: Extend new deposit endpoint to support metadata-only deposits, a subtask of T1021: SWORD deposit of metadata about an existing SWH object, from Resolved to Migrated.

Jan 8 2023, 4:31 PM · Core Loader, SWORD deposit

gitlab-migration changed the status of T2482: Migrate git loader tests code to pytest, a subtask of T2481: Migrate dvcs loader tests code to pytest, from Resolved to Migrated.

Jan 8 2023, 4:30 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2471: NPM package angular-ts-manage fails to be properly loaded from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:30 PM · Core Loader

gitlab-migration changed the status of T2462: svn loader: Refactor tests using pytest fixtures, a subtask of T2481: Migrate dvcs loader tests code to pytest, from Resolved to Migrated.

Jan 8 2023, 4:30 PM · SVN Loader, Mercurial loader, Git loader, Core Loader

gitlab-migration changed the status of T2428: Unstuck failing loaders in staging from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:30 PM · Staging environment, Core Loader

gitlab-migration changed the status of T2110: archive-loader: add support for .tar.Z and .tar.lz tarball types from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:29 PM · Core Loader, Origin-GNU

gitlab-migration changed the status of T2017: package loader: Discuss revision metadata normalization from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:28 PM · Metadata workflow, Restricted Project, Core Loader

gitlab-migration changed the status of T1626: loader-core: Unify counter with the storage's results from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:26 PM · Core Loader

gitlab-migration changed the status of T1461: Add loader-related metrics to swh-loader-core from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:26 PM · Core Loader, Metrics/monitoring

gitlab-migration changed the status of T1393: swh/loader/core/loader.py has no proper test coverage from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:26 PM · Core Loader

gitlab-migration changed the status of T1300: Release swh-loader-core v0.0.36 from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:25 PM · Core Loader

gitlab-migration changed the status of T1238: Loader: refactor loader tests to avoid code duplication from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:25 PM · Core Loader

gitlab-migration changed the status of T982: failing worker consumes remaining tasks without processing them from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:24 PM · Core Loader

gitlab-migration changed the status of T804: swh-loader-core: doesn't properly batch object insertions (for big requests) from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:23 PM · Core Loader

gitlab-migration changed the status of T519: loader core should not rely on SHA1 only to decide whether some content is missing or not from Invalid to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:19 PM · Core Loader

Dec 1 2022

vlorentz closed T3273: Use "fork" relationships to speed-up initial load of large repositories as Resolved.

Dec 1 2022, 4:18 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

vlorentz closed T4219: Investigate why GitHub fork detection did not bring a speed-up, a subtask of T3273: Use "fork" relationships to speed-up initial load of large repositories, as Resolved.

Dec 1 2022, 4:18 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

Oct 19 2022

gitlab-migration changed the status of T4266: prod: Deploy swh-loader-core 3.5.0 on staging from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:07 PM · System administration, Core Loader

gitlab-migration changed the status of T4238: Deploy latest loaders version (> v3.4) from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:06 PM · System administration, Core Loader

gitlab-migration changed the status of T4206: prod: Deploy metadata loader v0.0.2, a subtask of T3273: Use "fork" relationships to speed-up initial load of large repositories, from Resolved to Migrated.

Oct 19 2022, 6:06 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

gitlab-migration changed the status of T4204: prod: Deploy swh-scheduler 1.1.1, a subtask of T4186: Allow loaders to fetch extrinsic metadata, from Resolved to Migrated.

Oct 19 2022, 6:06 PM · Core Loader, Metadata workflow

gitlab-migration changed the status of T4205: prod: Deploy swh-loader-core 3.2.1, a subtask of T4204: prod: Deploy swh-scheduler 1.1.1, from Resolved to Migrated.

Oct 19 2022, 6:06 PM · System administration, Core Loader

gitlab-migration changed the status of T4205: prod: Deploy swh-loader-core 3.2.1 from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:06 PM · System administration, Core Loader

gitlab-migration changed the status of T4204: prod: Deploy swh-scheduler 1.1.1 from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:06 PM · System administration, Core Loader

gitlab-migration changed the status of T4194: staging: Deploy swh-scheduler 1.1.0, a subtask of T4187: Pass forge type to loaders, from Resolved to Migrated.

Oct 19 2022, 6:06 PM · Core Loader, Metadata workflow

gitlab-migration changed the status of T3973: Deploy swh.deposit v0.17 from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:06 PM · System administration, Core Loader, SWORD deposit

gitlab-migration changed the status of T3765: Deploy latest swh.loader.core and swh.lister from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:05 PM · System administration, Lister, Core Loader

gitlab-migration changed the status of T3143: Migrate revision metadata to extid in the storage from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:01 PM · System administration, Storage manager, Core Loader

gitlab-migration changed the status of T3143: Migrate revision metadata to extid in the storage, a subtask of T3139: Make loaders stop reading from the 'metadata' column of the 'revision' table, from Resolved to Migrated.

Oct 19 2022, 6:01 PM · Core Loader

gitlab-migration closed T2564: migrate existing revisions metadata extra_headers to actual extra_headers field, a subtask of T3090: Make loaders not rely on the 'metadata' column of the 'revision' table, as Migrated.

Oct 19 2022, 5:58 PM · Core Loader

gitlab-migration changed the status of T872: Deploy and restart indexers, a subtask of T871: Migrate swh-storage api functions relative to indexers to swh-indexer, from Resolved to Migrated.

Oct 19 2022, 5:51 PM · SWORD deposit, Core Loader, Web app, Development environment, Storage manager, Indexer

gitlab-migration changed the status of T872: Deploy and restart indexers from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 5:51 PM · SWORD deposit, Core Loader, Storage manager, Web app, Puppet recipes, Indexer

Aug 4 2022

ardumont changed the status of T4266: prod: Deploy swh-loader-core 3.5.0 on staging from Invalid to Resolved.

Aug 4 2022, 4:10 PM · System administration, Core Loader

Jun 23 2022

vlorentz closed T1461: Add loader-related metrics to swh-loader-core as Resolved.

Closing this, as these metrics are now visible on https://grafana.softwareheritage.org/d/FqGC4zu7z/vlorentz-loader-metrics

Jun 23 2022, 10:59 AM · Core Loader, Metrics/monitoring

May 30 2022

vlorentz added a parent task for T3273: Use "fork" relationships to speed-up initial load of large repositories: T4283: Load https://github.com/chromium/chromium with a higher packfile size limit.

May 30 2022, 3:41 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

May 23 2022

vlorentz closed T4266: prod: Deploy swh-loader-core 3.5.0 on staging as Invalid.

May 23 2022, 2:29 PM · System administration, Core Loader

vlorentz added a comment to T4266: prod: Deploy swh-loader-core 3.5.0 on staging.

oh you're right; this time status is undefined, not success

May 23 2022, 2:29 PM · System administration, Core Loader

ardumont moved T4266: prod: Deploy swh-loader-core 3.5.0 on staging from Backlog to Weekly backlog on the System administration board.

May 23 2022, 2:24 PM · System administration, Core Loader

ardumont added a comment to T4266: prod: Deploy swh-loader-core 3.5.0 on staging.

3.4.1 is deployed everywhere:

root@pergamon:~# clush -b -w @swh-workers -w @swh-workers-large -w @staging-workers 'dpkg -l python3-swh.loader.core'
---------------
worker[01-18],worker[0-3].internal.staging.swh.network (22)
---------------
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                    Version              Architecture Description
+++-=======================-====================-============-=================================
ii  python3-swh.loader.core 3.4.1-1~swh1~bpo10+1 all          Software Heritage Loader Core

May 23 2022, 2:24 PM · System administration, Core Loader

vlorentz triaged T4266: prod: Deploy swh-loader-core 3.5.0 on staging as Normal priority.

May 23 2022, 2:00 PM · System administration, Core Loader

May 13 2022

vlorentz added a revision to T3273: Use "fork" relationships to speed-up initial load of large repositories: D7831: Use all base snapshots in determine_wants().

May 13 2022, 3:23 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

ardumont closed T4238: Deploy latest loaders version (> v3.4) as Resolved.

May 13 2022, 11:47 AM · System administration, Core Loader

ardumont moved T4238: Deploy latest loaders version (> v3.4) from in-progress to deployed/landed/monitoring on the System administration board.

May 13 2022, 11:47 AM · System administration, Core Loader

ardumont updated the task description for T4238: Deploy latest loaders version (> v3.4).

May 13 2022, 11:47 AM · System administration, Core Loader

ardumont updated the task description for T4238: Deploy latest loaders version (> v3.4).

May 13 2022, 11:35 AM · System administration, Core Loader

ardumont updated the task description for T4238: Deploy latest loaders version (> v3.4).

May 13 2022, 10:42 AM · System administration, Core Loader

ardumont moved T4238: Deploy latest loaders version (> v3.4) from Weekly backlog to in-progress on the System administration board.

May 13 2022, 10:31 AM · System administration, Core Loader

ardumont updated the task description for T4238: Deploy latest loaders version (> v3.4).

May 13 2022, 10:23 AM · System administration, Core Loader

ardumont updated the task description for T4238: Deploy latest loaders version (> v3.4).

May 13 2022, 10:23 AM · System administration, Core Loader

ardumont moved T4238: Deploy latest loaders version (> v3.4) from Backlog to Weekly backlog on the System administration board.

May 13 2022, 10:19 AM · System administration, Core Loader

ardumont triaged T4238: Deploy latest loaders version (> v3.4) as High priority.

May 13 2022, 10:17 AM · System administration, Core Loader

May 10 2022

vlorentz added a comment to T3273: Use "fork" relationships to speed-up initial load of large repositories.

Currently can't do it on GitLab while logged out: https://gitlab.com/gitlab-org/gitlab/-/issues/361952

May 10 2022, 4:13 PM · Origin-GitHub, Origin-GitLab, Git loader, Extrinsic metadata, Core Loader

May 3 2022

vlorentz triaged T4201: Support empty directories as Normal priority.

Yes; and they should be already supported in SWH (if some places don't, then it's a bug)

May 3 2022, 4:44 PM · Core Loader, BZR loader