Page MenuHomeSoftware Heritage
Feed All Stories

Nov 17 2022

ardumont closed D8852: Migrate forge.s.o url to gitlab.s.o.
Nov 17 2022, 3:53 PM
ardumont committed rDDOC280074fb4ca3: deployment/argocd: Migrate forge.s.o url to gitlab.s.o (authored by ardumont).
deployment/argocd: Migrate forge.s.o url to gitlab.s.o
Nov 17 2022, 3:53 PM
ardumont requested review of D8852: Migrate forge.s.o url to gitlab.s.o.
Nov 17 2022, 3:48 PM
Harbormaster failed to build B32842: rDDEP5dfe8fd61bf0: docs: Fix typo for rDDEP5dfe8fd61bf0: docs: Fix typo!
Nov 17 2022, 3:29 PM
vlorentz committed rDDEP5dfe8fd61bf0: docs: Fix typo (authored by vlorentz).
docs: Fix typo
Nov 17 2022, 3:28 PM
ardumont committed rDDOCc345bf6b2145: sysadm: Migrate forge.s.o url to gitlab.s.o (authored by ardumont).
sysadm: Migrate forge.s.o url to gitlab.s.o
Nov 17 2022, 3:14 PM
ardumont committed rDDOCdcae6c16211e: sysadm/third-party: Update links and mention deprecated script (authored by ardumont).
sysadm/third-party: Update links and mention deprecated script
Nov 17 2022, 3:14 PM
ardumont committed rDDOC6dc30f752f63: sysadm/onboarding: Reference new gitlab registration step (authored by ardumont).
sysadm/onboarding: Reference new gitlab registration step
Nov 17 2022, 3:14 PM
ardumont closed D8851: sysadm: Migrate forge.s.o url to gitlab.s.o.
Nov 17 2022, 3:14 PM
ardumont committed rDDOCad4b1f45bef8: sysadm: Migrate forge.s.o url to gitlab.s.o (authored by ardumont).
sysadm: Migrate forge.s.o url to gitlab.s.o
Nov 17 2022, 3:14 PM
ardumont retitled D8851: sysadm: Migrate forge.s.o url to gitlab.s.o from [wip] sysadm: Migrate forge.s.o url to gitlab.s.o to sysadm: Migrate forge.s.o url to gitlab.s.o.
Nov 17 2022, 3:12 PM
ardumont requested review of D8851: sysadm: Migrate forge.s.o url to gitlab.s.o.
Nov 17 2022, 12:14 PM
vlorentz added a member for Reviewers: lunar.
Nov 17 2022, 11:23 AM
ardumont accepted D8848: fedora: Update versions in packages dict provided as loader argument.

lgtm, one question inline.

Nov 17 2022, 9:57 AM
ardumont abandoned D8850: swh-loader-svn: Build rsvndump dependency.
Nov 17 2022, 9:36 AM
ardumont added a comment to D8850: swh-loader-svn: Build rsvndump dependency.

Well, wrong forge to push diff.
Fixing that in our gitlab instance [1].

Nov 17 2022, 9:36 AM

Nov 16 2022

lunar updated subscribers of T4657: Allow object removal from journal.

After discussing D8833 and discussing the general API with @olasd, it feels like adding a delete method to swh.journal.writer.interface.JournalWriterInterface is the wrong approach. To try to capture our discussion

Nov 16 2022, 6:05 PM · Journal
anlambert committed rDWAPPS44dfeee1b591: package.json: Revert cypress upgrade to 11.x (authored by anlambert).
package.json: Revert cypress upgrade to 11.x
Nov 16 2022, 5:14 PM
ardumont added a reviewer for D8850: swh-loader-svn: Build rsvndump dependency: System administrators.
Nov 16 2022, 4:50 PM
ardumont requested review of D8850: swh-loader-svn: Build rsvndump dependency.
Nov 16 2022, 4:29 PM
Harbormaster failed to build B32836: rDWAPPS13ac1fd97a7e: api: Improve HTML documentation for rDWAPPS13ac1fd97a7e: api: Improve HTML documentation!
Nov 16 2022, 4:08 PM
Harbormaster failed to build B32837: rDWAPPSfc5bf62ea1c3: package.json: Upgrade depdendencies for rDWAPPSfc5bf62ea1c3: package.json: Upgrade depdendencies!
Nov 16 2022, 4:07 PM
anlambert committed rDWAPPSfc5bf62ea1c3: package.json: Upgrade depdendencies (authored by anlambert).
package.json: Upgrade depdendencies
Nov 16 2022, 3:57 PM
anlambert closed D8837: api: Improve HTML documentation.
Nov 16 2022, 3:57 PM
anlambert committed rDWAPPS13ac1fd97a7e: api: Improve HTML documentation (authored by anlambert).
api: Improve HTML documentation
Nov 16 2022, 3:57 PM
swh-public-ci added a comment to D8837: api: Improve HTML documentation.

Build is green

Nov 16 2022, 3:44 PM
anlambert updated the diff for D8837: api: Improve HTML documentation.

Rebase

Nov 16 2022, 3:30 PM
anlambert closed D8841: save_code_now: Allow request creation if previous for origin is running.
Nov 16 2022, 3:29 PM
anlambert committed rDWAPPSab849109ea1d: save_code_now: Allow request creation if previous for origin is running (authored by anlambert).
save_code_now: Allow request creation if previous for origin is running
Nov 16 2022, 3:29 PM
swh-public-ci added a comment to D8841: save_code_now: Allow request creation if previous for origin is running.

Build is green

Nov 16 2022, 3:26 PM
anlambert updated the diff for D8841: save_code_now: Allow request creation if previous for origin is running.

Rebase

Nov 16 2022, 3:15 PM
ardumont accepted D8841: save_code_now: Allow request creation if previous for origin is running.
Nov 16 2022, 2:56 PM
KShivendu closed D8753: feat: Introduce RPM loader.
Nov 16 2022, 2:19 PM
KShivendu committed rDLDBASEa196c85d5ab1: feat: Incremental RPM loader implementation (authored by KShivendu).
feat: Incremental RPM loader implementation
Nov 16 2022, 2:19 PM
anlambert accepted D8753: feat: Introduce RPM loader.

Looks good to me, thanks !

Nov 16 2022, 2:18 PM
swh-public-ci added a comment to D8753: feat: Introduce RPM loader.

Build is green

Nov 16 2022, 2:13 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Fix license headers
  • Fix test cases to match Fedora lister output
Nov 16 2022, 2:09 PM
anlambert requested changes to D8753: feat: Introduce RPM loader.

@KShivendu , before landing this please fix the year in some license headers and update tests to match fedora lister output.

Nov 16 2022, 10:56 AM
bchauvet updated the task description for T4104: Ingest crates.io (Rust).
Nov 16 2022, 10:45 AM · Crates loader, Crates lister, Archive coverage
swh-public-ci added a comment to D8753: feat: Introduce RPM loader.

Build is green

Nov 16 2022, 9:55 AM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Use intrinsic version (and other suggestions by @anlambert)
  • Update tests for the same
Nov 16 2022, 9:51 AM
franckbret added a comment to T4687: Hex.pm lister (Erlang package manager).

@KShivendu Hi, we had a similar situation with rubygems loader, see its _load_directory method

Nov 16 2022, 9:23 AM · Lister

Nov 15 2022

anlambert added inline comments to D8753: feat: Introduce RPM loader.
Nov 15 2022, 5:41 PM
anlambert requested changes to D8753: feat: Introduce RPM loader.

@KShivendu , we are almost in a landable state for the RPM loader.

Nov 15 2022, 5:34 PM
anlambert requested review of D8848: fedora: Update versions in packages dict provided as loader argument.
Nov 15 2022, 5:17 PM
anlambert added a revision to T4448: Implementation of Fedora Lister: D8848: fedora: Update versions in packages dict provided as loader argument.
Nov 15 2022, 5:11 PM · RPM lister, Archive coverage
Harbormaster failed to build B32823: rDWAPPS694802b2ac85: browse: Display branches names in snapshot releases view for rDWAPPS694802b2ac85: browse: Display branches names in snapshot releases view!
Nov 15 2022, 5:00 PM
anlambert closed D8847: browse: Display branches names in snapshot releases view.
Nov 15 2022, 4:50 PM
anlambert committed rDWAPPS694802b2ac85: browse: Display branches names in snapshot releases view (authored by anlambert).
browse: Display branches names in snapshot releases view
Nov 15 2022, 4:50 PM
KShivendu added inline comments to D8753: feat: Introduce RPM loader.
Nov 15 2022, 3:43 PM
swh-public-ci added a comment to D8753: feat: Introduce RPM loader.

Build is green

Nov 15 2022, 3:42 PM
KShivendu updated the diff for D8753: feat: Introduce RPM loader.
  • Rebase against master
Nov 15 2022, 3:35 PM
KShivendu updated the task description for T4687: Hex.pm lister (Erlang package manager).
Nov 15 2022, 3:30 PM · Lister
KShivendu added a comment to T4687: Hex.pm lister (Erlang package manager).

I have a question: The .tar files obtained from this have a contents.tar.gz which contains the real source code. Is there a way to extract this internal .tar file using the current tar loader?

Nov 15 2022, 3:21 PM · Lister
KShivendu updated the task description for T4687: Hex.pm lister (Erlang package manager).
Nov 15 2022, 3:19 PM · Lister
KShivendu created T4687: Hex.pm lister (Erlang package manager).
Nov 15 2022, 3:17 PM · Lister
lunar committed rCJSWH693ebea630a4: Use a volatile resource lock for host port 5080 (authored by lunar).
Use a volatile resource lock for host port 5080
Nov 15 2022, 2:57 PM
lunar closed D8838: Use a volatile resource lock for host port 5080.
Nov 15 2022, 2:57 PM
lunar committed rCJSWH7a97d4a503de: Convert swh-docker-dev job from freestyle to Pipeline (authored by lunar).
Convert swh-docker-dev job from freestyle to Pipeline
Nov 15 2022, 2:57 PM
olasd accepted D8838: Use a volatile resource lock for host port 5080.

Thank you!

Nov 15 2022, 2:56 PM
swh-public-ci added a comment to D6380: Allow partial snapshot creation during ingestion.

Build is green

Nov 15 2022, 2:29 PM
ardumont updated the diff for D6380: Allow partial snapshot creation during ingestion.

Amend impacted docstrings

Nov 15 2022, 2:25 PM
swh-public-ci added a comment to D8847: browse: Display branches names in snapshot releases view.

Build is green

Nov 15 2022, 2:09 PM
swh-public-ci added a comment to D6380: Allow partial snapshot creation during ingestion.

Build is green

Nov 15 2022, 2:05 PM
ardumont accepted D8847: browse: Display branches names in snapshot releases view.
Nov 15 2022, 2:01 PM
ardumont updated the diff for D6380: Allow partial snapshot creation during ingestion.

Adapt according to discussion

Nov 15 2022, 1:59 PM
anlambert updated the diff for D8847: browse: Display branches names in snapshot releases view.

Update commit message

Nov 15 2022, 1:58 PM
anlambert retitled D8847: browse: Display branches names in snapshot releases view from browse: Display branch name in snapshot releases view to browse: Display branches names in snapshot releases view.
Nov 15 2022, 1:57 PM
anlambert requested review of D8847: browse: Display branches names in snapshot releases view.
Nov 15 2022, 1:54 PM
lunar closed D8846: Remove obsolete test setup command from developper documentation.
Nov 15 2022, 1:53 PM
lunar committed rDJNL0d63977a19ce: Remove obsolete test setup command from developper documentation (authored by lunar).
Remove obsolete test setup command from developper documentation
Nov 15 2022, 1:53 PM
anlambert added a revision to T4686: Duplicated branch names are rendered by the webapp: D8847: browse: Display branches names in snapshot releases view.
Nov 15 2022, 1:44 PM · Web app
anlambert accepted D8846: Remove obsolete test setup command from developper documentation.
Nov 15 2022, 12:32 PM
lunar requested review of D8846: Remove obsolete test setup command from developper documentation.
Nov 15 2022, 12:27 PM
ardumont closed D8845: Drop unused core.loader.DVCSLoader class.
Nov 15 2022, 12:12 PM
ardumont committed rDLDBASE31ab1aa69e5b: Drop unused core.loader.DVCSLoader class (authored by ardumont).
Drop unused core.loader.DVCSLoader class
Nov 15 2022, 12:12 PM
vlorentz accepted D8845: Drop unused core.loader.DVCSLoader class.
Nov 15 2022, 12:11 PM
ardumont requested review of D8845: Drop unused core.loader.DVCSLoader class.
Nov 15 2022, 12:09 PM
anlambert closed D8840: maven: Add support for md5 checkums to check download integrity.
Nov 15 2022, 11:48 AM
anlambert committed rDLDBASE301502cbf8dc: maven: Add support for md5 checkums to check download integrity (authored by anlambert).
maven: Add support for md5 checkums to check download integrity
Nov 15 2022, 11:48 AM
anlambert closed D8839: maven: Simplify tests with requests_mock_datadir fixture.
Nov 15 2022, 11:48 AM
anlambert committed rDLDBASE5778cfce008a: maven: Simplify tests with requests_mock_datadir fixture (authored by anlambert).
maven: Simplify tests with requests_mock_datadir fixture
Nov 15 2022, 11:48 AM
vlorentz closed D8832: luigi: Add DownloadFromS3 task.
Nov 15 2022, 11:46 AM
vlorentz committed rDDATASET23853dbfacd4: luigi: Add DownloadFromS3 task (authored by vlorentz).
luigi: Add DownloadFromS3 task
Nov 15 2022, 11:46 AM
vlorentz committed rDDATASET418cf1837e26: luigi: Make Format and ObjectType public (authored by vlorentz).
luigi: Make Format and ObjectType public
Nov 15 2022, 11:46 AM
vlorentz added inline comments to D6380: Allow partial snapshot creation during ingestion.
Nov 15 2022, 11:46 AM
vlorentz added inline comments to D6380: Allow partial snapshot creation during ingestion.
Nov 15 2022, 11:44 AM
anlambert added a comment to T4686: Duplicated branch names are rendered by the webapp.

The issue comes from the fact that the webapp gets the release names from the release objects of our data model and not
from the branch names targeting releases in associated snapshot object.

Nov 15 2022, 11:34 AM · Web app
KShivendu closed D8386: feat(fedora): Introduce fedora lister.
Nov 15 2022, 11:31 AM
KShivendu committed rDLS6ad61aec2394: feat(fedora): Introduce fedora lister (authored by KShivendu).
feat(fedora): Introduce fedora lister
Nov 15 2022, 11:31 AM
swh-public-ci added a comment to D8386: feat(fedora): Introduce fedora lister.

Build is green

Nov 15 2022, 11:30 AM
KShivendu updated the diff for D8386: feat(fedora): Introduce fedora lister.
  • Rebase against master
Nov 15 2022, 11:24 AM
anlambert renamed T4686: Duplicated branch names are rendered by the webapp from Duplicate branch names rendered by the web app to Duplicated branch names are rendered by the webapp.
Nov 15 2022, 11:22 AM · Web app
KShivendu created T4686: Duplicated branch names are rendered by the webapp.
Nov 15 2022, 11:21 AM · Web app
anlambert added a comment to D8386: feat(fedora): Introduce fedora lister.
In D8386#229912, @olasd wrote:
In D8386#229882, @olasd wrote:

@anlambert

I noticed that https://archive.softwareheritage.org/browse/origin/directory/?origin_url=deb://Ubuntu/packages/nginx has duplicate branch names, which is very confusing. In fact, even the default branch is repeated twice and I see two check marks. If we use branch names like 0.3.9-15.fc26, won't the same happen with Fedora listers? It doesn't seem to differentiate between the editions. (or does it?)

This seems like a misfeature in the webapp:

https://archive.softwareheritage.org/api/1/snapshot/158a3f36b0bd3da461fb7458de44cfa2c94e4270/

The snapshot has multiple branches, with the same version suffix, pointing at the same objects (because the exact same version of the package is present in multiple Ubuntu suites).

I'm not 100% sure how we should be fixing that, but that bug shouldn't prevent you from giving the fedora snapshots the "semantically correct" structure.

I also noticed that yesterday evening and I was also wondering what is the best way to fix that. I see two possible options:

  1. We change the names of the keys in snapshot branches dictionary in order to use the intrinsic version of a debian package instead of its extrinsic one (meaning releases/bionic-security/main/1.14.0-0ubuntu1.10 should rather be releases/1.14.0-0ubuntu1.10)
  2. We update the webapp to filter duplicated releases before display as the release name is used instead of the snapshot branches key associated to the release

I would rather go for the second one as keeping the debian/ubuntu suites and components in keys of snapshot branches dictionary seems of interest.
We could do the same for the fedora case as based on my tests it is quite common that extrinsic versions in the form [0-9].[0-9].[0-9]-[0-9].fc[0-9]+
target the same intrinsic version [0-9].[0-9].[0-9]-[0-9].

We should move this branch (ha) of discussion to a separate task. I would really want us to keep (and display) the information about what suites have been detected to contain what package.

How does the webapp get the release name showed in the menu? From the release object, or from the branch name?

I think it would make sense to display the branch name after the release name, if it didn't match the usual refs/tags/$(release name) pattern

Nov 15 2022, 11:20 AM
ardumont added inline comments to D8842: metadata-search: Return swh-search even when missing from idx_storage.origin_intrinsic_metadata.
Nov 15 2022, 11:08 AM
vlorentz added inline comments to D8842: metadata-search: Return swh-search even when missing from idx_storage.origin_intrinsic_metadata.
Nov 15 2022, 11:07 AM
ardumont accepted D8832: luigi: Add DownloadFromS3 task.
Nov 15 2022, 11:06 AM
vlorentz closed D8844: origin-search: Only request 'url' field.
Nov 15 2022, 11:06 AM
vlorentz closed D8843: metadata-search: Skip query to swh-indexer when its results would be discarded.
Nov 15 2022, 11:05 AM