Page MenuHomeSoftware Heritage
Feed All Stories

Oct 31 2022

ardumont added a comment to T4538: Consider archiving NAR hashes.

Oh awesome! Thanks for pushing forward. :-)

Oct 31 2022, 12:14 PM · SVN Loader, Tarball loader, Nixguix loader
anlambert requested review of D8794: converters, dumb: Fix errors since latest dulwich release (0.20.50).
Oct 31 2022, 11:51 AM
swh-public-ci added a comment to D8793: replay: Ensure copyfrom operations are properly handled.

Build is green

Oct 31 2022, 11:43 AM
anlambert updated the diff for D8793: replay: Ensure copyfrom operations are properly handled.

Update:

  • rename some variables
  • fix some typing
  • harmonize replay.DirEditor.add_[file|directory] implementation
Oct 31 2022, 11:41 AM
vlorentz triaged T4666: swh-graph returns 500 errors when counting origins containing two particular contents as Low priority.
Oct 31 2022, 10:37 AM · Compressed graph service
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO41b5e6a54503: Updated backport on buster-swh from debian/1.7.2-1_swh1 (unstable-swh) (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Updated backport on buster-swh from debian/1.7.2-1_swh1 (unstable-swh)
Oct 31 2022, 10:10 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTOe06db735a30c: Merge tag 'debian/1.7.2-1_swh1' into debian/buster-swh (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Merge tag 'debian/1.7.2-1_swh1' into debian/buster-swh
Oct 31 2022, 10:10 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO86e611f08a02: pristine-tar data for swh-storage_1.7.2.orig.tar.gz (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
pristine-tar data for swh-storage_1.7.2.orig.tar.gz
Oct 31 2022, 10:01 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTOac3b2b8f614a: Updated debian changelog for version 1.7.2 (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Updated debian changelog for version 1.7.2
Oct 31 2022, 10:01 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO09379d859069: Update upstream source from tag 'debian/upstream/1.7.2' (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Update upstream source from tag 'debian/upstream/1.7.2'
Oct 31 2022, 10:01 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO3f35a0aa3fdb: New upstream version 1.7.2 (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
New upstream version 1.7.2
Oct 31 2022, 10:01 AM
Vesepeb638 updated Vesepeb638.
Oct 31 2022, 7:41 AM
Vesepeb638 updated Vesepeb638.
Oct 31 2022, 7:41 AM
Vesepeb638 updated Vesepeb638.
Oct 31 2022, 7:40 AM
Vesepeb638 updated Vesepeb638.
Oct 31 2022, 7:39 AM

Oct 28 2022

anlambert requested review of D8793: replay: Ensure copyfrom operations are properly handled.
Oct 28 2022, 6:18 PM
rdicosmo moved T3949: Refactor swh.web from Backlog to Work in progress on the Roadmap 2022 board.
Oct 28 2022, 4:23 PM · meta-task, Roadmap 2022, Web app
rdicosmo moved T2220: swh-graph in production from Backlog to Work in progress on the Roadmap 2022 board.
Oct 28 2022, 4:23 PM · Roadmap 2022, meta-task, Roadmap 2021, Compressed graph service
rdicosmo moved T3775: Dealing with repositories with contents that produces hash conflicts (example included from GitLab) from Backlog to Work in progress on the Roadmap 2022 board.
Oct 28 2022, 4:22 PM · meta-task, Roadmap 2022, Save Code Now, Origin-GitLab
rdicosmo moved T3960: Publicly available standard for SWHID version 1 from Backlog to Work in progress on the Roadmap 2022 board.
Oct 28 2022, 4:22 PM · meta-task, Roadmap 2022
rdicosmo moved T2225: Migrate to GitLab from Backlog to Work in progress on the Roadmap 2022 board.
Oct 28 2022, 4:22 PM · meta-task, Roadmap 2022, GitLab migration, Roadmap 2020
anlambert updated the diff for D8792: docker/conf/nginx: Remove no longer needed proxy and fix formatting.

Rebase

Oct 28 2022, 4:01 PM
anlambert updated the diff for D8791: docker/nginx: Align objstorage proxy_read_timeout value with production.

Remove extra line break

Oct 28 2022, 4:01 PM
anlambert requested review of D8792: docker/conf/nginx: Remove no longer needed proxy and fix formatting.
Oct 28 2022, 3:59 PM
anlambert requested review of D8791: docker/nginx: Align objstorage proxy_read_timeout value with production.
Oct 28 2022, 3:54 PM
anlambert closed D8790: webapp: Ensure monospace font family for docutils literals.
Oct 28 2022, 3:21 PM
anlambert committed rDWAPPS94a28741b69c: webapp: Ensure monospace font family for docutils literals (authored by anlambert).
webapp: Ensure monospace font family for docutils literals
Oct 28 2022, 3:21 PM
jayeshv accepted D8790: webapp: Ensure monospace font family for docutils literals.
Oct 28 2022, 3:09 PM
anlambert requested review of D8790: webapp: Ensure monospace font family for docutils literals.
Oct 28 2022, 3:08 PM
anlambert closed D8789: pubdev: Update outdated lister documentation.
Oct 28 2022, 11:55 AM
anlambert committed rDLS60707a45ddfd: pubdev: Update outdated lister documentation (authored by anlambert).
pubdev: Update outdated lister documentation
Oct 28 2022, 11:55 AM
anlambert requested changes to D8569: Add rubygems loader.

Could you also test that raw extrinsic metadata have been correctly stored in the storage and can be retrieved ?
You can inspire from the cpan loader test to implement it.

Oct 28 2022, 11:54 AM
swh-public-ci added a comment to D8789: pubdev: Update outdated lister documentation.

Build is green

Oct 28 2022, 11:51 AM
anlambert updated the diff for D8789: pubdev: Update outdated lister documentation.

Rebase

Oct 28 2022, 11:46 AM
anlambert added a comment to D8569: Add rubygems loader.

Rubygems: Improve loader to make use of artifacts and rubygems_metadata provided by the lister extra_loader_arguments

Use artifacts and rubygems_metadata to get list of versions, artifacts checksums and extrinsic metadata url
Add an EXTID manifest
Add metadata from extrinsic metadata

@anlambert Please note I used 'rubygems_metadata' instead of 'rubygem_metadata' as in the lister. Maybe I'm wrong but I think the lister should rename to rubygems_metadata?

Oct 28 2022, 11:32 AM
swh-public-ci added a comment to D8787: loader: Compress dump file and rework truncated dump handling.

Build is green

Oct 28 2022, 11:25 AM
swh-public-ci added a comment to D8786: svn: Ensure to quote URLs provided as parameters to client methods.

Build is green

Oct 28 2022, 11:24 AM
anlambert updated the diff for D8787: loader: Compress dump file and rework truncated dump handling.

Rebase

Oct 28 2022, 11:22 AM
anlambert updated the diff for D8786: svn: Ensure to quote URLs provided as parameters to client methods.

Only quote spaces as svn will also raise an assertion if we quote other special characters.

Oct 28 2022, 11:21 AM
samplet added a comment to D8760: Add payloads to ExtIDs.

It looks like the changes fail because CI doesn't know to use the model updates in D8759. Is there something I can do to fix it or do I just wait to run CI after D8759 is landed?

Oct 28 2022, 7:31 AM
Harbormaster failed remote builds in B32631: Diff 31679 for D8760: Add payloads to ExtIDs!
Oct 28 2022, 1:24 AM
swh-public-ci added a comment to D8760: Add payloads to ExtIDs.

Build has FAILED

Oct 28 2022, 1:24 AM
samplet updated the diff for D8760: Add payloads to ExtIDs.

Switch to using Git SHA-1 for the payload.

Oct 28 2022, 1:23 AM

Oct 27 2022

franckbret added a comment to D8569: Add rubygems loader.

Rubygems: Improve loader to make use of artifacts and rubygems_metadata provided by the lister extra_loader_arguments

Use artifacts and rubygems_metadata to get list of versions, artifacts checksums and extrinsic metadata url
Add an EXTID manifest
Add metadata from extrinsic metadata

Oct 27 2022, 7:05 PM
swh-public-ci added a comment to D8569: Add rubygems loader.

Build is green

Oct 27 2022, 7:05 PM
franckbret updated the diff for D8569: Add rubygems loader.

Rubygems: Improve loader to make use of artifacts and rubygems_metadata provided by the lister extra_loader_arguments

Oct 27 2022, 7:02 PM
vlorentz accepted D8787: loader: Compress dump file and rework truncated dump handling.
Oct 27 2022, 6:54 PM
vlorentz accepted D8789: pubdev: Update outdated lister documentation.
Oct 27 2022, 6:52 PM
anlambert closed D8782: templates/stage-cypress-tests: Prevent race condition with support file.
Oct 27 2022, 6:22 PM
anlambert committed rCJSWHa0fd3dc0fe59: templates/stage-cypress-tests: Prevent race condition with support file (authored by anlambert).
templates/stage-cypress-tests: Prevent race condition with support file
Oct 27 2022, 6:22 PM
anlambert added a comment to D8782: templates/stage-cypress-tests: Prevent race condition with support file.
In D8782#228466, @olasd wrote:

I'm a bit confused about who is copying what in there, but I trust your judgment...

Oct 27 2022, 6:21 PM
anlambert closed D8788: tests: Fix test_replay_content_static_group_id.
Oct 27 2022, 6:17 PM
anlambert committed rDOBJSRPL0c0fa5c73374: tests: Fix test_replay_content_static_group_id (authored by anlambert).
tests: Fix test_replay_content_static_group_id
Oct 27 2022, 6:17 PM
olasd accepted D8788: tests: Fix test_replay_content_static_group_id.

Oh wow, what an awful hack (I should know, I wrote it).

Oct 27 2022, 6:02 PM
olasd accepted D8782: templates/stage-cypress-tests: Prevent race condition with support file.

I'm a bit confused about who is copying what in there, but I trust your judgment...

Oct 27 2022, 6:00 PM
anlambert requested review of D8789: pubdev: Update outdated lister documentation.
Oct 27 2022, 5:02 PM
anlambert requested review of D8788: tests: Fix test_replay_content_static_group_id.
Oct 27 2022, 4:57 PM
anlambert added a comment to T4665: [pudbev lister] use alternative packages list endpoint.

We are already using the https://pub.dev/api/package-names endpoint to list pubdev packages (see code), it is just the lister documentation that is outdated.

I will push a diff to fix the doc.

Oct 27 2022, 4:57 PM · PubDev lister
anlambert added a project to T4665: [pudbev lister] use alternative packages list endpoint: PubDev lister.
Oct 27 2022, 4:57 PM · PubDev lister
swh-public-ci added a comment to D8787: loader: Compress dump file and rework truncated dump handling.

Build is green

Oct 27 2022, 4:41 PM
anlambert closed T4665: [pudbev lister] use alternative packages list endpoint as Invalid.

We are already using the https://pub.dev/api/package-names endpoint to list pubdev packages (see code), it is just the lister documentation that is outdated.

Oct 27 2022, 4:41 PM · PubDev lister
anlambert closed T4665: [pudbev lister] use alternative packages list endpoint, a subtask of T4465: Ingest pub.dev (Dart, Flutter), as Invalid.
Oct 27 2022, 4:41 PM · PubDev loader, PubDev lister, Archive coverage
anlambert updated the diff for D8787: loader: Compress dump file and rework truncated dump handling.

Rebase

Oct 27 2022, 4:37 PM
anlambert updated the summary of D8787: loader: Compress dump file and rework truncated dump handling.
Oct 27 2022, 4:37 PM
franckbret added a comment to T4665: [pudbev lister] use alternative packages list endpoint.

Ok, thanks for the feedback, will have a look on this next week, I will be out of office until wednesday.

Oct 27 2022, 4:10 PM · PubDev lister
anlambert requested review of D8787: loader: Compress dump file and rework truncated dump handling.
Oct 27 2022, 3:53 PM
KShivendu requested review of D8753: feat: Introduce RPM loader.
Oct 27 2022, 3:32 PM
swh-public-ci added a comment to D8786: svn: Ensure to quote URLs provided as parameters to client methods.

Build is green

Oct 27 2022, 2:42 PM
anlambert updated the diff for D8786: svn: Ensure to quote URLs provided as parameters to client methods.

Update safe chars list

Oct 27 2022, 2:36 PM
anlambert closed T4645: SWHAP Event - Link to sciencestories.io is dead as Resolved.
Oct 27 2022, 2:28 PM · Website
anlambert added a comment to T4645: SWHAP Event - Link to sciencestories.io is dead.

@vlorentz, you misread the task, protocol should be http not https

Oct 27 2022, 2:28 PM · Website
vlorentz closed D8778: codemeta: Fix incorrect output namespace for dates and URLs.
Oct 27 2022, 2:24 PM
vlorentz closed D8779: codemeta: Fix malformed dates that used to be allowed by the deposit.
Oct 27 2022, 2:24 PM
vlorentz committed rDCIDXc0052f8e48fa: codemeta: Fix incorrect output namespace for dates and URLs (authored by vlorentz).
codemeta: Fix incorrect output namespace for dates and URLs
Oct 27 2022, 2:24 PM
vlorentz committed rDCIDX3bad41489c4b: codemeta: Fix malformed dates that used to be allowed by the deposit (authored by vlorentz).
codemeta: Fix malformed dates that used to be allowed by the deposit
Oct 27 2022, 2:24 PM
olasd committed rCDFJ49db710b38ad: Install cpio in base image (authored by olasd).
Install cpio in base image
Oct 27 2022, 2:24 PM
olasd committed rCDFJ3a34a564856d: Upgrade kafka to 3.3.1 (authored by olasd).
Upgrade kafka to 3.3.1
Oct 27 2022, 2:24 PM
vlorentz closed D8772: metadata_dictionary: Systematically check input URLs before adding to graph.
Oct 27 2022, 2:23 PM
vlorentz committed rDCIDXa66d5b240ab7: metadata_dictionary: Systematically check input URLs before adding to graph (authored by vlorentz).
metadata_dictionary: Systematically check input URLs before adding to graph
Oct 27 2022, 2:23 PM
vlorentz added a comment to T4599: Github descriptions are not used to search origins.

depends on https://gitlab.softwareheritage.org/infra/sysadm-environment/-/issues/4652

Oct 27 2022, 2:22 PM · Metadata workflow, Archive search
swh-public-ci added a comment to D8664: Raise 400 instead of 500 on 'incomplete' multipart request.

Build is green

Oct 27 2022, 2:21 PM
vlorentz reopened T4645: SWHAP Event - Link to sciencestories.io is dead as "Open".
Oct 27 2022, 2:20 PM · Website
vlorentz closed D8664: Raise 400 instead of 500 on 'incomplete' multipart request.
Oct 27 2022, 2:18 PM
vlorentz committed rDDEPe0d553df6ffe: Raise 400 instead of 500 on 'incomplete' multipart request (authored by vlorentz).
Raise 400 instead of 500 on 'incomplete' multipart request
Oct 27 2022, 2:18 PM
bchauvet added a parent task for T4665: [pudbev lister] use alternative packages list endpoint: T4465: Ingest pub.dev (Dart, Flutter).
Oct 27 2022, 2:14 PM · PubDev lister
bchauvet added a subtask for T4465: Ingest pub.dev (Dart, Flutter): T4665: [pudbev lister] use alternative packages list endpoint.
Oct 27 2022, 2:14 PM · PubDev loader, PubDev lister, Archive coverage
vlorentz updated the diff for D8664: Raise 400 instead of 500 on 'incomplete' multipart request.

rebase

Oct 27 2022, 2:13 PM
bchauvet triaged T4665: [pudbev lister] use alternative packages list endpoint as Normal priority.
Oct 27 2022, 2:13 PM · PubDev lister
vlorentz added a comment to T4645: SWHAP Event - Link to sciencestories.io is dead.

Nope. And curl sees the same thing I do:

Oct 27 2022, 2:12 PM · Website
anlambert added a comment to T4645: SWHAP Event - Link to sciencestories.io is dead.

@vlorentz, I do not from my side, link is fixed. I just forced browser cache to be invalidated for our WP site assets. Did it change something for you ?

Oct 27 2022, 1:49 PM · Website
vlorentz triaged T4664: Message keys written by the indexers contain indexer_configuration_id as Low priority.
Oct 27 2022, 1:42 PM · Indexer
vlorentz added a comment to T4645: SWHAP Event - Link to sciencestories.io is dead.

@anlambert I still see the old link on https://www.softwareheritage.org/news/events/swhap_days_2022/

Oct 27 2022, 1:26 PM · Website
KShivendu closed D8784: base/buster: add rpm2cpio dependency.
Oct 27 2022, 12:12 PM
KShivendu committed rCDFJ2b7e64f0ff39: base/buster: add rpm2cpio dependency (authored by KShivendu).
base/buster: add rpm2cpio dependency
Oct 27 2022, 12:12 PM
anlambert closed T4645: SWHAP Event - Link to sciencestories.io is dead as Resolved.
Oct 27 2022, 11:45 AM · Website
anlambert requested review of D8786: svn: Ensure to quote URLs provided as parameters to client methods.
Oct 27 2022, 11:41 AM
vlorentz added a comment to T4657: Allow object removal from journal.

heh that's funny, olasd and I also discussed this yesterday (though we were interested in changing objects rather than deleting):

Oct 27 2022, 11:36 AM · Journal
vlorentz triaged T4657: Allow object removal from journal as Normal priority.
Oct 27 2022, 11:32 AM · Journal
vlorentz triaged T4645: SWHAP Event - Link to sciencestories.io is dead as Unbreak Now! priority.
Oct 27 2022, 11:32 AM · Website
vlorentz requested changes to D8760: Add payloads to ExtIDs.

Thanks, looks good too.

Oct 27 2022, 10:03 AM