Page MenuHomeSoftware Heritage
Feed All Stories

Oct 26 2021

ardumont removed a task from D5966: Update documentation entry point with the new loader cvs module: T2845: Improve Subversion loader and develop CVS loader.
Oct 26 2021, 11:56 AM
ardumont removed a task from D5988: initial CVS loader stub: T2845: Improve Subversion loader and develop CVS loader.
Oct 26 2021, 11:56 AM
ardumont removed a task from D6278: Update container to install the cvs dependency: T2845: Improve Subversion loader and develop CVS loader.
Oct 26 2021, 11:56 AM
ardumont removed revisions from T2845: Improve Subversion loader and develop CVS loader: D5988: initial CVS loader stub, D6278: Update container to install the cvs dependency, D5966: Update documentation entry point with the new loader cvs module.
Oct 26 2021, 11:56 AM · Archive coverage
ardumont added a task to D6278: Update container to install the cvs dependency: T3691: Implement CVS loader.
Oct 26 2021, 11:56 AM
ardumont added revisions to T3691: Implement CVS loader: D6278: Update container to install the cvs dependency, D5966: Update documentation entry point with the new loader cvs module.
Oct 26 2021, 11:56 AM · CVS loader, Archive coverage
ardumont added a task to D5966: Update documentation entry point with the new loader cvs module: T3691: Implement CVS loader.
Oct 26 2021, 11:56 AM
ardumont added a task to D5988: initial CVS loader stub: T3691: Implement CVS loader.
Oct 26 2021, 11:55 AM
ardumont added a revision to T3691: Implement CVS loader: D5988: initial CVS loader stub.
Oct 26 2021, 11:55 AM · CVS loader, Archive coverage
anlambert requested review of D6552: assets/save: Simplify origin URLs validation code.
Oct 26 2021, 11:55 AM
ardumont triaged T3691: Implement CVS loader as Normal priority.
Oct 26 2021, 11:51 AM · CVS loader, Archive coverage
anlambert requested review of D6551: assets/save: Fix sourceforge git repository URL validation.
Oct 26 2021, 11:45 AM
zack changed the edit policy for T2845: Improve Subversion loader and develop CVS loader.
Oct 26 2021, 11:44 AM · Archive coverage
anlambert added a reviewer for D6547: jobs/swh-environment: Remove pip version restriction: Reviewers.
Oct 26 2021, 11:31 AM
anlambert added a revision to T3690: Save code now ui rejects valid git sourceforge origin: D6551: assets/save: Fix sourceforge git repository URL validation.
Oct 26 2021, 11:30 AM · Save Code Now
ardumont updated the task description for T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs.
Oct 26 2021, 11:25 AM · System administration, Documentation
anlambert added a comment to T3690: Save code now ui rejects valid git sourceforge origin.

This is because I made the wrong assumption that URL paths to git repos for the git.code.sf.net netloc are necessarily prefixed by /p/.
So URLs starting with https://git.code.sf.net/u/ are currently rejected when submitting save requests through the Web UI , see related frontend code.

Oct 26 2021, 11:18 AM · Save Code Now
ardumont closed D6549: Migrate upgrade swh services and storage db migration.
Oct 26 2021, 11:12 AM
ardumont committed rDDOC8ab8240dc0e8: Migrate upgrade swh services and storage db migration (authored by ardumont).
Migrate upgrade swh services and storage db migration
Oct 26 2021, 11:12 AM
ardumont requested review of D6550: wip: Log full pack size and check pack file limit after log instruction.
Oct 26 2021, 10:52 AM
ardumont added a revision to T3627: Consider dropping pull request references from the git loader ingestion: D6550: wip: Log full pack size and check pack file limit after log instruction.
Oct 26 2021, 10:50 AM · Git loader
ardumont updated the task description for T3690: Save code now ui rejects valid git sourceforge origin.
Oct 26 2021, 10:40 AM · Save Code Now
ardumont added a project to T3690: Save code now ui rejects valid git sourceforge origin: Save Code Now.
Oct 26 2021, 10:38 AM · Save Code Now
ardumont updated subscribers of T3690: Save code now ui rejects valid git sourceforge origin.
Oct 26 2021, 10:37 AM · Save Code Now
ardumont triaged T3690: Save code now ui rejects valid git sourceforge origin as Normal priority.
Oct 26 2021, 10:37 AM · Save Code Now
ardumont created P1209 ingestion ok.
Oct 26 2021, 10:32 AM
swh-public-ci added a comment to D6424: Perfect hashmap C implementation.

Build is green

Oct 26 2021, 9:33 AM
dachary updated the diff for D6424: Perfect hashmap C implementation.

fix documentation bugs

Oct 26 2021, 9:32 AM
Harbormaster failed remote builds in B24690: Diff 23806 for D6424: Perfect hashmap C implementation!
Oct 26 2021, 9:19 AM
swh-public-ci added a comment to D6424: Perfect hashmap C implementation.

Build has FAILED

Oct 26 2021, 9:18 AM
dachary updated the task description for T3670: fed4fire setup for winery benchmarks.
Oct 26 2021, 9:18 AM · Object storage
dachary updated the diff for D6424: Perfect hashmap C implementation.

document the benchmark process

Oct 26 2021, 9:17 AM
dachary added a comment to T3670: fed4fire setup for winery benchmarks.

But then Fed4Fire killed the experiment prematurely, ignoring the Grid5000 extension. I'll have another go at it.

Oct 26 2021, 9:16 AM · Object storage
dachary added a comment to T3670: fed4fire setup for winery benchmarks.

When trying to extend the duration of an experiment (slice in the Fed4Fire parlance), an error occurs.

Oct 26 2021, 5:57 AM · Object storage
dachary added a comment to T3670: fed4fire setup for winery benchmarks.

The CLI is actually more complicated because it requires input that are difficult to figure out:

Oct 26 2021, 4:32 AM · Object storage

Oct 25 2021

dachary added a comment to T3634: Create swh-perfecthash module.

@olasd thanks for adding the dependencies 🎉 D6545

Oct 25 2021, 11:26 PM · Object storage
ardumont created P1208 sampled origins: Patch (drop PR branches) or no patch (current version), failure to ingest with huge packfile.
Oct 25 2021, 9:46 PM · Git loader
borisbaldassari added a comment to D6395: lister: Add new maven lister.

Let's keep this as is now.
For next time though, please keep the old diff id around even if you think the review is
harder ;)

Sure. Sorry for the mess. :-)
One note through: the old diff at [1] has been constantly updated and is still relevant for history.

Oct 25 2021, 7:13 PM
Harbormaster failed remote builds in B24689: Diff 23805 for D6520: Add a new cli endpoint to schedule recurrent visits in Celery!
Oct 25 2021, 7:07 PM
swh-public-ci added a comment to D6520: Add a new cli endpoint to schedule recurrent visits in Celery.

Build has FAILED

Oct 25 2021, 7:07 PM
olasd updated the diff for D6520: Add a new cli endpoint to schedule recurrent visits in Celery.

Amend with reimplementation from pair programming session

Oct 25 2021, 7:04 PM
olasd commandeered D6520: Add a new cli endpoint to schedule recurrent visits in Celery.
Oct 25 2021, 7:02 PM
swh-public-ci added a comment to D6395: lister: Add new maven lister.

Build is green

Oct 25 2021, 6:51 PM
ardumont requested review of D6549: Migrate upgrade swh services and storage db migration.
Oct 25 2021, 6:50 PM
borisbaldassari updated the diff for D6395: lister: Add new maven lister.
  • maven-lister: Add statefulness
Oct 25 2021, 6:48 PM
ardumont added a revision to T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs: D6549: Migrate upgrade swh services and storage db migration.
Oct 25 2021, 6:30 PM · System administration, Documentation
ardumont added inline comments to D6548: Instantiate a noop objstorage for testing purposes.
Oct 25 2021, 5:38 PM
ardumont requested review of D6548: Instantiate a noop objstorage for testing purposes.
Oct 25 2021, 5:34 PM
ardumont added a revision to T3627: Consider dropping pull request references from the git loader ingestion: D6548: Instantiate a noop objstorage for testing purposes.
Oct 25 2021, 5:25 PM · Git loader
anlambert closed D6546: api/graph: Fix error responses for invalid request Accept header.
Oct 25 2021, 3:44 PM
anlambert committed rDWAPPSeb1d40198a08: api/graph: Fix error responses for invalid request Accept header (authored by anlambert).
api/graph: Fix error responses for invalid request Accept header
Oct 25 2021, 3:44 PM
vlorentz accepted D6546: api/graph: Fix error responses for invalid request Accept header.
Oct 25 2021, 3:12 PM
anlambert requested review of D6547: jobs/swh-environment: Remove pip version restriction.
Oct 25 2021, 2:45 PM
anlambert requested review of D6546: api/graph: Fix error responses for invalid request Accept header.
Oct 25 2021, 2:01 PM
ardumont closed T3675: Investigate why cran origins are marked as non-visited in the scheduler metrics as Resolved.
Oct 25 2021, 1:11 PM · Scheduling utilities
ardumont added a comment to T3675: Investigate why cran origins are marked as non-visited in the scheduler metrics.

Remaining coverage fix is deployed.

Oct 25 2021, 1:10 PM · Scheduling utilities
ardumont accepted D6545: base-buster: add valgrind and googletest for swh.perfecthash.
Oct 25 2021, 12:10 PM
ardumont committed rDWAPPS1edcf35c4849: coverage: Drop the cran origins computation workaround (authored by ardumont).
coverage: Drop the cran origins computation workaround
Oct 25 2021, 12:07 PM
ardumont closed D6544: coverage: Drop the cran origins computation workaround.
Oct 25 2021, 12:07 PM
dachary added a comment to T3670: fed4fire setup for winery benchmarks.

The https://jfed.ilabt.imec.be/downloads/ CLI may be easier to use than the graphical client when repeatin experiments.

Oct 25 2021, 11:56 AM · Object storage
swh-public-ci added a comment to D6544: coverage: Drop the cran origins computation workaround.

Build is green

Oct 25 2021, 11:43 AM
anlambert accepted D6544: coverage: Drop the cran origins computation workaround.

LGTM, thanks !

Oct 25 2021, 11:38 AM
ardumont updated the diff for D6544: coverage: Drop the cran origins computation workaround.

Adapt correctly according to review

Oct 25 2021, 11:30 AM
dachary retitled D6545: base-buster: add valgrind and googletest for swh.perfecthash from base-buster: add valgrind for swh.perfecthash to base-buster: add valgrind and googletest for swh.perfecthash.
Oct 25 2021, 11:18 AM
dachary updated the diff for D6545: base-buster: add valgrind and googletest for swh.perfecthash.

googletest is also needed

Oct 25 2021, 11:18 AM
swh-public-ci added a comment to D6424: Perfect hashmap C implementation.

Build is green

Oct 25 2021, 11:08 AM
dachary updated the diff for D6424: Perfect hashmap C implementation.

run the valgrind tests

Oct 25 2021, 11:08 AM
ardumont accepted D6545: base-buster: add valgrind and googletest for swh.perfecthash.
Oct 25 2021, 11:02 AM
ardumont added reviewers for D6545: base-buster: add valgrind and googletest for swh.perfecthash: System administrators, Reviewers.
Oct 25 2021, 11:00 AM
dachary retitled D6545: base-buster: add valgrind and googletest for swh.perfecthash from base-buster: add valrind for swh.perfecthash to base-buster: add valgrind for swh.perfecthash.
Oct 25 2021, 10:58 AM
dachary updated the diff for D6545: base-buster: add valgrind and googletest for swh.perfecthash.

fix typo in the commit comment

Oct 25 2021, 10:57 AM
anlambert added inline comments to D6544: coverage: Drop the cran origins computation workaround.
Oct 25 2021, 10:57 AM
dachary requested review of D6545: base-buster: add valgrind and googletest for swh.perfecthash.
Oct 25 2021, 10:56 AM
dachary added a revision to T3634: Create swh-perfecthash module: D6545: base-buster: add valgrind and googletest for swh.perfecthash.
Oct 25 2021, 10:56 AM · Object storage
dachary added a comment to T3634: Create swh-perfecthash module.

It's at https://forge.softwareheritage.org/source/swh-jenkins-dockerfiles/browse/master/base-buster/Dockerfile$39

Oct 25 2021, 10:55 AM · Object storage
ardumont planned changes to D6520: Add a new cli endpoint to schedule recurrent visits in Celery.

Discussing with @olasd, possible improvments:

  • add backoff in between task type iteration
  • one thread per task type
Oct 25 2021, 10:51 AM
vlorentz placed T3569: Document the RPC architecture up for grabs.
Oct 25 2021, 10:37 AM · Core & foundations, Documentation
vlorentz claimed T3569: Document the RPC architecture.
Oct 25 2021, 10:37 AM · Core & foundations, Documentation
ardumont added a comment to D6395: lister: Add new maven lister.

That said if you want mNe to reuse the previous diff (D6133), now that I know how to
do that properly (arc diff commit_ref), I'll do it happily (but it will be messy).

Oct 25 2021, 10:19 AM
ardumont added inline comments to D6544: coverage: Drop the cran origins computation workaround.
Oct 25 2021, 10:04 AM
ardumont added inline comments to D6544: coverage: Drop the cran origins computation workaround.
Oct 25 2021, 10:01 AM

Oct 24 2021

swh-public-ci added a comment to D6395: lister: Add new maven lister.

Build is green

Oct 24 2021, 5:24 PM
borisbaldassari updated the diff for D6395: lister: Add new maven lister.
  • maven-lister: Add maven lister with review D6133.
  • maven-lister: Add statefulness
Oct 24 2021, 5:21 PM
swh-public-ci added a comment to D6133: maven-lister: initialise lister..

Build is green

Oct 24 2021, 5:20 PM
borisbaldassari updated the diff for D6133: maven-lister: initialise lister..
  • maven-lister: initialise lister.
  • maven-lister: Refactor file get loop (review D6133).
  • maven-lister: update readme, set tests to use http_index
  • maven-lister: Add statefulness
  • maven-lister: Add incremental task and tests
Oct 24 2021, 5:17 PM
rdicosmo committed rMSLD58b45184c301: Add reference to EU recommendation 2018 (authored by rdicosmo).
Add reference to EU recommendation 2018
Oct 24 2021, 4:10 PM
rdicosmo committed rMSLD537807672db6: Fix typos (authored by rdicosmo).
Fix typos
Oct 24 2021, 10:38 AM
rdicosmo committed rMSLD1200b0a41987: Add support to remove appendix in generic Makefile (authored by rdicosmo).
Add support to remove appendix in generic Makefile
Oct 24 2021, 10:38 AM

Oct 23 2021

borisbaldassari added a comment to D6395: lister: Add new maven lister.

Thanks for your time, @ardumont.

Oct 23 2021, 11:26 AM
swh-public-ci added a comment to D6395: lister: Add new maven lister.

Build is green

Oct 23 2021, 11:21 AM
borisbaldassari updated the diff for D6395: lister: Add new maven lister.
  • maven-lister: Add statefulness
Oct 23 2021, 11:18 AM

Oct 22 2021

anlambert requested changes to D6544: coverage: Drop the cran origins computation workaround.
Oct 22 2021, 6:44 PM
zack added a comment to T3621: Create a production read-only objstorage.

@vsellier sure, and thanks! "Basic auth" is in the HTTP sense, right? So username/password pairs that we can add on demand, correct?

Oct 22 2021, 6:12 PM · System administration
ardumont requested review of D6544: coverage: Drop the cran origins computation workaround.
Oct 22 2021, 5:44 PM
ardumont added a comment to T3675: Investigate why cran origins are marked as non-visited in the scheduler metrics.

Cran loader worked hard and fast, metrics got updated and now are consistent with reality [1]
So we can land D6544, deploy and be done with this issue.

Oct 22 2021, 5:32 PM · Scheduling utilities
ardumont added a revision to T3675: Investigate why cran origins are marked as non-visited in the scheduler metrics: D6544: coverage: Drop the cran origins computation workaround.
Oct 22 2021, 5:29 PM · Scheduling utilities
ardumont requested review of D6543: docker: Install scheduler runner which deals with recurrent tasks.
Oct 22 2021, 5:19 PM
ardumont added a revision to T3667: Orchestrate origins scheduling according to scheduler metrics feedback: D6543: docker: Install scheduler runner which deals with recurrent tasks.
Oct 22 2021, 5:19 PM · System administration, Sprint 2021 01, Archive coverage, Scheduling utilities
anlambert closed T3645: cgit: handle "429 Too Many Requests" errors as Resolved.

Fixed in swh-lister v2.2.0 and deployed to production, closing this.

Oct 22 2021, 5:16 PM · Origin-CGit, CGit lister
anlambert closed T3686: svn: Add missing coverage on svn implementation as Resolved by committing rDLDSVN6578b85dd024: loader: Fix SvnLoaderFromDumpArchive and add test.
Oct 22 2021, 5:09 PM · SVN Loader