Page MenuHomeSoftware Heritage
Feed All Stories

Oct 22 2021

anlambert updated the diff for D6533: algos/revisions_walker: Handle case of revision without committer date.

Improve comment

Oct 22 2021, 11:33 AM
ardumont requested review of D6537: Dockerfile: Avoid pip 21.3 version preventing working container override.
Oct 22 2021, 11:32 AM
ardumont added a revision to T3685: Our pip usage is broken since a recent pip release: D6537: Dockerfile: Avoid pip 21.3 version preventing working container override.
Oct 22 2021, 11:32 AM · Development environment
anlambert accepted D6441: Deprecate /browse/origin/log/ URLs.

Looks good to me, thanks ! I added a last batch of nitpick comments to handle before landing this.

Oct 22 2021, 11:29 AM
ardumont updated the task description for T3685: Our pip usage is broken since a recent pip release.
Oct 22 2021, 11:26 AM · Development environment
ardumont updated the task description for T3685: Our pip usage is broken since a recent pip release.
Oct 22 2021, 11:26 AM · Development environment
swh-public-ci added a comment to D6536: Restrict the click version to avoid conflict version with celery's.

Build is green

Oct 22 2021, 11:24 AM
ardumont updated the diff for D6536: Restrict the click version to avoid conflict version with celery's.

Adapt

Oct 22 2021, 11:22 AM
ardumont updated the task description for T3685: Our pip usage is broken since a recent pip release.
Oct 22 2021, 11:20 AM · Development environment
ardumont triaged T3685: Our pip usage is broken since a recent pip release as Normal priority.
Oct 22 2021, 11:19 AM · Development environment
anlambert accepted D6536: Restrict the click version to avoid conflict version with celery's.

Looks good to me, I can also reproduce the issues after click upgrade to 8.x.

Oct 22 2021, 11:13 AM
ardumont added inline comments to D6533: algos/revisions_walker: Handle case of revision without committer date.
Oct 22 2021, 10:59 AM
anlambert added inline comments to D6533: algos/revisions_walker: Handle case of revision without committer date.
Oct 22 2021, 10:55 AM
ardumont updated the task description for T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs.
Oct 22 2021, 10:55 AM · System administration, Documentation
ardumont requested review of D6536: Restrict the click version to avoid conflict version with celery's.
Oct 22 2021, 10:54 AM
vsellier committed rDSNIP5411afb8619d: grid5000/cassandra: deploy the vault components (authored by vsellier).
grid5000/cassandra: deploy the vault components
Oct 22 2021, 10:44 AM
ardumont closed D6534: Migrate idrac documentation to sysadm docs.
Oct 22 2021, 10:21 AM
ardumont committed rDDOC4e9e5ce0712e: Migrate idrac documentation to sysadm docs (authored by ardumont).
Migrate idrac documentation to sysadm docs
Oct 22 2021, 10:21 AM
ardumont added a comment to P1205 docker-compose no longer allows override on containers.

After patching the swh-environment/docker/dockerfile:

Oct 22 2021, 10:20 AM
swh-public-ci added a comment to D6534: Migrate idrac documentation to sysadm docs.

Build is green

Oct 22 2021, 10:11 AM
ardumont updated the diff for D6534: Migrate idrac documentation to sysadm docs.

Adapt according to suggestion

Oct 22 2021, 9:59 AM
rdicosmo closed T3682: Public annex page does not show updated content as Resolved.
Oct 22 2021, 9:58 AM · System administration
rdicosmo added a comment to T3682: Public annex page does not show updated content.

Thank you, tried again, and now it works! :-)

Oct 22 2021, 9:58 AM · System administration
olasd added a comment to T3682: Public annex page does not show updated content.

In git annex vicfg, I've changed the profile of the saam copy of the annex to "backup". This should tell further calls to git annex sync --content to send everything there regardless of the numcopies setting. Hopefully?

Oct 22 2021, 9:56 AM · System administration
olasd added a comment to T3682: Public annex page does not show updated content.

Can you please provide a full log of the git annex sync --content operation? It seems that the content hasn't been pushed to saam, which hosts the copy of the data that's served through the web.

Oct 22 2021, 9:39 AM · System administration
swh-public-ci added a comment to D6441: Deprecate /browse/origin/log/ URLs.

Build is green

Oct 22 2021, 9:26 AM

Oct 21 2021

Harbormaster failed remote builds in B24642: Diff 23755 for D6441: Deprecate /browse/origin/log/ URLs!
Oct 21 2021, 6:12 PM
swh-public-ci added a comment to D6441: Deprecate /browse/origin/log/ URLs.

Build was aborted

Oct 21 2021, 6:12 PM
ardumont added inline comments to D6534: Migrate idrac documentation to sysadm docs.
Oct 21 2021, 6:12 PM
vsellier added inline comments to D6534: Migrate idrac documentation to sysadm docs.
Oct 21 2021, 6:11 PM
ardumont added inline comments to D6534: Migrate idrac documentation to sysadm docs.
Oct 21 2021, 6:07 PM
vsellier accepted D6534: Migrate idrac documentation to sysadm docs.

One comment inline, but otherwise it's all good. Thanks

Oct 21 2021, 5:57 PM
ardumont updated the task description for T3684: swh-scheduler: Document the design of the scheduler in its documentation.
Oct 21 2021, 5:55 PM · Scheduling utilities
ardumont triaged T3684: swh-scheduler: Document the design of the scheduler in its documentation as Normal priority.
Oct 21 2021, 5:54 PM · Scheduling utilities
ardumont added inline comments to D6533: algos/revisions_walker: Handle case of revision without committer date.
Oct 21 2021, 5:53 PM
swh-public-ci added a comment to D6534: Migrate idrac documentation to sysadm docs.

Build is green

Oct 21 2021, 5:51 PM
jayeshv updated the diff for D6441: Deprecate /browse/origin/log/ URLs.

Documentation updates

Oct 21 2021, 5:48 PM
vsellier added a comment to T3683: cassandra - benchmark the vault.

The deployment scripts to deploy the vault and the associated components have to be adapted to be deployed on the grid5000 cluster. work in progress...

Oct 21 2021, 5:45 PM · System administration
ardumont accepted D6535: templates/revision-info: Display author fullname when name is None.
Oct 21 2021, 5:45 PM
vsellier changed the status of T3683: cassandra - benchmark the vault from Open to Work in Progress.
Oct 21 2021, 5:43 PM · System administration
ardumont updated the diff for D6534: Migrate idrac documentation to sysadm docs.

Fix noisy characters (trailing ` and some white space)

Oct 21 2021, 5:40 PM
anlambert requested review of D6535: templates/revision-info: Display author fullname when name is None.
Oct 21 2021, 5:40 PM
ardumont added inline comments to D6534: Migrate idrac documentation to sysadm docs.
Oct 21 2021, 5:39 PM
anlambert added inline comments to D6534: Migrate idrac documentation to sysadm docs.
Oct 21 2021, 5:36 PM
rdicosmo renamed T3682: Public annex page does not show updated content from Public annex page does not show updated contet to Public annex page does not show updated content.
Oct 21 2021, 5:34 PM · System administration
rdicosmo triaged T3682: Public annex page does not show updated content as High priority.
Oct 21 2021, 5:34 PM · System administration
moranegg added a comment to T3677: Separate origin-source-code and provenance-metadata in the deposit.

Oh and another use-case we discussed: when browsing an origin/directory/..., showing on the side the set of sources that provided metadata for that origin/directory/.... This is, again, to improve discoverability.

Oct 21 2021, 5:34 PM · SWORD deposit
ardumont added inline comments to D6534: Migrate idrac documentation to sysadm docs.
Oct 21 2021, 5:33 PM
ardumont requested review of D6534: Migrate idrac documentation to sysadm docs.
Oct 21 2021, 5:31 PM
ardumont added a comment to D6492: Add support for pathslicing in seaweedfs backend.

and this one for the pathslicing stuff (may not be kept in the end).

Oct 21 2021, 5:31 PM
ardumont accepted D6532: test_revisions_walker: Migrate from unittest to pytest.
Oct 21 2021, 5:30 PM
anlambert requested review of D6533: algos/revisions_walker: Handle case of revision without committer date.
Oct 21 2021, 5:26 PM
anlambert requested review of D6532: test_revisions_walker: Migrate from unittest to pytest.
Oct 21 2021, 5:24 PM
ardumont added a revision to T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs: D6534: Migrate idrac documentation to sysadm docs.
Oct 21 2021, 5:20 PM · System administration, Documentation
ardumont added a comment to D6520: Add a new cli endpoint to schedule recurrent visits in Celery.

Can you describe the purpose, design, etc. in docs/?

Oct 21 2021, 5:14 PM
swh-public-ci added a comment to D6441: Deprecate /browse/origin/log/ URLs.

Build is green

Oct 21 2021, 4:59 PM
jayeshv updated the diff for D6441: Deprecate /browse/origin/log/ URLs.

Requested changes

Oct 21 2021, 4:44 PM
ardumont updated the title for P1205 docker-compose no longer allows override on containers from untitled to docker-compose no longer allows me to override some containers.
Oct 21 2021, 4:43 PM
swh-public-ci added a comment to D6441: Deprecate /browse/origin/log/ URLs.

Build is green

Oct 21 2021, 4:32 PM
vlorentz closed D6530: Remove unused 'known_artifacts' code.
Oct 21 2021, 4:29 PM
vlorentz committed rDLDBASE9f8827935da6: Remove unused 'known_artifacts' code (authored by vlorentz).
Remove unused 'known_artifacts' code
Oct 21 2021, 4:29 PM
anlambert requested changes to D6441: Deprecate /browse/origin/log/ URLs.

This is moving forward, great ! I added a couple of inline comments to handle.

Oct 21 2021, 4:28 PM
moranegg updated the task description for T3680: Modify deposit admin view to better reflect metadata-only deposits.
Oct 21 2021, 4:26 PM · SWORD deposit
swh-public-ci added a comment to D6530: Remove unused 'known_artifacts' code.

Build is green

Oct 21 2021, 4:25 PM
vlorentz added a comment to T3677: Separate origin-source-code and provenance-metadata in the deposit.

Oh and another use-case we discussed: when browsing an origin/directory/..., showing on the side the set of sources that provided metadata for that origin/directory/.... This is, again, to improve discoverability.

Oct 21 2021, 4:25 PM · SWORD deposit
vlorentz added a comment to D6520: Add a new cli endpoint to schedule recurrent visits in Celery.

Can you describe the purpose, design, etc. in docs/?

Oct 21 2021, 4:23 PM
moranegg updated the task description for T3680: Modify deposit admin view to better reflect metadata-only deposits.
Oct 21 2021, 4:22 PM · SWORD deposit
vlorentz updated the diff for D6530: Remove unused 'known_artifacts' code.

rebase

Oct 21 2021, 4:22 PM
vlorentz committed rDLDBASEf6905cdf547c: deposit: Remove 'parent' deposit (authored by vlorentz).
deposit: Remove 'parent' deposit
Oct 21 2021, 4:21 PM
vlorentz closed D6529: deposit: Remove 'parent' deposit.
Oct 21 2021, 4:21 PM
moranegg added a comment to T3677: Separate origin-source-code and provenance-metadata in the deposit.

Following our discussion, we might call the tag metadata-source, since using the term origin for metadata can be misleading.

Oct 21 2021, 4:21 PM · SWORD deposit
vlorentz added a comment to T3677: Separate origin-source-code and provenance-metadata in the deposit.

Why do we want that:

  1. as you analyzed: recognize the deposit in the UI
Oct 21 2021, 4:21 PM · SWORD deposit
jayeshv updated the diff for D6441: Deprecate /browse/origin/log/ URLs.

Removed unused code

Oct 21 2021, 4:17 PM
ardumont created P1205 docker-compose no longer allows override on containers.
Oct 21 2021, 4:16 PM
ardumont published D6520: Add a new cli endpoint to schedule recurrent visits in Celery for review.
Oct 21 2021, 4:02 PM
ardumont updated the task description for T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs.
Oct 21 2021, 3:29 PM · System administration, Documentation
ardumont updated the task description for T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs.
Oct 21 2021, 3:27 PM · System administration, Documentation
moranegg added a comment to T3677: Separate origin-source-code and provenance-metadata in the deposit.

Why do we want that:

  1. as you analyzed: recognize the deposit in the UI
  2. recognize in the deposit storage, what is the source (origin) of the deposit
    • here it happens that the deposit itself is only metadata
  3. identify which deposits are metadata-only deposits
Oct 21 2021, 3:27 PM · SWORD deposit
ardumont updated the task description for T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs.
Oct 21 2021, 3:26 PM · System administration, Documentation
ardumont updated the task description for T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs.
Oct 21 2021, 3:24 PM · System administration, Documentation
ardumont updated the task description for T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs.
Oct 21 2021, 3:24 PM · System administration, Documentation
anlambert closed D6531: services/keycloak: Update keycloak_swh_setup.py script.
Oct 21 2021, 3:20 PM
anlambert committed rDENVaa331105560e: services/keycloak: Update keycloak_swh_setup.py script (authored by anlambert).
services/keycloak: Update keycloak_swh_setup.py script
Oct 21 2021, 3:20 PM
swh-public-ci added a comment to D6441: Deprecate /browse/origin/log/ URLs.

Build is green

Oct 21 2021, 3:18 PM
ardumont accepted D6529: deposit: Remove 'parent' deposit.

Mmmm, ok i guess. I don't recall the reason that made it that way initially.

Oct 21 2021, 3:14 PM
ardumont added 1 blocking reviewer(s) for D6529: deposit: Remove 'parent' deposit: moranegg.
Oct 21 2021, 3:14 PM
ardumont accepted D6530: Remove unused 'known_artifacts' code.

apparently so.

Oct 21 2021, 3:12 PM
jayeshv updated the summary of D6441: Deprecate /browse/origin/log/ URLs.
Oct 21 2021, 3:10 PM
ardumont accepted D6531: services/keycloak: Update keycloak_swh_setup.py script.
Oct 21 2021, 3:10 PM
anlambert requested review of D6531: services/keycloak: Update keycloak_swh_setup.py script.
Oct 21 2021, 3:06 PM
vlorentz requested review of D6530: Remove unused 'known_artifacts' code.
Oct 21 2021, 3:05 PM
jayeshv updated the diff for D6441: Deprecate /browse/origin/log/ URLs.

Changes requested in the review

Oct 21 2021, 2:58 PM
vsellier added a task to D6494: cassandra: Fix incomplete check of content existence in object_find_by_sha1_git: T3577: Parallel loaders performances .
Oct 21 2021, 2:58 PM
vsellier added revisions to T3577: Parallel loaders performances : D6423: cassandra: Add alternative algorithms to list missing objects, D6494: cassandra: Fix incomplete check of content existence in object_find_by_sha1_git, D6495: cassandra: Rewrite content_missing to run queries concurrently..
Oct 21 2021, 2:58 PM · System administration, Storage manager
vsellier closed T3577: Parallel loaders performances , a subtask of T3357: Perform some tests of the cassandra storage on Grid5000, as Resolved.
Oct 21 2021, 2:56 PM · System administration, Storage manager
vsellier closed T3577: Parallel loaders performances as Resolved.

Stopping here the investigations as the limit is now the hardware running cassandra (HDD). Great improvements were made compared to the first tries

Oct 21 2021, 2:56 PM · System administration, Storage manager
vlorentz requested review of D6529: deposit: Remove 'parent' deposit.
Oct 21 2021, 2:38 PM
vsellier added a comment to T3577: Parallel loaders performances .

Changing the cache size doesn't seem very effective for the particular workload of the loaders:
Exception some contextual differences due probably due to the different origin loaded, the performances are quite similar.
The hit ratios between the different configurations are also very close

Oct 21 2021, 2:36 PM · System administration, Storage manager
anlambert closed D6528: keycloak: Add swh.web.admin.list_deposits role to swh-web client.
Oct 21 2021, 2:34 PM
anlambert committed rSPSITEe7759626904c: keycloak: Add swh.web.admin.list_deposits role to swh-web client (authored by anlambert).
keycloak: Add swh.web.admin.list_deposits role to swh-web client
Oct 21 2021, 2:34 PM