Page MenuHomeSoftware Heritage
Feed All Stories

Sep 3 2021

vlorentz committed rDWAPPS23d3d68f1968: vault: Rename downloaded files to replace ':' with '_' in SWHIDs (authored by vlorentz).
vault: Rename downloaded files to replace ':' with '_' in SWHIDs
Sep 3 2021, 1:43 PM
vlorentz closed D6171: Switch the UI to use bare git archives, instead of gitfast..
Sep 3 2021, 1:43 PM
vlorentz committed rDWAPPSf324ac3a549c: Switch the UI to use bare git archives, instead of gitfast. (authored by vlorentz).
Switch the UI to use bare git archives, instead of gitfast.
Sep 3 2021, 1:43 PM
swh-public-ci added a comment to D6175: vault-ui: Remove remaining references to object_type.

Build is green

Sep 3 2021, 1:43 PM
swh-public-ci added a comment to D6174: vault: Rename downloaded files to replace ':' with '_' in SWHIDs.

Build is green

Sep 3 2021, 1:43 PM
olasd added inline comments to D6177: Send scheduler metrics to prometheus.
Sep 3 2021, 1:40 PM
vlorentz updated the diff for D6175: vault-ui: Remove remaining references to object_type.

rebase

Sep 3 2021, 1:28 PM
vlorentz updated the diff for D6174: vault: Rename downloaded files to replace ':' with '_' in SWHIDs.

rebase

Sep 3 2021, 1:28 PM
vlorentz updated the diff for D6171: Switch the UI to use bare git archives, instead of gitfast..

rebase

Sep 3 2021, 1:28 PM
anlambert added a comment to T3489: Implement iframe view for content and directory elements.

I discussed this option with Kat from Data Current (in charge of the SWH stories interface) and she was wondering if the highlighted text will be centered.
Seems from your example that this will be the case.

Sep 3 2021, 1:22 PM · Software Stories, Web app
ardumont accepted D6178: Update netbox to 2.11.12.

awesome, thx.

Sep 3 2021, 12:43 PM
ardumont updated the test plan for D6177: Send scheduler metrics to prometheus.
Sep 3 2021, 12:40 PM
ardumont updated the summary of D6177: Send scheduler metrics to prometheus.
Sep 3 2021, 12:39 PM
ardumont updated the diff for D6177: Send scheduler metrics to prometheus.
  • Adapt according to review:
    • Distinguish correctly between labels and values
    • Make last_update a timestamp
    • Drop '_metrics' from the name as prometheus will add it itself
    • Keep name and instance_name as distinguished labels
  • db1.staging: Activate metrics for staging scheduler db
  • Drop secondary cluster as it's unnecessary a filter
  • Make the db name a regexp to be compatible both with prod and staging dbs
Sep 3 2021, 12:35 PM
ardumont edited P1149 scheduler metrics output tryout.
Sep 3 2021, 12:31 PM
moranegg added a comment to T3489: Implement iframe view for content and directory elements.

This is really good!
I discussed this option with Kat from Data Current (in charge of the SWH stories interface) and she was wondering if the highlighted text will be centered.
Seems from your example that this will be the case.

Sep 3 2021, 12:28 PM · Software Stories, Web app
moranegg updated subscribers of T3375: Add column 'client' in moderation view.

@anlambert we have discussed this task this morning with @ardumont and @vlorentz.
I want to start working on the improvements of the deposit admin view to open it up for deposit clients.
Would you have time during September to help on this task? (it is a roadmap task, btw)

Sep 3 2021, 12:24 PM · Monitoring, SWORD deposit, Web app
moranegg added a parent task for T3549: Analyze deposit-client on InvenioRDM compatibility : T2344: Build a connector for software deposit via Zenodo/InvenioRDM.
Sep 3 2021, 12:21 PM · SWORD deposit
moranegg added a subtask for T2344: Build a connector for software deposit via Zenodo/InvenioRDM: T3549: Analyze deposit-client on InvenioRDM compatibility .
Sep 3 2021, 12:21 PM · meta-task, Roadmap 2022, Roadmap 2020, SWORD deposit, Scientific Community Building
olasd added a comment to D6177: Send scheduler metrics to prometheus.

We should keep lister_name and lister_instance as separate labels. Labels don't cost anything as they're stored once per time series (and it would allow us to generate aggregated metrics for all instances of a given lister).

Sep 3 2021, 12:12 PM
moranegg assigned T3192: Add possibility to fetch last SWHID for a deposit using an origin on deposit cli to ardumont.
Sep 3 2021, 12:07 PM · Easy hack, SWORD deposit
vlorentz claimed T3376: Visualize metadata of a deposit in the admin (moderation) view.

Easiest option would be to add a link to the API endpoint.

Sep 3 2021, 12:07 PM · Monitoring, SWORD deposit, Web app
moranegg updated subscribers of T3473: Discuss the multi-request deposit evolution .

To discuss with @rdicosmo

Sep 3 2021, 12:03 PM · SWORD deposit
vlorentz renamed T3473: Discuss the multi-request deposit evolution from Discuss the multi-part deposit evolution to Discuss the multi-request deposit evolution .
Sep 3 2021, 12:01 PM · SWORD deposit
vsellier accepted D6178: Update netbox to 2.11.12.

thanks
LGTM

Sep 3 2021, 11:59 AM
moranegg triaged T3549: Analyze deposit-client on InvenioRDM compatibility as Normal priority.
Sep 3 2021, 11:58 AM · SWORD deposit
olasd requested review of D6178: Update netbox to 2.11.12.
Sep 3 2021, 11:50 AM
vlorentz added a comment to T3192: Add possibility to fetch last SWHID for a deposit using an origin on deposit cli.

This is possible with the main API:

Sep 3 2021, 11:44 AM · Easy hack, SWORD deposit
vlorentz moved T1021: SWORD deposit of metadata about an existing SWH object from Backlog to Deployed on the SWORD deposit board.
Sep 3 2021, 11:39 AM · Core Loader, SWORD deposit
vlorentz closed T1021: SWORD deposit of metadata about an existing SWH object as Resolved.
Sep 3 2021, 11:38 AM · Core Loader, SWORD deposit
vlorentz added a revision to T3018: Allow querying raw_extrinsic_metadata by hash in swh-storage: D5865: Add endpoints to access REMD by id.
Sep 3 2021, 11:38 AM · Storage manager, Extrinsic metadata
vlorentz added a task to D5865: Add endpoints to access REMD by id: T3018: Allow querying raw_extrinsic_metadata by hash in swh-storage.
Sep 3 2021, 11:38 AM
vlorentz closed T3018: Allow querying raw_extrinsic_metadata by hash in swh-storage, a subtask of T2703: Use intrinsic identifiers/hashes for RawExtrinsicMetadata objects, as Resolved.
Sep 3 2021, 11:38 AM · Data Model, Storage manager, Extrinsic metadata
vlorentz closed T3018: Allow querying raw_extrinsic_metadata by hash in swh-storage as Resolved.
Sep 3 2021, 11:38 AM · Storage manager, Extrinsic metadata
moranegg moved T3466: Enable Sentry for the deposit server from Backlog to Landed/Tests/Validations (staging) on the SWORD deposit board.
Sep 3 2021, 11:36 AM · System administration, Sentry, SWORD deposit
ardumont closed T3546: Status "scheduled" over hours, but archiving doesn't seem to start as Resolved.
Sep 3 2021, 11:28 AM · Save Code Now
ardumont added a comment to T3546: Status "scheduled" over hours, but archiving doesn't seem to start.

I've opened a task to try and monitor this pattern [1].

Sep 3 2021, 11:28 AM · Save Code Now
ardumont triaged T3548: Monitor and raise alert when save code now requests seem unusually long as Normal priority.
Sep 3 2021, 11:27 AM · System administration, Save Code Now, Monitoring
ardumont added a comment to T3546: Status "scheduled" over hours, but archiving doesn't seem to start.

I actually requested https://github.com/tue-alga/CoordinatedSchematization which did
not start for over 45 minutes or so (which is unusual), then noticed that the one
above was also in "scheduled" state over hours and decided to open a task here.

Sep 3 2021, 11:25 AM · Save Code Now
mdidas added a comment to T3546: Status "scheduled" over hours, but archiving doesn't seem to start.

Thanks for the quick help!

Sep 3 2021, 11:22 AM · Save Code Now
mdidas added a comment to T3546: Status "scheduled" over hours, but archiving doesn't seem to start.

I actually requested https://github.com/tue-alga/CoordinatedSchematization which did not start for over 45 minutes or so (which is unusual), then noticed that the one above was also in "scheduled" state over hours and decided to open a task here.

Sep 3 2021, 11:21 AM · Save Code Now
ardumont added a comment to T3546: Status "scheduled" over hours, but archiving doesn't seem to start.

For some unknown reason, that particular origins is scheduled but I don't see any log about that run.
I triggered it back and it got finished almost immediately (uneventful).

Sep 3 2021, 11:20 AM · Save Code Now
anlambert accepted D6171: Switch the UI to use bare git archives, instead of gitfast..
Sep 3 2021, 11:12 AM
ardumont added a comment to T3546: Status "scheduled" over hours, but archiving doesn't seem to start.

(was looking into it)
I'd say this one qualifies (in UTC+2):

Sep 3 2021, 11:09 AM · Save Code Now
vlorentz updated the task description for T3547: Debian loader: TypeError: __init__() missing 1 required positional argument: 'md5sum'.
Sep 3 2021, 11:07 AM · Debian loader
vlorentz updated the task description for T3547: Debian loader: TypeError: __init__() missing 1 required positional argument: 'md5sum'.
Sep 3 2021, 11:06 AM · Debian loader
vlorentz added a project to T3547: Debian loader: TypeError: __init__() missing 1 required positional argument: 'md5sum': Debian loader.
Sep 3 2021, 11:06 AM · Debian loader
vlorentz triaged T3547: Debian loader: TypeError: __init__() missing 1 required positional argument: 'md5sum' as High priority.
Sep 3 2021, 11:06 AM · Debian loader
vlorentz triaged T3546: Status "scheduled" over hours, but archiving doesn't seem to start as Unbreak Now! priority.

@mdidas what are the repositories you requested?

Sep 3 2021, 10:52 AM · Save Code Now
douardda abandoned D5648: Add a bit of logging in the buffer proxy storage.
Sep 3 2021, 10:50 AM
douardda abandoned D4920: Randomize last_update in generated ListedOrigins in fill_test_data.
Sep 3 2021, 10:49 AM
ardumont updated the test plan for D6177: Send scheduler metrics to prometheus.
Sep 3 2021, 10:14 AM
vsellier added a comment to T3433: Deploy swh.search v0.10/v0.11.
  • puppet configuration deployed in staging
  • read index updated with this script:
#!/bin/bash
Sep 3 2021, 9:57 AM · System administration, Archive search
vsellier closed D6176: swh-search: deploy v0.11.4 in staging.
Sep 3 2021, 9:47 AM
vsellier committed rSPSITEf8bd91737496: swh-search: deploy v0.11.4 in staging (authored by vsellier).
swh-search: deploy v0.11.4 in staging
Sep 3 2021, 9:47 AM
ardumont updated the diff for D6177: Send scheduler metrics to prometheus.

Format query

Sep 3 2021, 9:47 AM
ardumont updated the test plan for D6177: Send scheduler metrics to prometheus.
Sep 3 2021, 9:46 AM
vsellier updated the test plan for D6176: swh-search: deploy v0.11.4 in staging.
Sep 3 2021, 9:45 AM
ardumont updated the summary of D6177: Send scheduler metrics to prometheus.
Sep 3 2021, 9:36 AM
ardumont created P1149 scheduler metrics output tryout.
Sep 3 2021, 9:35 AM
ardumont requested review of D6177: Send scheduler metrics to prometheus.
Sep 3 2021, 9:34 AM
ardumont added a revision to T3538: Send scheduler metrics to prometheus: D6177: Send scheduler metrics to prometheus.
Sep 3 2021, 9:34 AM · System administration, Monitoring, Scheduling utilities
ardumont moved T3538: Send scheduler metrics to prometheus from Backlog to Weekly backlog on the System administration board.
Sep 3 2021, 9:02 AM · System administration, Monitoring, Scheduling utilities
ardumont added a project to T3538: Send scheduler metrics to prometheus: System administration.
Sep 3 2021, 9:02 AM · System administration, Monitoring, Scheduling utilities
ardumont accepted D6176: swh-search: deploy v0.11.4 in staging.
Sep 3 2021, 9:00 AM
vsellier requested review of D6176: swh-search: deploy v0.11.4 in staging.
Sep 3 2021, 8:42 AM
vsellier added a revision to T3433: Deploy swh.search v0.10/v0.11: D6176: swh-search: deploy v0.11.4 in staging.
Sep 3 2021, 8:42 AM · System administration, Archive search
mdidas updated the task description for T3546: Status "scheduled" over hours, but archiving doesn't seem to start.
Sep 3 2021, 8:40 AM · Save Code Now
vsellier added a comment to T3433: Deploy swh.search v0.10/v0.11.

The lag has recovered in ~ 12hours.
The content of the index looks goods (just cherry picked a couple of origin).

Sep 3 2021, 8:34 AM · System administration, Archive search
mdidas created T3546: Status "scheduled" over hours, but archiving doesn't seem to start.
Sep 3 2021, 7:45 AM · Save Code Now

Sep 2 2021

vlorentz added inline comments to D6171: Switch the UI to use bare git archives, instead of gitfast..
Sep 2 2021, 7:29 PM
anlambert accepted D6175: vault-ui: Remove remaining references to object_type.
Sep 2 2021, 6:53 PM
vlorentz requested review of D6175: vault-ui: Remove remaining references to object_type.
Sep 2 2021, 6:05 PM
ardumont added a comment to T2687: nixguix: Add support for downloads over FTP.

also i just realize i should have mentioned this earlier @KShivendu.
To reproduce the issue, no need for any loader or whatever else,
just ipython in your venv:

Sep 2 2021, 5:52 PM · Nixguix loader, Easy hack
ardumont added a comment to T2687: nixguix: Add support for downloads over FTP.

Possibly a composition of requests adapter [1] and urllib.request.urlretrieve [2] should
or could be enough instead.

Sep 2 2021, 5:48 PM · Nixguix loader, Easy hack
ardumont updated the task description for T3545: Update the journalbeat version package.
Sep 2 2021, 5:32 PM · Packagers, System administration
ardumont updated the task description for T3545: Update the journalbeat version package.
Sep 2 2021, 5:31 PM · Packagers, System administration
anlambert accepted D6174: vault: Rename downloaded files to replace ':' with '_' in SWHIDs.

Looks good to me.

Sep 2 2021, 5:28 PM
ardumont added a comment to T3545: Update the journalbeat version package.
17:17 <+ardumont> just out of curiosity, not that i'm on that right now
17:20 <+ardumont> on an elastic node, we'd get
17:20 <+ardumont> ardumont@esnode1:~% apt-cache show journalbeat | grep Version | awk '{print $2}' | sort -V -r | head -1
17:20 <+ardumont> 7.14.1
17:20 <+ardumont> we'd have some choice
17:22 <+olasd> ardumont: no objections. the main concern is that the upstream mapping from journalctl fields to elasticsearch documents has changed, so we'd need to adapt the (overall, very few) filters we have
17:22 <+olasd> and probably some dashboard churn
Sep 2 2021, 5:27 PM · Packagers, System administration
ardumont added a comment to T2687: nixguix: Add support for downloads over FTP.

This could possibly be done using https://pypi.org/project/requests-ftp/

Sep 2 2021, 5:26 PM · Nixguix loader, Easy hack
ardumont closed T3518: Enable vault cookers to access swh-graph, a subtask of T3505: Make the git-bare cooker available to the staff and beta-testers in the production webapp, as Resolved.
Sep 2 2021, 5:24 PM · Vault, Web app
ardumont closed T3518: Enable vault cookers to access swh-graph as Resolved.
Sep 2 2021, 5:24 PM · System administration, Vault
vlorentz requested review of D6174: vault: Rename downloaded files to replace ':' with '_' in SWHIDs.
Sep 2 2021, 5:23 PM
anlambert added a comment to T3489: Implement iframe view for content and directory elements.

@anlambert : Looks very good!
Can you show an example for a code fragment where you view a content with a few lines marked-up ?

Sep 2 2021, 5:22 PM · Software Stories, Web app
swh-public-ci added a comment to D6171: Switch the UI to use bare git archives, instead of gitfast..

Build is green

Sep 2 2021, 5:17 PM
anlambert added inline comments to D6171: Switch the UI to use bare git archives, instead of gitfast..
Sep 2 2021, 5:02 PM
vlorentz updated the diff for D6171: Switch the UI to use bare git archives, instead of gitfast..

undo change to dir tarball name

Sep 2 2021, 5:02 PM
vlorentz added inline comments to D6171: Switch the UI to use bare git archives, instead of gitfast..
Sep 2 2021, 5:01 PM
anlambert requested changes to D6171: Switch the UI to use bare git archives, instead of gitfast..

I found a couple of issues while testing, see inline comments.

Sep 2 2021, 4:42 PM
GitHub <noreply@github.com> committed R206:b32de84d31dd: feat(smtp): Add hostname to SMTP (#1076) (authored by Burak Yigit Kaya <byk@sentry.io>).
feat(smtp): Add hostname to SMTP (#1076)
Sep 2 2021, 4:40 PM
GitHub <noreply@github.com> committed R206:674a600770ac: fix(compose): Drop version from compose file (#1074) (authored by Burak Yigit Kaya <byk@sentry.io>).
fix(compose): Drop version from compose file (#1074)
Sep 2 2021, 4:40 PM
GitHub <noreply@github.com> committed R206:610f7136bc76: fix(healthcheck): Increase retries to 5 (#1072) (authored by Burak Yigit Kaya <byk@sentry.io>).
fix(healthcheck): Increase retries to 5 (#1072)
Sep 2 2021, 4:40 PM
GitHub <noreply@github.com> committed R206:59c0df393f8b: fix(clickhouse): Use correct HTTP port for healthcheck (#1069) (authored by Burak Yigit Kaya <byk@sentry.io>).
fix(clickhouse): Use correct HTTP port for healthcheck (#1069)
Sep 2 2021, 4:40 PM
GitHub <noreply@github.com> committed R206:7b96408734c9: fix(requirements): Make compose version check bw-compatible (#1068) (authored by Burak Yigit Kaya <byk@sentry.io>).
fix(requirements): Make compose version check bw-compatible (#1068)
Sep 2 2021, 4:40 PM
GitHub <noreply@github.com> committed R206:4b5ab5fb2119: docs(changelog): Add missing entries for docker-compose changes (#1071) (authored by Burak Yigit Kaya <byk@sentry.io>).
docs(changelog): Add missing entries for docker-compose changes (#1071)
Sep 2 2021, 4:40 PM
GitHub <noreply@github.com> committed R206:497759c1e832: ci: Test with the required minimum docker-compose (#1066) (authored by Burak Yigit Kaya <byk@sentry.io>).
ci: Test with the required minimum docker-compose (#1066)
Sep 2 2021, 4:40 PM
GitHub <noreply@github.com> committed R206:47c4a1df299f: meta(gha): Deploy action issue-routing-helper.yml (#1064) (authored by Chad Whitacre <chadwhitacre@sentry.io>).
meta(gha): Deploy action issue-routing-helper.yml (#1064)
Sep 2 2021, 4:40 PM
getsentry-bot <bot@getsentry.com> committed R206:42aa60e42fa3: build: Set master version to nightly (authored by getsentry-bot <bot@getsentry.com>).
build: Set master version to nightly
Sep 2 2021, 4:40 PM
GitHub <noreply@github.com> committed R206:672b1f7f4d22: feat: Have an empty certificates folder ready (#1065) (authored by Burak Yigit Kaya <byk@sentry.io>).
feat: Have an empty certificates folder ready (#1065)
Sep 2 2021, 4:40 PM
GitHub <noreply@github.com> committed R206:9aa34dd6120f: feat: Leverage health checks for depends_on (#1057) (authored by Burak Yigit Kaya <byk@sentry.io>).
feat: Leverage health checks for depends_on (#1057)
Sep 2 2021, 4:40 PM