Page MenuHomeSoftware Heritage
Feed All Stories

Sep 9 2021

aeviso accepted D6220: Added test only method info in the interface doc strings.

Not sure we really want to restrict this methods to test-only in the documentation. But we can change this in the future anyway

Sep 9 2021, 10:32 AM
ardumont updated the diff for D6199: Install graph services as-is.

Rebase

Sep 9 2021, 10:31 AM
olasd requested review of D6221: Update common configs from swh-py-template.
Sep 9 2021, 10:30 AM
ardumont accepted D6219: common/utils: Add function to get origin visit types dynamically.
Sep 9 2021, 10:13 AM
douardda accepted D6215: docker/conf: Fix search journal client configurations.
Sep 9 2021, 10:11 AM
olasd committed rCJSWHdf0aec4cf7fb: Add swh.dataset jobs (authored by olasd).
Add swh.dataset jobs
Sep 9 2021, 10:09 AM
moranegg updated the task description for T3482: Accompany design of the SWH stories data model.
Sep 9 2021, 10:00 AM · Acquisition Process (SWHAP), Software Stories
moranegg added inline comments to D6201: Add an overview of the metadata workflow.
Sep 9 2021, 9:48 AM
vsellier added a comment to D6139: cassandra: Add option to select (hopefully) more efficient batch insertion algos.

Thanks for the last fix, it looks better with a smaller batch size:
5 nodes:


The ingestion is ~7500 ops/s in batch compared to ~6500 before

Sep 9 2021, 9:13 AM
douardda requested changes to D6220: Added test only method info in the interface doc strings.

Please use imperative style in the got commit message
https://chris.beams.io/posts/git-commit/

Sep 9 2021, 9:03 AM
jayeshv added a reviewer for D6220: Added test only method info in the interface doc strings: aeviso.
Sep 9 2021, 9:00 AM
jayeshv requested review of D6220: Added test only method info in the interface doc strings.
Sep 9 2021, 8:49 AM

Sep 8 2021

vlorentz updated the diff for D6201: Add an overview of the metadata workflow.

apply @moranegg's comments

Sep 8 2021, 11:48 PM
olasd accepted D6209: Remove locks from PostgreSQL backend insertions.
Sep 8 2021, 6:35 PM
anlambert requested review of D6219: common/utils: Add function to get origin visit types dynamically.
Sep 8 2021, 5:55 PM
ardumont accepted D6215: docker/conf: Fix search journal client configurations.
Sep 8 2021, 5:43 PM
anlambert added a revision to T3441: Implement query to get origin visit types dynamically: D6219: common/utils: Add function to get origin visit types dynamically.
Sep 8 2021, 5:42 PM · Archive search
anlambert requested changes to D6218: directory-display: Show human-readable file sizes (KB, MB, ...)..

Looks good but filesizeformat is already used in many views implementation (grep for it).

Sep 8 2021, 5:41 PM
ardumont accepted D6218: directory-display: Show human-readable file sizes (KB, MB, ...)..
Sep 8 2021, 5:37 PM
vlorentz requested review of D6218: directory-display: Show human-readable file sizes (KB, MB, ...)..
Sep 8 2021, 5:35 PM
anlambert closed D6186: misc: Add iframe view for contents and directories.
Sep 8 2021, 5:34 PM
anlambert committed rDWAPPSe2b48563ac20: misc: Add iframe view for contents and directories (authored by anlambert).
misc: Add iframe view for contents and directories
Sep 8 2021, 5:34 PM
marla.dasilva added a comment to T3536: Blog post Easter Eggs NLNet.

Loïc m'a répondu : Ce n'est pas encore fait (le contrat) : le process est très long... nous sommes à la phase ou il s'agit de négocier les montants et le workplan. Le principe du financement est définitivement acté cependant: cela ne sera pas remis en cause (ouf).

Sep 8 2021, 5:24 PM · Unknown Object (Project)
swh-public-ci added a comment to D6186: misc: Add iframe view for contents and directories.

Build is green

Sep 8 2021, 5:15 PM
vlorentz accepted D6186: misc: Add iframe view for contents and directories.

thanks!

Sep 8 2021, 5:15 PM
vlorentz requested review of D6217: translator: Fix parsing of multibyte characters.
Sep 8 2021, 5:14 PM
vlorentz requested review of D6216: utils: Fix unescape() on non-ASCII strings..
Sep 8 2021, 5:13 PM
marla.dasilva closed T3534: Update team page with John Hsieh as Resolved.
Sep 8 2021, 5:12 PM · Unknown Object (Project)
marla.dasilva added a comment to T3534: Update team page with John Hsieh.

Publication made:

Sep 8 2021, 5:12 PM · Unknown Object (Project)
anlambert updated the diff for D6186: misc: Add iframe view for contents and directories.

Update:

  • rebase
  • ensure click handler for code highlighting will not be created in iframe mode
Sep 8 2021, 5:00 PM
anlambert requested review of D6215: docker/conf: Fix search journal client configurations.
Sep 8 2021, 4:42 PM
swh-public-ci added a comment to D6209: Remove locks from PostgreSQL backend insertions.

Build is green

Sep 8 2021, 4:42 PM
aeviso updated the diff for D6209: Remove locks from PostgreSQL backend insertions.
  • Remove autocommit isolation level from ProvenanceStoragePostgreSql
Sep 8 2021, 4:39 PM
vlorentz changed the status of T3466: Enable Sentry for the deposit server from Resolved to Invalid.
Sep 8 2021, 4:38 PM · System administration, Sentry, SWORD deposit
anlambert added a comment to D6186: misc: Add iframe view for contents and directories.

I disabled lines highlighting for the iframe view through javascript, did you recompile the static assets before your tests ?

Yes (I have to, the python process crashes when I don't)

I just tried, and it seems I can't anymore... most of the time.

However, if I click on a line number right after it displayed the column, I can still one line and it will scroll to it.

Sep 8 2021, 4:34 PM
ardumont changed the status of T3192: Add possibility to fetch last SWHID for a deposit using an origin on deposit cli, a subtask of T2996: Add possibility to fetch a list of deposits on the deposit cli, from Open to Work in Progress.
Sep 8 2021, 4:31 PM · SWORD deposit
ardumont changed the status of T3192: Add possibility to fetch last SWHID for a deposit using an origin on deposit cli from Open to Work in Progress.
Sep 8 2021, 4:31 PM · Easy hack, SWORD deposit
ardumont changed the status of T3466: Enable Sentry for the deposit server from Invalid to Resolved.
Sep 8 2021, 4:31 PM · System administration, Sentry, SWORD deposit
ardumont added a comment to T3192: Add possibility to fetch last SWHID for a deposit using an origin on deposit cli.

@moranegg I've went the script way in the end (what val initially proposed).

Sep 8 2021, 4:28 PM · Easy hack, SWORD deposit
vlorentz added a comment to D6186: misc: Add iframe view for contents and directories.

I disabled lines highlighting for the iframe view through javascript, did you recompile the static assets before your tests ?

Yes (I have to, the python process crashes when I don't)

Sep 8 2021, 4:20 PM
vlorentz closed D6190: Add link to extrinsic metadata API from the browse view.
Sep 8 2021, 4:14 PM
vlorentz committed rDWAPPSec19895012ab: Add link to extrinsic metadata API from the browse view (authored by vlorentz).
Add link to extrinsic metadata API from the browse view
Sep 8 2021, 4:14 PM
swh-public-ci added a comment to D6190: Add link to extrinsic metadata API from the browse view.

Build is green

Sep 8 2021, 4:13 PM
Harbormaster failed remote builds in B23439: Diff 22489 for D6190: Add link to extrinsic metadata API from the browse view!
Sep 8 2021, 4:12 PM
swh-public-ci added a comment to D6190: Add link to extrinsic metadata API from the browse view.

Build has FAILED

Sep 8 2021, 4:12 PM
ardumont committed rDSNIP5002184516fb: Compute the last swhid of an origin (authored by ardumont).
Compute the last swhid of an origin
Sep 8 2021, 4:02 PM
vlorentz updated the diff for D6190: Add link to extrinsic metadata API from the browse view.

rebase

Sep 8 2021, 3:59 PM
anlambert accepted D6190: Add link to extrinsic metadata API from the browse view.
Sep 8 2021, 3:58 PM
vlorentz added inline comments to D6190: Add link to extrinsic metadata API from the browse view.
Sep 8 2021, 3:56 PM
vlorentz updated the diff for D6190: Add link to extrinsic metadata API from the browse view.

fix label and add missing </div>

Sep 8 2021, 3:56 PM
swh-public-ci added a comment to D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI.

Build is green

Sep 8 2021, 3:53 PM
vlorentz requested review of D6214: keycloak: Declare 'swh.vault.git_bare.ui' role.
Sep 8 2021, 3:43 PM
vlorentz removed a revision from T3505: Make the git-bare cooker available to the staff and beta-testers in the production webapp: D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI.
Sep 8 2021, 3:41 PM · Vault, Web app
vlorentz added a revision to T3565: Add an option in swh-web to allow git-bare cooking via the UI for beta-testers: D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI.
Sep 8 2021, 3:41 PM · Vault, Web app
vlorentz edited tasks for D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI, added: T3565: Add an option in swh-web to allow git-bare cooking via the UI for beta-testers; removed T3505: Make the git-bare cooker available to the staff and beta-testers in the production webapp.
Sep 8 2021, 3:41 PM
vlorentz triaged T3565: Add an option in swh-web to allow git-bare cooking via the UI for beta-testers as Normal priority.
Sep 8 2021, 3:41 PM · Vault, Web app
anlambert requested review of D6213: templates/layout: Refine piwik script inclusion test.
Sep 8 2021, 3:41 PM
vlorentz committed rDENV7301df0e4092: keycloak: Declare 'swh.vault.git_bare.ui' role (authored by vlorentz).
keycloak: Declare 'swh.vault.git_bare.ui' role
Sep 8 2021, 3:40 PM
vlorentz reopened T3505: Make the git-bare cooker available to the staff and beta-testers in the production webapp as "Work in Progress".
Sep 8 2021, 3:39 PM · Vault, Web app
vlorentz reopened T3505: Make the git-bare cooker available to the staff and beta-testers in the production webapp, a subtask of T3504: Make the git-bare cooker publicly available, as Work in Progress.
Sep 8 2021, 3:39 PM · Vault, Web app
vlorentz added a task to D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI: T3505: Make the git-bare cooker available to the staff and beta-testers in the production webapp.
Sep 8 2021, 3:39 PM
vlorentz added a revision to T3505: Make the git-bare cooker available to the staff and beta-testers in the production webapp: D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI.
Sep 8 2021, 3:39 PM · Vault, Web app
vlorentz renamed T3505: Make the git-bare cooker available to the staff and beta-testers in the production webapp from Make the git-bare cooker available to the staff in the production webapp to Make the git-bare cooker available to the staff and beta-testers in the production webapp.
Sep 8 2021, 3:39 PM · Vault, Web app
vlorentz closed D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI.
Sep 8 2021, 3:38 PM
vlorentz committed rDWAPPS95440eca9997: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI (authored by vlorentz).
Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI
Sep 8 2021, 3:38 PM
vlorentz updated the diff for D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI.

rebase

Sep 8 2021, 3:38 PM
vlorentz added a task to D6190: Add link to extrinsic metadata API from the browse view: T3376: Visualize metadata of a deposit in the admin (moderation) view.
Sep 8 2021, 3:36 PM
vlorentz added a revision to T3376: Visualize metadata of a deposit in the admin (moderation) view: D6190: Add link to extrinsic metadata API from the browse view.
Sep 8 2021, 3:36 PM · Monitoring, SWORD deposit, Web app
vlorentz closed T2590: Finish the indexer -> swh-search pipeline, a subtask of T1117: Origin search is *slow* when you look for very common words, as Resolved.
Sep 8 2021, 3:35 PM · Web app, Storage manager
vlorentz closed T2590: Finish the indexer -> swh-search pipeline, a subtask of T2182: Switch production swh-web to use swh-search instead of postgresql search., as Resolved.
Sep 8 2021, 3:35 PM · System administration, Archive search, Storage manager
vlorentz closed T2590: Finish the indexer -> swh-search pipeline as Resolved.
Sep 8 2021, 3:35 PM · Journal, Archive search
vsellier closed T3040: [production] Enable swh-search's journal-client for indexed objects, a subtask of T2590: Finish the indexer -> swh-search pipeline, as Resolved.
Sep 8 2021, 3:24 PM · Journal, Archive search
vsellier closed T3040: [production] Enable swh-search's journal-client for indexed objects as Resolved.

metadata searches are now done in Elasticsearch since the deployment of T3433

Sep 8 2021, 3:24 PM · System administration, Journal, Archive search
vsellier renamed T3433: Deploy swh.search v0.10/v0.11 from Deploy swh.search v0.10/v0.11 on staging to Deploy swh.search v0.10/v0.11.
Sep 8 2021, 3:21 PM · System administration, Archive search
vsellier closed T3433: Deploy swh.search v0.10/v0.11 as Resolved.

Everything is deployed and look functional.

Sep 8 2021, 3:21 PM · System administration, Archive search
anlambert accepted D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI.

Looks good to me.

Sep 8 2021, 3:18 PM
vlorentz requested review of D6212: Add 'swh.vault.git_bare.ui' use role, to display the 'git bare' button on the UI.
Sep 8 2021, 3:12 PM
vlorentz committed rDENVb29a1367e0ac: nginx: Pass the right Host header to swh-web and swh-deposit (authored by vlorentz).
nginx: Pass the right Host header to swh-web and swh-deposit
Sep 8 2021, 2:52 PM
ardumont added a comment to D6206: webapp: support new metadata search backend configuation.

I gather from this that the actual configuration structure changed as well.

Sep 8 2021, 2:30 PM
vsellier closed D6206: webapp: support new metadata search backend configuation.
Sep 8 2021, 2:29 PM
vsellier committed rSPSITEd19dc2f55c01: webapp: support new metadata search backend configuation (authored by vsellier).
webapp: support new metadata search backend configuation
Sep 8 2021, 2:29 PM
vsellier accepted D6199: Install graph services as-is.

LGTM

Sep 8 2021, 2:22 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDWCLI9685b9a50d05: Merge tag 'debian/0.5.0-1_swh2' into debian/buster-swh (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Merge tag 'debian/0.5.0-1_swh2' into debian/buster-swh
Sep 8 2021, 2:21 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDWCLI33706989e582: Updated backport on buster-swh from debian/0.5.0-1_swh2 (unstable-swh) (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Updated backport on buster-swh from debian/0.5.0-1_swh2 (unstable-swh)
Sep 8 2021, 2:21 PM
anlambert added a comment to T3458: save code now: Requests are not getting updated from time to time.

Indeed some save requests are still marked as running while they clearly are not.

Sep 8 2021, 2:21 PM · Save Code Now
ardumont committed rDWCLI34a2acdefc8f: d/control: Update deps (authored by ardumont).
d/control: Update deps
Sep 8 2021, 2:20 PM
ardumont committed rDWCLIc4b7a833e00f: d/changelog: Bump new release (authored by ardumont).
d/changelog: Bump new release
Sep 8 2021, 2:20 PM
vsellier accepted D6200: Add icinga checks around the graph service.

LGTM

Sep 8 2021, 2:18 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDWCLIcde564eb4407: pristine-tar data for swh-web-client_0.5.0.orig.tar.gz (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
pristine-tar data for swh-web-client_0.5.0.orig.tar.gz
Sep 8 2021, 1:34 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDWCLI7cef6f11859e: Updated debian changelog for version 0.5.0 (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Updated debian changelog for version 0.5.0
Sep 8 2021, 1:34 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDWCLI1e31b4b35cc8: New upstream version 0.5.0 (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
New upstream version 0.5.0
Sep 8 2021, 1:34 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDWCLI71eb9a4a584f: Update upstream source from tag 'debian/upstream/0.5.0' (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Update upstream source from tag 'debian/upstream/0.5.0'
Sep 8 2021, 1:34 PM
ardumont closed D6211: client: Open last_visit method to retrieve last visit of an origin.
Sep 8 2021, 1:31 PM
ardumont committed rDWCLI32b05c4581ea: client: Open last_visit method to retrieve last visit of an origin (authored by ardumont).
client: Open last_visit method to retrieve last visit of an origin
Sep 8 2021, 1:31 PM
ardumont updated the summary of D6211: client: Open last_visit method to retrieve last visit of an origin.
Sep 8 2021, 1:30 PM
anlambert accepted D6211: client: Open last_visit method to retrieve last visit of an origin.

Looks good to me.

Sep 8 2021, 1:29 PM
ardumont abandoned D6210: wip: collection api: Open filtering deposit per user and origin_url.
Sep 8 2021, 1:26 PM
ardumont accepted D6206: webapp: support new metadata search backend configuation.

I gather from this that the actual configuration structure changed as well.

Sep 8 2021, 1:22 PM
ardumont updated the summary of D6211: client: Open last_visit method to retrieve last visit of an origin.
Sep 8 2021, 1:18 PM
swh-public-ci added a comment to D6211: client: Open last_visit method to retrieve last visit of an origin.

Build is green

Sep 8 2021, 1:13 PM