Page MenuHomeSoftware Heritage
Feed All Stories

Jan 25 2022

vsellier updated the task description for T3889: Admin database backup.
Jan 25 2022, 5:28 PM · System administration
vsellier created T3889: Admin database backup.
Jan 25 2022, 5:27 PM · System administration
ardumont added inline comments to D7035: Automate weekly-planning script.
Jan 25 2022, 5:23 PM
ardumont added inline comments to D7035: Automate weekly-planning script.
Jan 25 2022, 5:22 PM
ardumont updated the diff for D7035: Automate weekly-planning script.

Adapt what's possible from the last round of review
Thanks!

Jan 25 2022, 5:22 PM
ardumont updated the diff for D7035: Automate weekly-planning script.

Wrap command call with chronic

Jan 25 2022, 5:10 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO271a259ae50c: Updated backport on buster-swh from debian/0.42.0-1_swh1 (unstable-swh) (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Updated backport on buster-swh from debian/0.42.0-1_swh1 (unstable-swh)
Jan 25 2022, 5:08 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO8079f65a9e19: Merge tag 'debian/0.42.0-1_swh1' into debian/buster-swh (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Merge tag 'debian/0.42.0-1_swh1' into debian/buster-swh
Jan 25 2022, 5:08 PM
swh-public-ci added a comment to D6979: Document the mirror credentials management.

Build is green

Jan 25 2022, 5:07 PM
anlambert abandoned D7037: package.json: Upgrade dependencies.

Build passed on Jenkins, great ! Abandoning this, will land it later.

Jan 25 2022, 5:07 PM
vsellier closed D6979: Document the mirror credentials management.
Jan 25 2022, 5:05 PM
vsellier committed rDDOC4ac8648fb0a7: Document the mirror credentials management (authored by vsellier).
Document the mirror credentials management
Jan 25 2022, 5:05 PM
anlambert requested review of D7037: package.json: Upgrade dependencies.
Jan 25 2022, 5:02 PM
olasd added inline comments to D7035: Automate weekly-planning script.
Jan 25 2022, 5:00 PM
vsellier updated the diff for D6979: Document the mirror credentials management.
  • rebase
  • update according the feedback
Jan 25 2022, 4:57 PM
ardumont updated the diff for D7035: Automate weekly-planning script.

Make the script fail fast

Jan 25 2022, 4:57 PM
ardumont updated the diff for D7035: Automate weekly-planning script.

Adapt according to review

Jan 25 2022, 4:55 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO121b00c50361: pristine-tar data for swh-storage_0.42.0.orig.tar.gz (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
pristine-tar data for swh-storage_0.42.0.orig.tar.gz
Jan 25 2022, 4:54 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTOb752a68566ae: Updated debian changelog for version 0.42.0 (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Updated debian changelog for version 0.42.0
Jan 25 2022, 4:54 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTOc2a93a6f4dbf: Update upstream source from tag 'debian/upstream/0.42.0' (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Update upstream source from tag 'debian/upstream/0.42.0'
Jan 25 2022, 4:54 PM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTObedc3722881b: New upstream version 0.42.0 (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
New upstream version 0.42.0
Jan 25 2022, 4:54 PM
olasd committed rDSNIP27369c56cac9: recover_corrupt_objects: Remove spurious select (authored by olasd).
recover_corrupt_objects: Remove spurious select
Jan 25 2022, 4:52 PM
olasd committed rDSNIP44fe78774db2: recover_corrupt_objects: Pull config from a configfile instead of the cli (authored by olasd).
recover_corrupt_objects: Pull config from a configfile instead of the cli
Jan 25 2022, 4:52 PM
ardumont added inline comments to D7035: Automate weekly-planning script.
Jan 25 2022, 4:50 PM
dachary added a comment to D7033: the desired key len is 32 for sha256.
In D7033#182832, @olasd wrote:

Thanks! (and sorry for the hash algo ping-pong)

Jan 25 2022, 4:48 PM
anlambert requested review of D7032: assets/highlightjs: Add new languages to highlight and fix some issues.
Jan 25 2022, 4:40 PM
vsellier added inline comments to D6979: Document the mirror credentials management.
Jan 25 2022, 4:36 PM
vsellier accepted D7034: common/origin_save: Fix extraction of elasticsearch document fields.

LGTM

Jan 25 2022, 4:34 PM
vsellier accepted D7035: Automate weekly-planning script.

thanks, just a small inlined remark regarding the httpie package installation

Jan 25 2022, 4:28 PM
dachary requested review of D7036: switch to sha256 for hashing & better read performances.
Jan 25 2022, 4:20 PM
swh-public-ci added a comment to D7029: Filter out extids targeting non-existing releases.

Build is green

Jan 25 2022, 4:13 PM
anlambert requested review of D7034: common/origin_save: Fix extraction of elasticsearch document fields.
Jan 25 2022, 4:13 PM
vlorentz updated the diff for D7029: Filter out extids targeting non-existing releases.

add logging

Jan 25 2022, 4:11 PM
ardumont added a comment to T3877: Automate the weekly-planning script.

remove the 'scribe' attribution

Jan 25 2022, 4:09 PM · System administration
vlorentz added a comment to D7029: Filter out extids targeting non-existing releases.
In D7029#182839, @olasd wrote:

I wonder if that'd be worth a warning. May end up being a bit noisy though.

Jan 25 2022, 4:09 PM
ardumont updated the summary of D7035: Automate weekly-planning script.
Jan 25 2022, 4:04 PM
ardumont updated the test plan for D7035: Automate weekly-planning script.
Jan 25 2022, 4:03 PM
ardumont updated the diff for D7035: Automate weekly-planning script.

Actually send the email instead of just displaying it

Jan 25 2022, 4:02 PM
ardumont added a revision to T3877: Automate the weekly-planning script: D7035: Automate weekly-planning script.
Jan 25 2022, 4:02 PM · System administration
ardumont requested review of D7035: Automate weekly-planning script.
Jan 25 2022, 4:02 PM
olasd added a comment to D7029: Filter out extids targeting non-existing releases.

I wonder if that'd be worth a warning. May end up being a bit noisy though.

Jan 25 2022, 3:52 PM
dachary closed D7033: the desired key len is 32 for sha256.
Jan 25 2022, 3:48 PM
dachary committed rDOPH3b5a0b09c9d3: the desired key len is 32 for sha256 (authored by dachary).
the desired key len is 32 for sha256
Jan 25 2022, 3:48 PM
olasd accepted D7033: the desired key len is 32 for sha256.

Thanks! (and sorry for the hash algo ping-pong)

Jan 25 2022, 3:46 PM
dachary requested review of D7033: the desired key len is 32 for sha256.
Jan 25 2022, 3:37 PM
dachary closed D7031: the key has a fixed len: do not hardcode it.
Jan 25 2022, 3:24 PM
dachary committed rDOPHd79745c582c8: the key has a fixed len: do not hardcode it (authored by dachary).
the key has a fixed len: do not hardcode it
Jan 25 2022, 3:24 PM
vlorentz accepted D7031: the key has a fixed len: do not hardcode it.

I'd reword the commit message as "Deduplicate hardcoded key size"

Jan 25 2022, 3:23 PM
dachary requested review of D7031: the key has a fixed len: do not hardcode it.
Jan 25 2022, 3:20 PM
ardumont changed the status of T3877: Automate the weekly-planning script from Open to Work in Progress.
Jan 25 2022, 3:12 PM · System administration
vlorentz added projects to T3887: Storing multiple authors in Revisions and Releases: Data Model, SWORD deposit.
Jan 25 2022, 3:08 PM · SWORD deposit, Data Model, BZR loader
ardumont updated subscribers of T3887: Storing multiple authors in Revisions and Releases.

Heads up, we have some other tasks relating to this T3284 T1645.

Jan 25 2022, 3:03 PM · SWORD deposit, Data Model, BZR loader
ardumont added a comment to T3746: staging: Deploy maven indexer/lister/loader.

That sounds ok to me: scm entries are deduplicated. 2 pages corresponding to the same scm entry (i.e. with the same scm url) will produce only one output.
In this sample we should indeed have 2 jars and 1 scm.

Jan 25 2022, 2:59 PM · Maven loader, Maven lister, System administration, Archive coverage
ardumont updated the task description for T3864: staging: Deploy swh.loader.svn v1.0.0.
Jan 25 2022, 2:59 PM · System administration, SVN Loader
ardumont accepted D7003: journal: Document the new format for gitdate..
Jan 25 2022, 2:24 PM
Alphare added a comment to D6344: Non-incremental, non-delta Bazaar/Breezy loader.

I'm answering your questions now so you can get a change to come back to me before I start working on this again tomorrow.

Jan 25 2022, 2:23 PM
ardumont committed rDLSa599493b4846: maven: Let logging instruction do the formatting (authored by ardumont).
maven: Let logging instruction do the formatting
Jan 25 2022, 2:18 PM
ardumont closed D7025: maven: Fix logging instruction and add more debug ones.
Jan 25 2022, 2:18 PM
ardumont committed rDLS8667b04abcbb: maven: Add more debug logging instruction (authored by ardumont).
maven: Add more debug logging instruction
Jan 25 2022, 2:18 PM
Alphare added a task to D6344: Non-incremental, non-delta Bazaar/Breezy loader: T3888: Storing associated bugs.
Jan 25 2022, 2:16 PM
Alphare added a revision to T3888: Storing associated bugs: D6344: Non-incremental, non-delta Bazaar/Breezy loader.
Jan 25 2022, 2:16 PM · BZR loader
Alphare triaged T3888: Storing associated bugs as Normal priority.
Jan 25 2022, 2:16 PM · BZR loader
Alphare triaged T3887: Storing multiple authors in Revisions and Releases as Normal priority.
Jan 25 2022, 2:03 PM · SWORD deposit, Data Model, BZR loader
vlorentz committed rDSNIP3c8ad398dabf: recover_corrupt_objects.py: Remove print, cleanup import (authored by vlorentz).
recover_corrupt_objects.py: Remove print, cleanup import
Jan 25 2022, 2:02 PM
vlorentz committed rDSNIP219ca5b64358: recover_corrupt_objects.py: Double-check insertion (authored by vlorentz).
recover_corrupt_objects.py: Double-check insertion
Jan 25 2022, 1:52 PM
vlorentz closed D6957: Add recover_corrupt_objects.py.
Jan 25 2022, 1:52 PM
vlorentz committed rDSNIP7a7d0b300045: Add recover_corrupt_objects.py (authored by vlorentz).
Add recover_corrupt_objects.py
Jan 25 2022, 1:52 PM
olasd accepted D6957: Add recover_corrupt_objects.py.

I'm not 100% convinced we need to recheck the objects at every addition (within a transaction that can still fail to commit) instead of afterwards, but it doesn't /hurt/ either. We'll make a full pass on all objects again later anyway.

Jan 25 2022, 1:43 PM
vlorentz updated the task description for T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:35 PM · BZR loader, Data Model
vlorentz updated the task description for T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:35 PM · BZR loader, Data Model
zack triaged T3885: Filter rows of size >32MB from dataset export as Normal priority.
Jan 25 2022, 1:32 PM · Datasets
vlorentz updated the task description for T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:31 PM · BZR loader, Data Model
vlorentz updated the task description for T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:29 PM · BZR loader, Data Model
vlorentz updated subscribers of T3886: Representing timezones with non-integer number of minutes.
Jan 25 2022, 1:29 PM · BZR loader, Data Model
vlorentz added inline comments to D6344: Non-incremental, non-delta Bazaar/Breezy loader.
Jan 25 2022, 1:28 PM
vlorentz triaged T3886: Representing timezones with non-integer number of minutes as Normal priority.
Jan 25 2022, 1:28 PM · BZR loader, Data Model
vlorentz requested review of D7003: journal: Document the new format for gitdate..
Jan 25 2022, 1:15 PM
dachary closed D7030: implement statistics collection during benchmarks.
Jan 25 2022, 1:04 PM
dachary committed rDOBJS7b5538279be0: implement statistics collection during benchmarks (authored by dachary).
implement statistics collection during benchmarks
Jan 25 2022, 1:04 PM
vlorentz added inline comments to D6990: Add partial implementation of `ArchiveGraph` class.
Jan 25 2022, 1:04 PM
swh-public-ci added a comment to D7030: implement statistics collection during benchmarks.

Build is green

Jan 25 2022, 1:01 PM
vlorentz accepted D7030: implement statistics collection during benchmarks.

doesn't matplotlib work with the built-in tkinter by default?

Jan 25 2022, 1:00 PM
dachary added inline comments to D7030: implement statistics collection during benchmarks.
Jan 25 2022, 12:58 PM
dachary updated the diff for D7030: implement statistics collection during benchmarks.

address review comments

Jan 25 2022, 12:58 PM
ardumont closed T3873: Migrate inventory to the admin vlan as Resolved.
Jan 25 2022, 12:19 PM · System administration
ardumont closed T3873: Migrate inventory to the admin vlan, a subtask of T3850: Migrate admin apps node to admin vlan, as Resolved.
Jan 25 2022, 12:19 PM · System administration
ardumont updated the task description for T3873: Migrate inventory to the admin vlan.
Jan 25 2022, 12:18 PM · System administration
ardumont committed rSPRE5b4c9e4ac41e: Register bojimans in the admin vlan (authored by ardumont).
Register bojimans in the admin vlan
Jan 25 2022, 12:18 PM
vlorentz added inline comments to D7030: implement statistics collection during benchmarks.
Jan 25 2022, 12:04 PM
anlambert closed T3870: Analyze svn externals repository ingestion with loader.svn v1.0, a subtask of T3864: staging: Deploy swh.loader.svn v1.0.0, as Resolved.
Jan 25 2022, 11:50 AM · System administration, SVN Loader
anlambert closed T3870: Analyze svn externals repository ingestion with loader.svn v1.0 as Resolved.

swh-loader-svn v1.1.0 has been released, new tests have been scheduled on staging (T3875#77706).

Jan 25 2022, 11:50 AM · SVN Loader
anlambert added a comment to T3875: Deploy loader.svn v1.1 and analyze remaining errors.

Subversion origins that failed to load or with a partial visit status when using swh-loader-svn v1.0.0 (T3870#77580)
have been resubmitted on staging to check how many can now be fully loaded using swh-loader-svn v1.1.0.

Jan 25 2022, 11:49 AM · System administration, SVN Loader
vlorentz added a comment to D6344: Non-incremental, non-delta Bazaar/Breezy loader.

Looks good. I trust you for the bzr part, so I didn't check it

Jan 25 2022, 11:42 AM
swh-public-ci added a comment to D7025: maven: Fix logging instruction and add more debug ones.

Build is green

Jan 25 2022, 11:35 AM
ardumont updated the diff for D7025: maven: Fix logging instruction and add more debug ones.

Rebase

Jan 25 2022, 11:32 AM
ardumont renamed T3875: Deploy loader.svn v1.1 and analyze remaining errors from Deploy loader.svn v1.1 to staging: Deploy loader.svn v1.1.
Jan 25 2022, 11:29 AM · System administration, SVN Loader
ardumont added a comment to T3875: Deploy loader.svn v1.1 and analyze remaining errors.

staging:

  • Package python3-swh.loader.svn upgraded
  • Service swh-worker@loader_{svn,high_priority} restarted
Jan 25 2022, 11:29 AM · System administration, SVN Loader
ardumont moved T3875: Deploy loader.svn v1.1 and analyze remaining errors from in-progress to deployed/landed/monitoring on the System administration board.
Jan 25 2022, 11:28 AM · System administration, SVN Loader
swh-public-ci added a comment to D7030: implement statistics collection during benchmarks.

Build is green

Jan 25 2022, 11:27 AM
ardumont updated the task description for T3873: Migrate inventory to the admin vlan.
Jan 25 2022, 11:27 AM · System administration