Page MenuHomeSoftware Heritage
Feed All Stories

Jan 26 2022

anlambert closed D7043: loader: Fix if condition in SvnLoaderFromRemoteDump.dump_revisions.
Jan 26 2022, 4:26 PM
anlambert closed D7042: replay: Fix links restoration in DirEditor.remove_external_path.
Jan 26 2022, 4:26 PM
anlambert committed rDLDSVN2bbbd6f99f9c: replay: Fix links restoration in DirEditor.remove_external_path (authored by anlambert).
replay: Fix links restoration in DirEditor.remove_external_path
Jan 26 2022, 4:25 PM
anlambert committed rDLDSVNda1a81bbdb01: loader: Fix if condition in SvnLoaderFromRemoteDump.dump_revisions (authored by anlambert).
loader: Fix if condition in SvnLoaderFromRemoteDump.dump_revisions
Jan 26 2022, 4:25 PM
anlambert closed D7041: svn: Ignore externals parsing related errors when exporting.
Jan 26 2022, 4:25 PM
anlambert committed rDLDSVN34f61cb58157: svn: Ignore externals parsing related errors when exporting (authored by anlambert).
svn: Ignore externals parsing related errors when exporting
Jan 26 2022, 4:25 PM
ardumont accepted D7043: loader: Fix if condition in SvnLoaderFromRemoteDump.dump_revisions.
Jan 26 2022, 4:21 PM
ardumont accepted D7042: replay: Fix links restoration in DirEditor.remove_external_path.
Jan 26 2022, 4:20 PM
ardumont accepted D7041: svn: Ignore externals parsing related errors when exporting.
Jan 26 2022, 4:19 PM
ardumont updated the task description for T3891: Migrate sentry to admin vlan.
Jan 26 2022, 4:17 PM · System administration
ardumont updated the task description for T3891: Migrate sentry to admin vlan.
Jan 26 2022, 4:15 PM · System administration
anlambert updated the summary of D7043: loader: Fix if condition in SvnLoaderFromRemoteDump.dump_revisions.
Jan 26 2022, 3:54 PM
anlambert requested review of D7043: loader: Fix if condition in SvnLoaderFromRemoteDump.dump_revisions.
Jan 26 2022, 3:53 PM
douardda added inline comments to D7003: journal: Document the new format for gitdate..
Jan 26 2022, 3:50 PM
anlambert requested review of D7042: replay: Fix links restoration in DirEditor.remove_external_path.
Jan 26 2022, 3:38 PM
anlambert requested review of D7041: svn: Ignore externals parsing related errors when exporting.
Jan 26 2022, 3:33 PM
anlambert added a revision to T3875: Deploy loader.svn v1.1 and analyze remaining errors: D7041: svn: Ignore externals parsing related errors when exporting.
Jan 26 2022, 3:31 PM · System administration, SVN Loader
anlambert requested review of D7040: assets/vault: Trigger task creation when hitting enter key on modals.
Jan 26 2022, 3:28 PM
swh-sentry-integration added a comment to T3875: Deploy loader.svn v1.1 and analyze remaining errors.

Sentry issue: SWH-LOADER-SVN-54

Jan 26 2022, 3:28 PM · System administration, SVN Loader
anlambert renamed T3875: Deploy loader.svn v1.1 and analyze remaining errors from staging: Deploy loader.svn v1.1 to staging: Deploy loader.svn v1.1 and analyze remaining errors.
Jan 26 2022, 3:27 PM · System administration, SVN Loader
ardumont updated the task description for T3891: Migrate sentry to admin vlan.
Jan 26 2022, 3:10 PM · System administration
anlambert added a revision to T3845: Requesting cooking with email address + "Enter " key returns an error: D7040: assets/vault: Trigger task creation when hitting enter key on modals.
Jan 26 2022, 3:09 PM · Web app
ardumont updated the task description for T3891: Migrate sentry to admin vlan.
Jan 26 2022, 3:08 PM · System administration
ardumont updated the task description for T3891: Migrate sentry to admin vlan.
Jan 26 2022, 3:02 PM · System administration
vlorentz added inline comments to D7003: journal: Document the new format for gitdate..
Jan 26 2022, 3:02 PM
ardumont updated the task description for T3891: Migrate sentry to admin vlan.
Jan 26 2022, 2:51 PM · System administration
vlorentz created P1265 git smart strace.
Jan 26 2022, 2:42 PM
ardumont triaged T3891: Migrate sentry to admin vlan as Normal priority.
Jan 26 2022, 2:42 PM · System administration
douardda added a comment to D7039: Update the debian local package building section.
In D7039#183023, @olasd wrote:

Thanks.

--build-dep-resolver=aptitude should only be used when building with an extra-repository which has a non-default priority, that is only when using a -backports suite (so only for the bullseye and buster instructions). It should probably be documented in the list of "useful options" rather than as the default.

Jan 26 2022, 2:26 PM
douardda requested changes to D7003: journal: Document the new format for gitdate..
Jan 26 2022, 2:24 PM
douardda added inline comments to D7003: journal: Document the new format for gitdate..
Jan 26 2022, 2:23 PM
douardda added inline comments to D7003: journal: Document the new format for gitdate..
Jan 26 2022, 2:18 PM
anlambert closed D7034: common/origin_save: Fix extraction of elasticsearch document fields.
Jan 26 2022, 1:58 PM
anlambert committed rDWAPPS0733a8de5b10: common/origin_save: Fix extraction of elasticsearch document fields (authored by anlambert).
common/origin_save: Fix extraction of elasticsearch document fields
Jan 26 2022, 1:58 PM
swh-public-ci added a comment to D7034: common/origin_save: Fix extraction of elasticsearch document fields.

Build is green

Jan 26 2022, 1:55 PM
anlambert updated the diff for D7034: common/origin_save: Fix extraction of elasticsearch document fields.

Rebase

Jan 26 2022, 1:37 PM
anlambert committed rDWAPPS54abba37bde4: tests: Fix flaky test introduced in 7ea3aab33ed8 (authored by anlambert).
tests: Fix flaky test introduced in 7ea3aab33ed8
Jan 26 2022, 1:36 PM
olasd committed rDSNIP0455611094ee: recover_corrupt_objects: clean imports (authored by olasd).
recover_corrupt_objects: clean imports
Jan 26 2022, 1:02 PM
olasd committed rDSNIP5e20eee1b438: recover_corrupt_objects: use logging instead of tqdm (authored by olasd).
recover_corrupt_objects: use logging instead of tqdm
Jan 26 2022, 1:02 PM
olasd committed rDSNIP871da6ca0a09: recover_corrupt_objects: use stream_results instead of stream_results_optional (authored by olasd).
recover_corrupt_objects: use stream_results instead of stream_results_optional
Jan 26 2022, 1:02 PM
douardda requested review of D7039: Update the debian local package building section.
Jan 26 2022, 12:33 PM
vlorentz committed rDMODa0674ad802ea: Fix crash in check_entries. (authored by vlorentz).
Fix crash in check_entries.
Jan 26 2022, 12:17 PM
Harbormaster failed to build B26354: rDMOD7364adcd57b4: Fix f-string for rDMOD7364adcd57b4: Fix f-string!
Jan 26 2022, 12:09 PM
vlorentz committed rDMOD7364adcd57b4: Fix f-string (authored by vlorentz).
Fix f-string
Jan 26 2022, 12:08 PM
vlorentz triaged T3890: schema of timezones in the journal as Normal priority.
Jan 26 2022, 11:57 AM · Journal, Data Model
Harbormaster failed remote builds in B26352: Diff 25523 for D7034: common/origin_save: Fix extraction of elasticsearch document fields!
Jan 26 2022, 11:43 AM
swh-public-ci added a comment to D7034: common/origin_save: Fix extraction of elasticsearch document fields.

Build has FAILED

Jan 26 2022, 11:43 AM
vlorentz added inline comments to D7011: Revert "Restore 'offset' and 'negative_utc' arguments and make them optional".
Jan 26 2022, 11:37 AM
ardumont accepted D7011: Revert "Restore 'offset' and 'negative_utc' arguments and make them optional".

couple of question/suggestion inline.

Jan 26 2022, 11:34 AM
vlorentz closed D7029: Filter out extids targeting non-existing releases.
Jan 26 2022, 11:30 AM
vlorentz committed rDLDBASE4718713b9ae0: Filter out extids targeting non-existing releases (authored by vlorentz).
Filter out extids targeting non-existing releases
Jan 26 2022, 11:30 AM
anlambert updated the diff for D7034: common/origin_save: Fix extraction of elasticsearch document fields.

Rebase

Jan 26 2022, 11:22 AM
anlambert committed rDWAPPS7ea3aab33ed8: assets/highlightjs: Add new languages to highlight and fix some issues (authored by anlambert).
assets/highlightjs: Add new languages to highlight and fix some issues
Jan 26 2022, 11:22 AM
anlambert committed rDWAPPSed31490a3599: package.json: Upgrade dependencies (authored by anlambert).
package.json: Upgrade dependencies
Jan 26 2022, 11:22 AM
anlambert closed D7032: assets/highlightjs: Add new languages to highlight and fix some issues.
Jan 26 2022, 11:22 AM
anlambert committed rDWAPPSbce990970824: webpack/weblabels-plugin: Read package.json file only when required (authored by anlambert).
webpack/weblabels-plugin: Read package.json file only when required
Jan 26 2022, 11:22 AM
anlambert added inline comments to D7032: assets/highlightjs: Add new languages to highlight and fix some issues.
Jan 26 2022, 11:18 AM
ardumont accepted D7012: Remove deprecated property 'TimestampWithTimezone.offset'.
Jan 26 2022, 10:19 AM
dachary closed D7036: switch to sha256 for hashing & better read performances.
Jan 26 2022, 10:10 AM
dachary committed rDOBJS0b6bee78ab5c: switch to sha256 for hashing & better read performances (authored by dachary).
switch to sha256 for hashing & better read performances
Jan 26 2022, 10:10 AM
ardumont closed T3877: Automate the weekly-planning script as Resolved.
Jan 26 2022, 9:49 AM · System administration
ardumont accepted D7029: Filter out extids targeting non-existing releases.
Jan 26 2022, 9:46 AM
ardumont accepted D7036: switch to sha256 for hashing & better read performances.
Jan 26 2022, 9:34 AM
ardumont accepted D7038: ORC exporter: use ZST compression.
Jan 26 2022, 9:31 AM
ardumont accepted D7032: assets/highlightjs: Add new languages to highlight and fix some issues.

couple of question/suggestion inline.

Jan 26 2022, 9:31 AM
ardumont updated subscribers of T3746: staging: Deploy maven indexer/lister/loader.
  • a visit_type 'https' (record 3 below [1]).

*that* is fishy. I'm investigating.

Ok, got it: unluckily this one string is viciously malformed (it's wrong, but fits the
regexp anyway), and I had not considered this very case. Looking at the bright side,
this shows a tricky bug that will be fixed.

Jan 26 2022, 9:16 AM · Maven loader, Maven lister, System administration, Archive coverage
borisbaldassari added a comment to T3746: staging: Deploy maven indexer/lister/loader.
  • missing last_update entries in the task (I recalled we discussed and fixes those in the related diffs)
Jan 26 2022, 12:04 AM · Maven loader, Maven lister, System administration, Archive coverage

Jan 25 2022

vlorentz updated subscribers of T3527: Self-host Software Heritage on grid5000.

@vsellier already did this to benchmark cassandra. it's indeed necessary to see how the backends behave with real loader and vault workloads. (less so for the objstorage, since the workloads should be much more uniform)

Jan 25 2022, 11:30 PM · Object storage
borisbaldassari added a comment to T3746: staging: Deploy maven indexer/lister/loader.

Still, there is something fishy about:

  • a visit_type 'https' (record 3 below [1]).

*that* is fishy. I'm investigating.

Jan 25 2022, 10:40 PM · Maven loader, Maven lister, System administration, Archive coverage
dachary added a comment to T3527: Self-host Software Heritage on grid5000.

I'mt not exactly sure why I thought that would be necessary for benchmarking. In any case... it's not ;-)

Jan 25 2022, 9:53 PM · Object storage
dachary closed T3527: Self-host Software Heritage on grid5000, a subtask of T3432: Add winery backend, as Wontfix.
Jan 25 2022, 9:53 PM · Object storage
dachary closed T3527: Self-host Software Heritage on grid5000 as Wontfix.
Jan 25 2022, 9:53 PM · Object storage
dachary closed T3525: grid5000 tools and documentation, a subtask of T3432: Add winery backend, as Resolved.
Jan 25 2022, 9:52 PM · Object storage
dachary closed T3525: grid5000 tools and documentation as Resolved.
Jan 25 2022, 9:52 PM · Object storage
dachary added a comment to T3525: grid5000 tools and documentation.

The documentation is at:

Jan 25 2022, 9:52 PM · Object storage
dachary closed T3634: Create swh-perfecthash module, a subtask of T3104: Persistent readonly perfect hash table, as Resolved.
Jan 25 2022, 9:51 PM · Object storage (RedHat collaboration)
dachary closed T3634: Create swh-perfecthash module as Resolved.
Jan 25 2022, 9:51 PM · Object storage
dachary closed T3528: Add winery backend: grid5000 benchmark, a subtask of T3432: Add winery backend, as Resolved.
Jan 25 2022, 9:50 PM · Object storage
dachary closed T3528: Add winery backend: grid5000 benchmark as Resolved.
Jan 25 2022, 9:50 PM · Object storage
dachary added a comment to T3528: Add winery backend: grid5000 benchmark.

It's documented in the winery test environment and was actually able to use the instructions successfully (after a few fixes...). It does work an this can be closed as resolved.

Jan 25 2022, 9:50 PM · Object storage
dachary added a comment to T3432: Add winery backend.

Added a wiki page to be a more accessible version of the benchmark process than the README in the sources.

Jan 25 2022, 9:48 PM · Object storage
olasd committed rSPSITE89642e23c829: unattended_upgrades: Allow the new codename for debian-security (authored by olasd).
unattended_upgrades: Allow the new codename for debian-security
Jan 25 2022, 8:24 PM
seirl requested review of D7038: ORC exporter: use ZST compression.
Jan 25 2022, 7:54 PM
swh-public-ci added a comment to D7010: Add a command to generate a subdataset from a list of SWHIDs using S3.

Build is green

Jan 25 2022, 7:50 PM
seirl closed D7010: Add a command to generate a subdataset from a list of SWHIDs using S3.
Jan 25 2022, 7:48 PM
seirl committed rDDATASET027235d6d46d: Add a command to generate a subdataset from a list of SWHIDs using S3 (authored by seirl).
Add a command to generate a subdataset from a list of SWHIDs using S3
Jan 25 2022, 7:48 PM
seirl updated the diff for D7010: Add a command to generate a subdataset from a list of SWHIDs using S3.

Add special case for revisions, compress with ZST

Jan 25 2022, 7:48 PM
olasd committed rDSNIP99a3e4997191: recover_corrupt_objects: make assertions more useful (authored by olasd).
recover_corrupt_objects: make assertions more useful
Jan 25 2022, 6:27 PM
swh-public-ci added a comment to D7036: switch to sha256 for hashing & better read performances.

Build is green

Jan 25 2022, 6:13 PM
dachary updated the diff for D7036: switch to sha256 for hashing & better read performances.

Adjust the disk names on grid5000 to reflect changes at https://www.grid5000.fr/w/Grenoble:Hardware#dahu

Jan 25 2022, 6:10 PM
ardumont removed a project from T3874: staging: Analyze result of the maven listing and ingestion: System administration.
Jan 25 2022, 6:07 PM · Maven loader, Maven lister, Archive coverage
ardumont moved T3850: Migrate admin apps node to admin vlan from in-progress to Backlog on the System administration board.
Jan 25 2022, 6:06 PM · System administration
ardumont moved T3889: Admin database backup from Backlog to Weekly backlog on the System administration board.
Jan 25 2022, 6:06 PM · System administration
ardumont triaged T3889: Admin database backup as Normal priority.
Jan 25 2022, 6:06 PM · System administration
ardumont moved T3877: Automate the weekly-planning script from in-progress to deployed/landed/monitoring on the System administration board.
Jan 25 2022, 6:05 PM · System administration
vlorentz committed rDSNIPcb2ced00ae42: recover_corrupt_objects.py: Actually commit changes (authored by vlorentz).
recover_corrupt_objects.py: Actually commit changes
Jan 25 2022, 6:05 PM
ardumont closed D7035: Automate weekly-planning script.
Jan 25 2022, 5:52 PM
ardumont committed rSPSITEbd0bb835e693: Automate weekly-planning script (authored by ardumont).
Automate weekly-planning script
Jan 25 2022, 5:52 PM
ardumont updated the test plan for D7035: Automate weekly-planning script.
Jan 25 2022, 5:50 PM
ardumont added inline comments to D7035: Automate weekly-planning script.
Jan 25 2022, 5:46 PM