Page MenuHomeSoftware Heritage
Feed All Stories

Nov 26 2021

seirl closed D6676: Move SWHID<->node ID conversion in the Java backend.
Nov 26 2021, 5:05 PM
seirl committed rDGRPH0b33cff0d228: Move SWHID<->node ID conversion in the Java backend (authored by seirl).
Move SWHID<->node ID conversion in the Java backend
Nov 26 2021, 5:05 PM
vlorentz accepted D6676: Move SWHID<->node ID conversion in the Java backend.
Nov 26 2021, 4:55 PM
swh-public-ci added a comment to D6339: Add support for remote backend on existing storage tests.

Build is green

Nov 26 2021, 4:44 PM
vlorentz claimed T3594: Faithfully store weird git objects.
Nov 26 2021, 4:43 PM · meta-task, Data Model, Storage manager
vlorentz claimed T3753: Store original git manifests.
Nov 26 2021, 4:43 PM · Data Model, Storage manager
vlorentz triaged T3753: Store original git manifests as Normal priority.
Nov 26 2021, 4:43 PM · Data Model, Storage manager
vlorentz triaged T3752: Store/represent time offsets as strings as Normal priority.
Nov 26 2021, 4:42 PM · Data Model, Storage manager
vlorentz closed T3598: Support revisions with "extra headers" not at the end, a subtask of T3594: Faithfully store weird git objects, as Wontfix.
Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
vlorentz closed T3598: Support revisions with "extra headers" not at the end as Wontfix.

We decided to store manifests instead. T3594#74385

Nov 26 2021, 4:41 PM · Data Model, Storage manager
vlorentz closed T3596: Support "weird" permissions in directories as Wontfix.

We decided to store manifests instead. T3594#74385

Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
vlorentz closed T3596: Support "weird" permissions in directories, a subtask of T3594: Faithfully store weird git objects, as Wontfix.
Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
vlorentz closed T3595: Support disordered directory entries in git, a subtask of T3594: Faithfully store weird git objects, as Wontfix.
Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
vlorentz closed T3595: Support disordered directory entries in git as Wontfix.

We decided to store manifests instead. T3594#74385

Nov 26 2021, 4:41 PM · meta-task, Data Model, Storage manager
swh-public-ci added a comment to D6165: Add new RabbitMQ-based client/server API.

Build is green

Nov 26 2021, 4:41 PM
vlorentz added a comment to T3594: Faithfully store weird git objects.

Copy of an email I sent today:

Nov 26 2021, 4:40 PM · meta-task, Data Model, Storage manager
aeviso updated the diff for D6339: Add support for remote backend on existing storage tests.

rebase

Nov 26 2021, 4:38 PM
aeviso updated the diff for D6165: Add new RabbitMQ-based client/server API.

squash commits

Nov 26 2021, 4:38 PM
swh-public-ci added a comment to D6676: Move SWHID<->node ID conversion in the Java backend.

Build is green

Nov 26 2021, 4:35 PM
seirl updated the diff for D6676: Move SWHID<->node ID conversion in the Java backend.

Fix src/dst inversion, add regression test

Nov 26 2021, 4:33 PM
vlorentz accepted D6678: attempt to avoid content differences due to paths in keywords.

Ok. Make sure you document it (in docs/), so we don't forget. It might be useful to add some rsync:// tests, but that's out of scope for this diff

Nov 26 2021, 4:33 PM
swh-public-ci added a comment to D6339: Add support for remote backend on existing storage tests.

Build is green

Nov 26 2021, 4:30 PM
swh-public-ci added a comment to D6165: Add new RabbitMQ-based client/server API.

Build is green

Nov 26 2021, 4:26 PM
aeviso updated the diff for D6339: Add support for remote backend on existing storage tests.

rebase

Nov 26 2021, 4:25 PM
aeviso updated the diff for D6165: Add new RabbitMQ-based client/server API.
  • Add documentation for the remote storage backend
Nov 26 2021, 4:23 PM
vlorentz added a comment to T3638: Make package loaders create releases objects instead of revisions.

Copy of an email I sent on 2021-11-17:

Nov 26 2021, 4:02 PM · Package Loader, Data Model, Archive content
olasd added a comment to D6692: Rework the retry and reporting system in replay.py.

Could you make sure that this dashboard https://grafana.softwareheritage.org/d/d3l2oqXWz/s3-object-copy?orgId=1 is not affected (or that its functionality can be replaced easily?)

Nov 26 2021, 3:54 PM
olasd requested changes to D6692: Rework the retry and reporting system in replay.py.

You seem to have dropped the CONTENT_BYTES_METRIC ?

Nov 26 2021, 3:53 PM
vlorentz added a task to D6281: converters: Recompute hashes and check they match the originals: T399: (Re-)Compute data checksums before insertion.
Nov 26 2021, 3:52 PM
vlorentz added a revision to T399: (Re-)Compute data checksums before insertion: D6281: converters: Recompute hashes and check they match the originals.
Nov 26 2021, 3:52 PM · Storage manager
vlorentz accepted D6692: Rework the retry and reporting system in replay.py.

Good idea!

Nov 26 2021, 3:44 PM
swh-public-ci added a comment to D6678: attempt to avoid content differences due to paths in keywords.

Build is green

Nov 26 2021, 3:35 PM
stsp added a comment to D6678: attempt to avoid content differences due to paths in keywords.
In D6678#174111, @stsp wrote:
  • add a test for conversion of a file which contains a Header keyword
Nov 26 2021, 3:35 PM
stsp updated the diff for D6678: attempt to avoid content differences due to paths in keywords.
  • add a test for conversion of a file which contains a Header keyword
Nov 26 2021, 3:34 PM
jayeshv added a revision to T3608: Deprecate most of the /browse/origin/.* URLs: D6695: [WIP] Deprecate /origin/directrory and /snapshot/directory routes.
Nov 26 2021, 3:24 PM · Web app
jayeshv added a revision to T3698: Deprecate origin directory routes: D6695: [WIP] Deprecate /origin/directrory and /snapshot/directory routes.
Nov 26 2021, 3:24 PM · Web app
vlorentz updated the task description for T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.
Nov 26 2021, 3:16 PM · Roadmap 2020, Data Model, Web app, Roadmap 2021
jayeshv closed D6694: Fix for snapshot content redirection.
Nov 26 2021, 3:12 PM
jayeshv committed rDWAPPS806623eb53f9: Fix for snapshot content redirection (authored by jayeshv).
Fix for snapshot content redirection
Nov 26 2021, 3:12 PM
anlambert accepted D6694: Fix for snapshot content redirection.
Nov 26 2021, 3:10 PM
jayeshv requested review of D6694: Fix for snapshot content redirection.
Nov 26 2021, 3:05 PM
stsp added a comment to D6678: attempt to avoid content differences due to paths in keywords.
In D6678#174093, @stsp wrote:

And since our tests control the absolute paths to the test repositories they also control the expanded value of keywords like $Header$.
This means the tests would pass even without this fix in place. So I am not sure such a test would really be meaningful.

Nov 26 2021, 2:55 PM
stsp added a comment to D6678: attempt to avoid content differences due to paths in keywords.
In D6678#174093, @stsp wrote:

This means the tests would pass even without this fix in place. So I am not sure such a test would really be meaningful.

Nov 26 2021, 2:53 PM
stsp added a comment to D6678: attempt to avoid content differences due to paths in keywords.

wow...

Good catch, thanks. Could you add a test for this (so we don't accidentally change it if we ever refactor the loader in the future)?

Nov 26 2021, 2:50 PM
ardumont added inline comments to D6396: Implement maven jar source files loader.
Nov 26 2021, 2:47 PM
ardumont added a comment to D6395: lister: Add new maven lister.

Thanks for the update.

Nov 26 2021, 2:42 PM
vsellier closed T3750: Upgrade ELK stack to bullseye, a subtask of T3579: Meta-task: upgrade infrastructure to Debian Bullseye, as Resolved.
Nov 26 2021, 1:58 PM · System administration (Component upgrades)
vsellier closed T3750: Upgrade ELK stack to bullseye as Resolved.

All the nodes are updated

Nov 26 2021, 1:58 PM · System administration (Component upgrades)
vlorentz added a comment to D6678: attempt to avoid content differences due to paths in keywords.

Good catch, thanks. Could you add a test for this (so we don't accidentally change it if we ever refactor the loader in the future)?

Nov 26 2021, 1:53 PM
seirl added inline comments to D6676: Move SWHID<->node ID conversion in the Java backend.
Nov 26 2021, 1:47 PM
vlorentz closed D6675: git_bare: Fix and expand documentation.
Nov 26 2021, 1:39 PM
vlorentz committed rDVAU4d2d0dfdf8dc: git_bare: Fix and expand documentation (authored by vlorentz).
git_bare: Fix and expand documentation
Nov 26 2021, 1:39 PM
vlorentz closed D6673: Add an introduction to the vault documentation.
Nov 26 2021, 1:39 PM
vlorentz committed rDVAU35e4fd7378b7: Add an introduction to the vault documentation (authored by vlorentz).
Add an introduction to the vault documentation
Nov 26 2021, 1:39 PM
vlorentz closed D6671: git_bare: Fix counter of processed objects.
Nov 26 2021, 1:39 PM
vlorentz committed rDVAU2683d573a84d: git_bare: Fix counter of processed objects (authored by vlorentz).
git_bare: Fix counter of processed objects
Nov 26 2021, 1:39 PM
vlorentz added inline comments to D6675: git_bare: Fix and expand documentation.
Nov 26 2021, 1:38 PM
anlambert added a comment to D6691: common/archive: Add lookup_origin_snapshots function.

Where is it called from?

Nov 26 2021, 1:37 PM
vlorentz added inline comments to D6676: Move SWHID<->node ID conversion in the Java backend.
Nov 26 2021, 1:36 PM
douardda requested review of D6693: Add support for a redis-based reporter for failed replayed objects.
Nov 26 2021, 1:35 PM
douardda requested review of D6692: Rework the retry and reporting system in replay.py.
Nov 26 2021, 1:34 PM
douardda added a revision to T3693: Provide a mecanism to report (with persistence) objects that fails to get replayed (mirror): D6693: Add support for a redis-based reporter for failed replayed objects.
Nov 26 2021, 1:33 PM · Storage manager
vlorentz added a comment to D6691: common/archive: Add lookup_origin_snapshots function.

Where is it called from?

Nov 26 2021, 1:32 PM
rdicosmo committed rMSLDae7404bb7669: Update sponsor image with CEA (authored by rdicosmo).
Update sponsor image with CEA
Nov 26 2021, 1:19 PM
Harbormaster failed remote builds in B25185: Diff 24311 for D6395: lister: Add new maven lister!
Nov 26 2021, 11:13 AM
swh-public-ci added a comment to D6395: lister: Add new maven lister.

Build has FAILED

Nov 26 2021, 11:12 AM
borisbaldassari updated the diff for D6395: lister: Add new maven lister.

Make repo.time an iso8601 string

Nov 26 2021, 11:12 AM
ardumont created P1225 tryout.
Nov 26 2021, 10:40 AM
ardumont created P1224 puppet agent --test on workers complain about journalbeat.
Nov 26 2021, 10:20 AM
ardumont committed rSPSITE349d6cdd3e71: high_priority: Make celery workers fork more often (authored by ardumont).
high_priority: Make celery workers fork more often
Nov 26 2021, 10:11 AM
ardumont reassigned T3747: svn: FileExistsError when creating directory from ardumont to anlambert.
Nov 26 2021, 10:01 AM · SVN Loader
ardumont closed T3747: svn: FileExistsError when creating directory as Resolved.
Nov 26 2021, 10:01 AM · SVN Loader
ardumont added a comment to T3747: svn: FileExistsError when creating directory.

Diff deployed through the v0.10.1

Nov 26 2021, 9:55 AM · SVN Loader
ardumont added a comment to D6396: Implement maven jar source files loader.

Waiting for the discussion on the lister diff to settle prior to accepting it (as this could impact the loader as well).

Nov 26 2021, 9:54 AM
ardumont added a comment to D6395: lister: Add new maven lister.

Awesome, thanks for the last updates. LGTM.

Nov 26 2021, 9:51 AM
vsellier moved T3750: Upgrade ELK stack to bullseye from Backlog to In progress on the System administration (Component upgrades) board.
Nov 26 2021, 9:39 AM · System administration (Component upgrades)
vsellier edited projects for T3750: Upgrade ELK stack to bullseye, added: System administration (Component upgrades); removed System administration.
Nov 26 2021, 9:39 AM · System administration (Component upgrades)
vsellier moved T3751: Upgrade swh-search elasticsearch servers to bullseye from Backlog to In progress on the System administration (Component upgrades) board.
Nov 26 2021, 9:38 AM · System administration (Component upgrades)
vsellier added a comment to T3751: Upgrade swh-search elasticsearch servers to bullseye .
  • search-esnode0 updated without errors
Nov 26 2021, 9:38 AM · System administration (Component upgrades)
vsellier updated the task description for T3751: Upgrade swh-search elasticsearch servers to bullseye .
Nov 26 2021, 9:36 AM · System administration (Component upgrades)
vsellier changed the status of T3751: Upgrade swh-search elasticsearch servers to bullseye from Open to Work in Progress.
Nov 26 2021, 8:55 AM · System administration (Component upgrades)

Nov 25 2021

rdicosmo committed rMSLD0f23d96ba769: Update ambassador image (authored by rdicosmo).
Update ambassador image
Nov 25 2021, 7:22 PM
rdicosmo committed rMSLDceadcaeeaf21: Add road ahead (authored by rdicosmo).
Add road ahead
Nov 25 2021, 7:22 PM
vsellier added a comment to T3750: Upgrade ELK stack to bullseye.

The upgrade by itself was made with the same command as explained previously.
The shard allocation was disabled during the process to avoid unnecessary movements of shard in the cluster

Nov 25 2021, 7:19 PM · System administration (Component upgrades)
swh-public-ci added a comment to D6395: lister: Add new maven lister.

Build is green

Nov 25 2021, 7:16 PM
borisbaldassari updated the diff for D6395: lister: Add new maven lister.

Try to restart the build

Nov 25 2021, 7:13 PM
rdicosmo committed rMSLD06505de63d0d: Update Inria logo (authored by rdicosmo).
Update Inria logo
Nov 25 2021, 7:02 PM
rdicosmo committed rMSLDb92da98df45b: add spacing (authored by rdicosmo).
add spacing
Nov 25 2021, 7:02 PM
rdicosmo committed rMSLD33c0f2d1c0f9: EOSC Taks Force (authored by rdicosmo).
EOSC Taks Force
Nov 25 2021, 7:02 PM
rdicosmo committed rMSLD8485c02cf292: Update policy slide (authored by rdicosmo).
Update policy slide
Nov 25 2021, 7:02 PM
swh-public-ci added a comment to D6396: Implement maven jar source files loader.

Build is green

Nov 25 2021, 5:21 PM
borisbaldassari updated the diff for D6396: Implement maven jar source files loader.

Amend commit, last fixes after review.

Nov 25 2021, 5:18 PM
borisbaldassari updated the summary of D6396: Implement maven jar source files loader.
Nov 25 2021, 5:14 PM
borisbaldassari added a comment to D6395: lister: Add new maven lister.
Nov 25 2021, 5:13 PM
Harbormaster failed remote builds in B25182: Diff 24308 for D6395: lister: Add new maven lister!
Nov 25 2021, 5:07 PM
swh-public-ci added a comment to D6395: lister: Add new maven lister.

Build has FAILED

Nov 25 2021, 5:07 PM
borisbaldassari updated the diff for D6395: lister: Add new maven lister.

Updating D6395: rebase, update commit.

Nov 25 2021, 5:04 PM
borisbaldassari added a comment to D6395: lister: Add new maven lister.

Not totally. I meant, can you please change the commit message (first line) to

Nov 25 2021, 5:01 PM
seirl committed rDGRPH32bab89d4448: BidirectionalImmutableGraph: implement outdegrees and predecessorBigArray… (authored by seirl).
BidirectionalImmutableGraph: implement outdegrees and predecessorBigArray…
Nov 25 2021, 5:00 PM
ardumont retitled D6396: Implement maven jar source files loader from maven jar-loader: Initalise files to Implement maven jar source files loader.
Nov 25 2021, 4:53 PM