Page MenuHomeSoftware Heritage
Feed All Stories

Sep 22 2021

vlorentz closed T3582: cassandra: Use 'git ordering' for directory entries as Wontfix.

Heh actually that's not an issue, the directory_get_entries documentation does not guarantee an order.

Sep 22 2021, 1:44 PM · Storage manager
vlorentz updated the task description for T3594: Faithfully store weird git objects.
Sep 22 2021, 1:42 PM · meta-task, Data Model, Storage manager
vlorentz added a comment to T3596: Support "weird" permissions in directories.

Possible solution: store them as an ascii string instead of an integer.

Sep 22 2021, 1:38 PM · meta-task, Data Model, Storage manager
vlorentz added a comment to T3595: Support disordered directory entries in git.

Possible solution: store a rank along with each directory entry, but ignore it unless we are reconstructing a git object or computing a SWHID (v1?)

Sep 22 2021, 1:37 PM · meta-task, Data Model, Storage manager
vlorentz triaged T3596: Support "weird" permissions in directories as Normal priority.
Sep 22 2021, 1:36 PM · meta-task, Data Model, Storage manager
vlorentz updated the task description for T3595: Support disordered directory entries in git.
Sep 22 2021, 1:34 PM · meta-task, Data Model, Storage manager
vlorentz triaged T3595: Support disordered directory entries in git as Normal priority.
Sep 22 2021, 1:34 PM · meta-task, Data Model, Storage manager
vlorentz triaged T3594: Faithfully store weird git objects as Normal priority.
Sep 22 2021, 1:31 PM · meta-task, Data Model, Storage manager
ardumont added a comment to D6316: opam: Share opam root directory even on multiple instances.

Following what i said in the loader diff, i'm actually closing this.
Ack on the lock folder but i won't attend to it immediately.

[1] D6318

Sep 22 2021, 1:13 PM
swh-public-ci added a comment to D6318: opam: Allow shared state between loader runs using multi-instance opam root.

Build is green

Sep 22 2021, 1:07 PM
ardumont updated the diff for D6318: opam: Allow shared state between loader runs using multi-instance opam root.

Refactor a bit (drop --strict as suggested)

Sep 22 2021, 1:04 PM
zack added a comment to T1805: Public API v2.

it's true these do not come "for free" but I still have the impression there is an "Open API way" of handling these and we should stick to them.

Sep 22 2021, 1:03 PM · meta-task, Web app
ardumont reclaimed D6316: opam: Share opam root directory even on multiple instances.
Sep 22 2021, 1:02 PM
Harbormaster failed remote builds in B23890: Diff 22960 for D5992: add support for the CVS loader to 'Save Code Now'!
Sep 22 2021, 12:59 PM
swh-public-ci added a comment to D5992: add support for the CVS loader to 'Save Code Now'.

Build has FAILED

Sep 22 2021, 12:59 PM
ardumont requested review of D6318: opam: Allow shared state between loader runs using multi-instance opam root.
Sep 22 2021, 12:56 PM
olasd closed T3574: Upgrade hedgedoc to 1.9.0 as Resolved.

Done and dusted.

Sep 22 2021, 12:53 PM · System administration
olasd committed rSPSITEfe519c3939cd: Upgrade hedgedoc to 1.9.0 (authored by olasd).
Upgrade hedgedoc to 1.9.0
Sep 22 2021, 12:47 PM
olasd committed rSPSITEf5110a49138e: Add apt source configuration for nodejs LTS 14.x (authored by olasd).
Add apt source configuration for nodejs LTS 14.x
Sep 22 2021, 12:47 PM
olasd closed D6274: Upgrade hedgedoc to 1.9.0.
Sep 22 2021, 12:47 PM
olasd committed rSPSITE28c7f765463a: Update expired yarn public key (authored by olasd).
Update expired yarn public key
Sep 22 2021, 12:47 PM
olasd updated the diff for D6274: Upgrade hedgedoc to 1.9.0.

rebase

Sep 22 2021, 12:46 PM
stsp updated the diff for D5992: add support for the CVS loader to 'Save Code Now'.

rebased patch

Sep 22 2021, 12:45 PM
vsellier closed D6308: Add a documentation page to list the services urls.
Sep 22 2021, 12:35 PM · System administration
vsellier committed rDDOC3a935af020e7: add a documentation page to list the services urls (authored by vsellier).
add a documentation page to list the services urls
Sep 22 2021, 12:35 PM
stsp closed D6298: eliminate code duplication of logic in process_cvs_changesets().
Sep 22 2021, 12:29 PM
stsp committed rDLDCVSa7eaeb89f46f: eliminate code duplication of logic in process_cvs_changesets() (authored by stsp).
eliminate code duplication of logic in process_cvs_changesets()
Sep 22 2021, 12:29 PM
vsellier committed rDENV2507d723a1e6: POC a default smaller profile equivalent to the default docker-compose (authored by vsellier).
POC a default smaller profile equivalent to the default docker-compose
Sep 22 2021, 12:28 PM
vsellier committed rDENV46090bb0b458: Upgrade the registry-ui and fix a CORS issue blocking the access to the registry (authored by vsellier).
Upgrade the registry-ui and fix a CORS issue blocking the access to the registry
Sep 22 2021, 12:28 PM
swh-public-ci added a comment to D6298: eliminate code duplication of logic in process_cvs_changesets().

Build is green

Sep 22 2021, 12:18 PM
ardumont closed D6317: opam: Initialize opam root directory outside the constructor.
Sep 22 2021, 12:18 PM
ardumont committed rDLDBASEf4556e0113fa: opam: Initialize opam root directory outside the constructor (authored by ardumont).
opam: Initialize opam root directory outside the constructor
Sep 22 2021, 12:18 PM
stsp updated the diff for D6298: eliminate code duplication of logic in process_cvs_changesets().

rebased patch

Sep 22 2021, 12:16 PM
vsellier updated the task description for T3592: POC elastic worker infrastructure.
Sep 22 2021, 12:03 PM · System administration
ardumont abandoned D6316: opam: Share opam root directory even on multiple instances.

Following what i said in the loader diff, i'm actually closing this.
Ack on the lock folder but i won't attend to it immediately.

Sep 22 2021, 11:56 AM
ardumont added a comment to P1172 a bit of a scheduling predicament!?.

tl; dr Inhibit the cooldown* which default to 7 days.
As it got already scheduled (with whatever results), this is dropped from the grab call.

Sep 22 2021, 11:55 AM
vlorentz committed rDSNIP4dbfdc9d6bf9: vault_repro.py: update to latest CLI (authored by vlorentz).
vault_repro.py: update to latest CLI
Sep 22 2021, 11:52 AM
ardumont accepted D6308: Add a documentation page to list the services urls.
Sep 22 2021, 11:52 AM · System administration
vlorentz committed rDSNIPc9deabf1bb16: add check_consistency.py (authored by vlorentz).
add check_consistency.py
Sep 22 2021, 11:52 AM
douardda accepted D6308: Add a documentation page to list the services urls.

LGTM (not checked everything is accurate nor there are obvious missing services, but it's a huge improvement as is, thx)

Sep 22 2021, 11:42 AM · System administration
vlorentz requested review of D6319: model: optimize dictify().
Sep 22 2021, 11:41 AM
ardumont created P1172 a bit of a scheduling predicament!?.
Sep 22 2021, 11:37 AM
douardda added a comment to T1805: Public API v2.

Items 5, 6, 7 aka pagination, auth and batches - I believe these come naturally with item 4 (specification wise)

They don't. OpenAPI is a specification to describe APIs, and it contains absolutely nothing about pagination or batches.

Sep 22 2021, 11:36 AM · meta-task, Web app
vsellier updated the diff for D6308: Add a documentation page to list the services urls.
  • Remove useless ending '/'
  • define VPN / private meanings
Sep 22 2021, 11:25 AM · System administration
swh-public-ci added a comment to D6289: vault: Escape provided email address in the URL.

Build is green

Sep 22 2021, 11:19 AM
douardda added a comment to D6316: opam: Share opam root directory even on multiple instances.

I think there was already a problem before, but since we have now more chance to hit it, I'd really like the opam_init process to lock the directory when running opam commands.

Sep 22 2021, 11:16 AM
douardda added inline comments to D6133: maven-lister: initialise lister..
Sep 22 2021, 11:08 AM
aeviso updated the summary of D6165: Add new RabbitMQ-based client/server API.
Sep 22 2021, 11:05 AM
aeviso added a comment to D6165: Add new RabbitMQ-based client/server API.

What is the reason for this change? Is it more efficient assign requests to workers based on ID rather than randomly?

Sep 22 2021, 11:04 AM
vsellier added inline comments to D6308: Add a documentation page to list the services urls.
Sep 22 2021, 10:56 AM · System administration
aeviso added a comment to D6272: Remove remote storage based on `swh.core.api.RPCClient`.

Why the renaming? And it's an RPC API, not REST

Sep 22 2021, 10:54 AM
douardda added a comment to D6133: maven-lister: initialise lister..

It would be nice to have a README fil in swh/lister/maven/tests/data explaining what the data files are, where they come from, how they have been generated, etc.

Sep 22 2021, 10:52 AM
aeviso added a comment to D6273: Remove remote storage based on `swh.core.api.RPCClient`.

Why?

Sep 22 2021, 10:51 AM
douardda added a comment to D6165: Add new RabbitMQ-based client/server API.

What is the reason for this change? Is it more efficient assign requests to workers based on ID rather than randomly?

Sep 22 2021, 10:48 AM
jayeshv updated the summary of D6291: WIP - Unit tests for the mongo backend.
Sep 22 2021, 10:45 AM
douardda added inline comments to D6308: Add a documentation page to list the services urls.
Sep 22 2021, 10:43 AM · System administration
moranegg updated the task description for T3297: Prepare documentation sprints 2021.
Sep 22 2021, 10:42 AM · Documentation
vsellier added inline comments to D6308: Add a documentation page to list the services urls.
Sep 22 2021, 10:41 AM · System administration
ardumont updated the task description for T3584: loader mercurial edge case about missing mapping from revision to hgnode-id.
Sep 22 2021, 10:39 AM · Mercurial loader
douardda accepted D6317: opam: Initialize opam root directory outside the constructor.
Sep 22 2021, 10:38 AM
douardda accepted D6300: Capture missing revision <-> hgnode-id scenario in a xfail test.
Sep 22 2021, 10:37 AM
jayeshv retitled D6291: WIP - Unit tests for the mongo backend from Unit tests for content and directory add methods in mongo to WIP - Unit tests for the mongo backend.
Sep 22 2021, 10:36 AM
swh-public-ci added a comment to D6291: WIP - Unit tests for the mongo backend.

Build is green

Sep 22 2021, 10:33 AM
jayeshv updated the diff for D6291: WIP - Unit tests for the mongo backend.
  • added stubs
  • small fixes
  • more doc strings
  • Better doc strings
Sep 22 2021, 10:30 AM
ardumont committed rSPSITEcf6550783c19: Drop no longer needed configuration option. (authored by ardumont).
Drop no longer needed configuration option.
Sep 22 2021, 10:29 AM
ardumont updated the task description for T3581: List heptapod instance foss.heptapod.net.
Sep 22 2021, 10:14 AM · Archive coverage, System administration, Origin-GitLab
ardumont changed the status of T3581: List heptapod instance foss.heptapod.net from Open to Work in Progress.
Sep 22 2021, 9:50 AM · Archive coverage, System administration, Origin-GitLab
ardumont moved T3581: List heptapod instance foss.heptapod.net from Backlog to Weekly backlog on the System administration board.
Sep 22 2021, 9:50 AM · Archive coverage, System administration, Origin-GitLab
ardumont added a project to T3581: List heptapod instance foss.heptapod.net: System administration.
Sep 22 2021, 9:50 AM · Archive coverage, System administration, Origin-GitLab

Sep 21 2021

ardumont updated the summary of D6316: opam: Share opam root directory even on multiple instances.
Sep 21 2021, 7:55 PM
ardumont added a comment to D6316: opam: Share opam root directory even on multiple instances.

Ok.

What is the motivation for merging directories?

Sep 21 2021, 7:54 PM
vlorentz added a comment to D6316: opam: Share opam root directory even on multiple instances.

What is the motivation for merging directories?

Sep 21 2021, 7:46 PM
ardumont requested review of D6317: opam: Initialize opam root directory outside the constructor.
Sep 21 2021, 7:38 PM
ardumont added a revision to T3590: opam loader: Ensure required opam state is shared amongst ingestion/listing runs: D6317: opam: Initialize opam root directory outside the constructor.
Sep 21 2021, 7:36 PM · Archive coverage, Opam
ardumont added a comment to D6316: opam: Share opam root directory even on multiple instances.

Sorry, my question was about opam itself. Are you sure it won't corrupt data, or wait on a global lock when accessing that directory?

Sep 21 2021, 7:34 PM
vlorentz added a comment to D6316: opam: Share opam root directory even on multiple instances.

Sorry, my question was about opam itself. Are you sure it won't corrupt data, or wait on a global lock when accessing that directory?

Sep 21 2021, 7:29 PM
ardumont renamed T3590: opam loader: Ensure required opam state is shared amongst ingestion/listing runs from opam loader: Ensure the current opam state is shared amongst ingestion to opam loader: Ensure required opam state is shared amongst ingestion/listing runs.
Sep 21 2021, 7:28 PM · Archive coverage, Opam
vlorentz added a comment to D6272: Remove remote storage based on `swh.core.api.RPCClient`.

Same comment as on the other diff: looks like it is for consistency with the new rabbitmq RPC, but please explain it

Sep 21 2021, 7:27 PM
vlorentz added a comment to D6273: Remove remote storage based on `swh.core.api.RPCClient`.

Looks like it is supposed to be replaced by the rabbitmq-based RPC? Please explain it in the commit/diff message.

Sep 21 2021, 7:26 PM
ardumont added a comment to D6316: opam: Share opam root directory even on multiple instances.

Can two listers run in parallel while using the same directory?

Sep 21 2021, 7:26 PM
vlorentz added a comment to D6165: Add new RabbitMQ-based client/server API.

What is the reason for this change? Is it more efficient assign requests to workers based on ID rather than randomly?

Sep 21 2021, 7:26 PM
vlorentz added a comment to D6272: Remove remote storage based on `swh.core.api.RPCClient`.

Why the renaming? And it's an RPC API, not REST

Sep 21 2021, 7:18 PM
vlorentz added a comment to D6273: Remove remote storage based on `swh.core.api.RPCClient`.

Why?

Sep 21 2021, 7:16 PM
vlorentz added a comment to D6316: opam: Share opam root directory even on multiple instances.

Can two listers run in parallel while using the same directory?

Sep 21 2021, 7:15 PM
Harbormaster failed remote builds in B23878: Diff 22947 for D6289: vault: Escape provided email address in the URL!
Sep 21 2021, 6:34 PM
swh-public-ci added a comment to D6289: vault: Escape provided email address in the URL.

Build has FAILED

Sep 21 2021, 6:34 PM
ardumont requested review of D6316: opam: Share opam root directory even on multiple instances.
Sep 21 2021, 6:29 PM
swh-public-ci added a comment to D6269: cassandra: Make _content_get_from_hashes run concurrently.

Build is green

Sep 21 2021, 6:23 PM
vlorentz closed D6269: cassandra: Make _content_get_from_hashes run concurrently.
Sep 21 2021, 6:14 PM
vlorentz committed rDSTO61e9e4a3232f: cassandra: Make _content_get_from_hashes run concurrently (authored by vlorentz).
cassandra: Make _content_get_from_hashes run concurrently
Sep 21 2021, 6:14 PM
vlorentz updated the diff for D6269: cassandra: Make _content_get_from_hashes run concurrently.

rebase

Sep 21 2021, 6:14 PM
vlorentz updated the diff for D6289: vault: Escape provided email address in the URL.

fix test + use email-validator + rebase

Sep 21 2021, 6:13 PM
jayeshv closed D6220: Added test only method info in the interface doc strings.
Sep 21 2021, 6:11 PM
jayeshv committed rDPROV4c087ea0ece7: Docstring changes for test only methods (Minor). (authored by jayeshv).
Docstring changes for test only methods (Minor).
Sep 21 2021, 6:07 PM
swh-public-ci added a comment to D6299: Add a bit of typing.

Build is green

Sep 21 2021, 6:01 PM
swh-public-ci added a comment to D6297: Simplify exception handling..

Build is green

Sep 21 2021, 6:01 PM
swh-public-ci added a comment to D6296: Ignore generated .so files.

Build is green

Sep 21 2021, 6:01 PM
swh-public-ci added a comment to D6295: Use lazy substitution for logging + fix error messages.

Build is green

Sep 21 2021, 6:00 PM
swh-public-ci added a comment to D6293: load_status should not be 'uneventful' if the snapshot changed.

Build is green

Sep 21 2021, 6:00 PM
vlorentz closed D6299: Add a bit of typing.
Sep 21 2021, 5:58 PM