Page MenuHomeSoftware Heritage
Feed All Stories

Oct 14 2021

swh-public-ci added a comment to D6165: Add new RabbitMQ-based client/server API.

Build is green

Oct 14 2021, 1:46 PM
swh-public-ci added a comment to D6473: Improve PostgreSQL storage scheme for the `with-path-denormalized` flavor.

Build is green

Oct 14 2021, 1:44 PM
aeviso updated the diff for D6339: Add support for remote backend on existing storage tests.

rebase

Oct 14 2021, 1:42 PM
aeviso updated the diff for D6165: Add new RabbitMQ-based client/server API.

rebase

Oct 14 2021, 1:41 PM
aeviso updated the diff for D6473: Improve PostgreSQL storage scheme for the `with-path-denormalized` flavor.

rebase

Oct 14 2021, 1:39 PM
ardumont accepted D6472: Add a script for a 'monthly roadmap report' bot email.

I'm fine with the idea.
I've suggested a couple of fix, one actually be blocking.

Oct 14 2021, 1:32 PM
ardumont added inline comments to D6472: Add a script for a 'monthly roadmap report' bot email.
Oct 14 2021, 1:23 PM
ardumont added a comment to D6472: Add a script for a 'monthly roadmap report' bot email.

Do we have the same for the weekly report already?

Oct 14 2021, 1:22 PM
ardumont added a comment to D6472: Add a script for a 'monthly roadmap report' bot email.

Good idea.

Oct 14 2021, 1:20 PM
ardumont renamed T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs from proposal: sysadm docs: Move relevant and public doc from intranet to swh-docs to sysadm docs: Move relevant and public doc from intranet to swh-docs.
Oct 14 2021, 1:16 PM · System administration, Documentation
ardumont added a project to T3154: sysadm docs: Move relevant and public doc from intranet to swh-docs: System administration.
Oct 14 2021, 1:15 PM · System administration, Documentation
ardumont added a comment to T3651: doc: drop unneeded "Software Heritage" from page titles.

published ^

Oct 14 2021, 1:14 PM · Easy hack, Documentation
ardumont closed T3651: doc: drop unneeded "Software Heritage" from page titles as Resolved.
Oct 14 2021, 12:31 PM · Easy hack, Documentation
ardumont closed D6469: Drop unneeded "Software Heritage" from page titles.
Oct 14 2021, 12:30 PM
ardumont committed rDDOC67977961d24e: Drop unneeded "Software Heritage" from page titles (authored by ardumont).
Drop unneeded "Software Heritage" from page titles
Oct 14 2021, 12:30 PM
ardumont updated the diff for D6469: Drop unneeded "Software Heritage" from page titles.

Rebase

Oct 14 2021, 12:30 PM
ardumont added a comment to D6470: Make it explicit that the "main" docs page is actually devel doc.

duh, sorry, I missed it

Oct 14 2021, 12:28 PM
ardumont committed rDDOC40e75f112d72: Reference the doc contribution tutorial from the main menu (authored by ardumont).
Reference the doc contribution tutorial from the main menu
Oct 14 2021, 12:26 PM
ardumont committed rDDOC5f45a6e0e91a: Drop no longer needed network images reference (authored by ardumont).
Drop no longer needed network images reference
Oct 14 2021, 12:26 PM
vlorentz accepted D6469: Drop unneeded "Software Heritage" from page titles.
Oct 14 2021, 12:23 PM
vlorentz added a comment to D6470: Make it explicit that the "main" docs page is actually devel doc.

duh, sorry, I missed it

Oct 14 2021, 12:23 PM
aeviso requested review of D6473: Improve PostgreSQL storage scheme for the `with-path-denormalized` flavor.
Oct 14 2021, 12:20 PM
aeviso closed D6456: Move path normalization function to `util` submodule.
Oct 14 2021, 12:16 PM
aeviso committed rDPROV3e87301a2868: Move path normalization function to `util` submodule (authored by aeviso).
Move path normalization function to `util` submodule
Oct 14 2021, 12:16 PM
vlorentz updated the task description for T3609: SWHIDv2: List issues with SWHIDv1 that should be fixed.
Oct 14 2021, 12:15 PM · Roadmap 2020, Data Model, Web app, Roadmap 2021
vlorentz removed a subtask for T3134: SWHID v2: T1957: Handling missing DAG nodes.
Oct 14 2021, 12:14 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021
vlorentz removed a parent task for T1957: Handling missing DAG nodes: T3134: SWHID v2.
Oct 14 2021, 12:14 PM · Data Model
vlorentz added a subtask for T3134: SWHID v2: T1957: Handling missing DAG nodes.
Oct 14 2021, 12:13 PM · Roadmap 2022, Roadmap 2020, Data Model, Web app, meta-task, Roadmap 2021
vlorentz added a parent task for T1957: Handling missing DAG nodes: T3134: SWHID v2.
Oct 14 2021, 12:12 PM · Data Model
olasd added a comment to T1957: Handling missing DAG nodes.

In SWHIDv2, instead of having a hardcoded "pointer to another revision" directory entry type, we could enable pointers to more generic "unresolved external entities". When possible, we should make these pointers compatible with the current ExtID table, so that users of the data can look the contents of the pointed objects up lazily.

Oct 14 2021, 12:06 PM · Data Model
olasd updated subscribers of T1617: Experiment with generation numbers to improve revisions walk performance.

@vlorentz mentioned this idea in the context of T3655 (git loader global deduplication).

Oct 14 2021, 12:00 PM · Storage manager
olasd added a comment to T3635: git loader: enable "partial" global deduplication of revisions via the extid mapping table.

Then I don't really get how this can help if we don't load revisions in topological order.

Oct 14 2021, 11:54 AM · Git loader
vsellier committed rDDOC932896f50fd1: sysadm/network: remove a typo (authored by vsellier).
sysadm/network: remove a typo
Oct 14 2021, 11:53 AM
ardumont updated the summary of D6468: ci: Rename users documentation to user.
Oct 14 2021, 11:42 AM
ardumont updated the summary of D6467: Rename users documentation to user.
Oct 14 2021, 11:41 AM
ardumont updated the summary of D6468: ci: Rename users documentation to user.
Oct 14 2021, 11:41 AM
ardumont updated the summary of D6468: ci: Rename users documentation to user.
Oct 14 2021, 11:41 AM
olasd updated the task description for T3655: loader git: enable global deduplication of head branches before fetching them.
Oct 14 2021, 11:41 AM · Git loader
ardumont updated the summary of D6469: Drop unneeded "Software Heritage" from page titles.
Oct 14 2021, 11:40 AM
olasd triaged T3656: Survey revisions/releases with partially loaded history as Low priority.
Oct 14 2021, 11:40 AM · Archive content
ardumont added a comment to D6470: Make it explicit that the "main" docs page is actually devel doc.

Answered in the description, gist of it is "to reflect the docs reality today".

Oct 14 2021, 11:39 AM
ardumont updated the summary of D6470: Make it explicit that the "main" docs page is actually devel doc.
Oct 14 2021, 11:39 AM
vlorentz added a comment to T3573: [cassandra] directory and content read benchmarks.

What "directory_entries_insert_algo" did you use for this?

Oct 14 2021, 11:32 AM · System administration, Storage manager
vlorentz added a comment to D6470: Make it explicit that the "main" docs page is actually devel doc.

What is the motivation for this change?

Oct 14 2021, 11:29 AM
vsellier changed the status of T3630: staging - journal0 needs more space from Open to Work in Progress.
Oct 14 2021, 11:28 AM · System administration
Harbormaster failed to build B24418: rDWAPPSa200f70f2ce9: tests: Turn remaining hypothesis strategies into pytest fixtures for rDWAPPSa200f70f2ce9: tests: Turn remaining hypothesis strategies into pytest fixtures!
Oct 14 2021, 11:28 AM
Harbormaster failed to build B24415: rDWAPPSc4fed93688b1: tests: Turn release* hypothesis strategies into pytest fixtures for rDWAPPSc4fed93688b1: tests: Turn release* hypothesis strategies into pytest fixtures!
Oct 14 2021, 11:27 AM
Harbormaster failed to build B24416: rDWAPPS341454015ded: tests: Turn revision* hypothesis strategies into pytest fixtures for rDWAPPS341454015ded: tests: Turn revision* hypothesis strategies into pytest fixtures!
Oct 14 2021, 11:27 AM
Harbormaster failed to build B24417: rDWAPPSa91200cbe1e5: tests: Turn snapshot* hypothesis strategies into pytest fixtures for rDWAPPSa91200cbe1e5: tests: Turn snapshot* hypothesis strategies into pytest fixtures!
Oct 14 2021, 11:27 AM
Harbormaster failed to build B24413: rDWAPPS81486f517526: tests: Turn directory* hypothesis strategies into pytest fixtures for rDWAPPS81486f517526: tests: Turn directory* hypothesis strategies into pytest fixtures!
Oct 14 2021, 11:27 AM
Harbormaster failed to build B24414: rDWAPPSc2d10c90c3bd: tests: Turn origin* hypothesis strategies into pytest fixtures for rDWAPPSc2d10c90c3bd: tests: Turn origin* hypothesis strategies into pytest fixtures!
Oct 14 2021, 11:27 AM
Harbormaster failed to build B24412: rDWAPPS23a8dc33453f: tests: Turn content* hypothesis strategies into pytest fixtures for rDWAPPS23a8dc33453f: tests: Turn content* hypothesis strategies into pytest fixtures!
Oct 14 2021, 11:27 AM
vlorentz added 1 blocking reviewer(s) for D6472: Add a script for a 'monthly roadmap report' bot email: System administrators.
Oct 14 2021, 11:25 AM
vlorentz added reviewers for D6472: Add a script for a 'monthly roadmap report' bot email: System administrators, Reviewers.
Oct 14 2021, 11:25 AM
vlorentz requested review of D6472: Add a script for a 'monthly roadmap report' bot email.
Oct 14 2021, 11:24 AM
douardda added a comment to T3635: git loader: enable "partial" global deduplication of revisions via the extid mapping table.

Ok I think what puzzle me in this description is the fact the 2 first bullets of the "git loader adaptations" are actually only one point: at the end of a successful loading, store a mapping in the extid table.

Oct 14 2021, 11:23 AM · Git loader
olasd added a parent task for T3635: git loader: enable "partial" global deduplication of revisions via the extid mapping table: T3655: loader git: enable global deduplication of head branches before fetching them.
Oct 14 2021, 11:18 AM · Git loader
olasd added subtasks for T3655: loader git: enable global deduplication of head branches before fetching them: T3635: git loader: enable "partial" global deduplication of revisions via the extid mapping table, T3654: loader git: load revisions in topological order.
Oct 14 2021, 11:18 AM · Git loader
olasd added a parent task for T3654: loader git: load revisions in topological order: T3655: loader git: enable global deduplication of head branches before fetching them.
Oct 14 2021, 11:18 AM · Git loader
olasd triaged T3655: loader git: enable global deduplication of head branches before fetching them as Normal priority.
Oct 14 2021, 11:18 AM · Git loader
vsellier closed D6465: sphinx: fix pip cache directory permissions.
Oct 14 2021, 11:16 AM
vsellier committed rCDFJb2eae5d37d3e: sphinx: fix pip cache directory permissions (authored by vsellier).
sphinx: fix pip cache directory permissions
Oct 14 2021, 11:16 AM
olasd renamed T3635: git loader: enable "partial" global deduplication of revisions via the extid mapping table from Reduce git loader work (use extid mapping table) to git loader: enable "partial" global deduplication of revisions via the extid mapping table.
Oct 14 2021, 11:15 AM · Git loader
vsellier closed D6466: Proposal for network page.
Oct 14 2021, 11:14 AM
vsellier committed rDDOC449f7ad4d973: Proposal for network page (authored by vsellier).
Proposal for network page
Oct 14 2021, 11:14 AM
olasd added a comment to T3654: loader git: load revisions in topological order.

(I've removed T3653 as parent as this is a somewhat longer term endeavour. Not the topological sorting itself, but making sure that (most) existing revisions aren't dangling, before we can use this topological guarantee)

Oct 14 2021, 11:13 AM · Git loader
vsellier updated the diff for D6466: Proposal for network page.

rebase

Oct 14 2021, 11:13 AM
olasd removed a parent task for T3654: loader git: load revisions in topological order: T3653: Stabilize loader git.
Oct 14 2021, 11:12 AM · Git loader
olasd removed a subtask for T3653: Stabilize loader git: T3654: loader git: load revisions in topological order.
Oct 14 2021, 11:12 AM · Git loader
olasd triaged T3654: loader git: load revisions in topological order as Low priority.
Oct 14 2021, 11:11 AM · Git loader
vsellier updated the diff for D6466: Proposal for network page.

update according to the feedbacks

Oct 14 2021, 11:06 AM
anlambert closed D6463: tests: Turn remaining hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert closed T3603: Replace stateful hypothesis strategies by pytest fixtures as Resolved by committing rDWAPPSa200f70f2ce9: tests: Turn remaining hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM · Web app
anlambert committed rDWAPPSa200f70f2ce9: tests: Turn remaining hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn remaining hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert closed D6461: tests: Turn snapshot* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert closed D6460: tests: Turn revision* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert closed D6459: tests: Turn release* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPSa91200cbe1e5: tests: Turn snapshot* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn snapshot* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPS341454015ded: tests: Turn revision* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn revision* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPSc4fed93688b1: tests: Turn release* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn release* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert closed D6458: tests: Turn origin* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert closed D6457: tests: Turn directory* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPS81486f517526: tests: Turn directory* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn directory* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPSc2d10c90c3bd: tests: Turn origin* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn origin* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
anlambert closed D6455: tests: Turn content* hypothesis strategies into pytest fixtures.
Oct 14 2021, 11:04 AM
anlambert committed rDWAPPS23a8dc33453f: tests: Turn content* hypothesis strategies into pytest fixtures (authored by anlambert).
tests: Turn content* hypothesis strategies into pytest fixtures
Oct 14 2021, 11:04 AM
olasd accepted D6458: tests: Turn origin* hypothesis strategies into pytest fixtures.
In D6458#167772, @olasd wrote:
In D6458#167771, @olasd wrote:

Yeah, sure, I don't have a problem with that.

(That is, I don't have a problem with these changes landing first, as long as we make sure that eventually we have a proper way of reproducing test failures that have come out of "random" fixtures.)

I might have a proper solution to reproduce test failures involving random fixtures. I am going to land that pile of diffs and submit a new one for reproducibility afterwards.

Oct 14 2021, 11:03 AM
anlambert added a comment to D6458: tests: Turn origin* hypothesis strategies into pytest fixtures.
In D6458#167772, @olasd wrote:
In D6458#167771, @olasd wrote:

Yeah, sure, I don't have a problem with that.

(That is, I don't have a problem with these changes landing first, as long as we make sure that eventually we have a proper way of reproducing test failures that have come out of "random" fixtures.)

Oct 14 2021, 11:03 AM
ardumont accepted D6465: sphinx: fix pip cache directory permissions.
Oct 14 2021, 10:54 AM
ardumont added a reviewer for D6465: sphinx: fix pip cache directory permissions: Reviewers.
Oct 14 2021, 10:53 AM
ardumont updated the task description for T3653: Stabilize loader git.
Oct 14 2021, 10:46 AM · Git loader
ardumont added a subtask for T3653: Stabilize loader git: T3652: Cannot ingest git repositories with (too) large packfiles.
Oct 14 2021, 10:44 AM · Git loader
ardumont added a parent task for T3652: Cannot ingest git repositories with (too) large packfiles: T3653: Stabilize loader git.
Oct 14 2021, 10:44 AM · Git loader
ardumont updated the task description for T3653: Stabilize loader git.
Oct 14 2021, 10:41 AM · Git loader
ardumont updated the task description for T3653: Stabilize loader git.
Oct 14 2021, 10:39 AM · Git loader
ardumont closed T3625: Reduce git loader memory footprint as Resolved.

Actually deployed and the number of oom actually decreased.

Oct 14 2021, 10:38 AM · Git loader
ardumont closed T3625: Reduce git loader memory footprint, a subtask of T3653: Stabilize loader git, as Resolved.
Oct 14 2021, 10:38 AM · Git loader
ardumont updated the task description for T3653: Stabilize loader git.
Oct 14 2021, 10:38 AM · Git loader
ardumont added a parent task for T3640: Make long running task stop fast when warm shutdown is triggered: T3653: Stabilize loader git.
Oct 14 2021, 10:37 AM · System administration
ardumont added a parent task for T3625: Reduce git loader memory footprint: T3653: Stabilize loader git.
Oct 14 2021, 10:37 AM · Git loader