Page MenuHomeSoftware Heritage
Feed All Stories

Nov 2 2021

vsellier added a comment to T3683: cassandra - benchmark the vault.

When I tried to generate cook some vault in productions, several error where logged.
Some on missing commit, for example:

missing commit 338fb3eff7e93e121e7fe347b391a100ed0003c5
missing commit 73bc01d26ae7d61a74108b620e32458ef30c75c6
missing commit 78272793666b7354f6be373a73aa62426116d884
missing commit 83f434d5f256886fcfcfc354c3a63b91a71fce63

All these hash are identified in the work on T75 so they are not blockers

Nov 2 2021, 12:23 PM · System administration
olasd accepted D6448: Deploy a read-only objstorage on moma.

Nice, thanks!

Nov 2 2021, 11:44 AM
dachary added a comment to D6424: Perfect hashmap C implementation.

@ardumont I revisited all comments and noticed I missed a few, sorry about that. They should be good now :-) Is there anything else you need me to do for this proposed patch? Or should I just be patient and wait for whatever comes next in the review merge process? Thanks for your guidance!

Nov 2 2021, 9:46 AM
swh-public-ci added a comment to D6424: Perfect hashmap C implementation.

Build is green

Nov 2 2021, 9:45 AM
dachary updated the diff for D6424: Perfect hashmap C implementation.

add two missing type annotations

Nov 2 2021, 9:44 AM
andrey-star updated the summary of D6594: Add parameter to load a single graph direction in memory..
Nov 2 2021, 9:31 AM
andrey-star added a revision to T2983: graph service: allow loading in memory only one direction of the graph: D6594: Add parameter to load a single graph direction in memory..
Nov 2 2021, 8:55 AM · Compressed graph service
andrey-star added a task to D6594: Add parameter to load a single graph direction in memory.: T2983: graph service: allow loading in memory only one direction of the graph.
Nov 2 2021, 8:55 AM
andrey-star retitled D6594: Add parameter to load a single graph direction in memory. from Add IntelliJ and MacOS files to .gitignore to Add parameter to load a single graph direction in memory..
Nov 2 2021, 8:43 AM
andrey-star requested review of D6594: Add parameter to load a single graph direction in memory..
Nov 2 2021, 8:42 AM

Oct 31 2021

stsp requested review of D6593: handle Attic-only RCS files over CVS pserver.
Oct 31 2021, 11:33 PM
stsp added a revision to T3691: Implement CVS loader: D6593: handle Attic-only RCS files over CVS pserver.
Oct 31 2021, 11:32 PM · CVS loader, Archive coverage
zack added a comment to T2983: graph service: allow loading in memory only one direction of the graph.

Nope, if loaded with only one direction, traversals will only be possible in the loaded direction.
This will essentially be a trade-off setting for people who cannot (or doesn't want to) load both direction.
It is fine to fail (gracefully, with an error) traversals requested in a direction that corresponds to a non-loaded graph.

Oct 31 2021, 7:10 PM · Compressed graph service
andrey-star added a comment to T2983: graph service: allow loading in memory only one direction of the graph.

Should this still allow the possibility of browsing both directions?

Oct 31 2021, 5:35 PM · Compressed graph service
zack added a comment to T3627: Consider dropping pull request references from the git loader ingestion.

Thanks for the summaries @olasd, both here and on list.
I've followed up on list.

Oct 31 2021, 4:11 PM · Git loader

Oct 30 2021

stsp requested review of D6592: improve test coverage of file additions and deletions.
Oct 30 2021, 2:03 PM
stsp requested review of D6591: display file state in progress logging output.
Oct 30 2021, 2:03 PM
stsp added a revision to T3691: Implement CVS loader: D6590: add support for RCS keyword expansion over pserver protocol.
Oct 30 2021, 11:49 AM · CVS loader, Archive coverage
stsp added a task to D6590: add support for RCS keyword expansion over pserver protocol: T3691: Implement CVS loader.
Oct 30 2021, 11:49 AM
stsp requested review of D6590: add support for RCS keyword expansion over pserver protocol.
Oct 30 2021, 11:49 AM

Oct 29 2021

anlambert requested review of D6589: ra: Fix edge case in svn:eol-style property handling.
Oct 29 2021, 8:29 PM
anlambert added a revision to T3695: Investigate revision reconstruction discrepancy with subversion export: D6589: ra: Fix edge case in svn:eol-style property handling.
Oct 29 2021, 8:27 PM · SVN Loader
posenato added a comment to T3694: Investigate svn loading failure.

@anlambert, thank you for excellent analysis!
Yes, I made an adjustment of commit logs because there was an error during the release and I discover it very soon, so I preferred to make the release again adjusting the comments instead of to make a new release!

Oct 29 2021, 4:53 PM · SVN Loader
jayeshv closed T3641: Top level tabs not working in snapshot view as Resolved.

Got fixed with a741222342b5b74cd4ee9db1da8e99c280602524

Oct 29 2021, 4:43 PM · Web app
jayeshv closed D6541: Deprecate /origin/branches route.

a741222342b5b74cd4ee9db1da8e99c280602524

Oct 29 2021, 4:12 PM
jayeshv reopened D6541: Deprecate /origin/branches route.
Oct 29 2021, 4:10 PM
olasd requested review of D6588: recurrent visits: use policy weights instead of ratios.
Oct 29 2021, 4:05 PM
jayeshv closed D6541: Deprecate /origin/branches route.
Oct 29 2021, 4:04 PM
jayeshv closed T3688: Deprecate /browse/origin/releases URLs as Resolved by committing rDWAPPS5d56274de716: Deprecate /origin/releases route.
Oct 29 2021, 4:03 PM · Web app
jayeshv closed T3688: Deprecate /browse/origin/releases URLs, a subtask of T3608: Deprecate most of the /browse/origin/.* URLs, as Resolved.
Oct 29 2021, 4:03 PM · Web app
jayeshv closed D6586: Deprecate /origin/releases route.
Oct 29 2021, 4:03 PM
jayeshv committed rDWAPPS5d56274de716: Deprecate /origin/releases route (authored by jayeshv).
Deprecate /origin/releases route
Oct 29 2021, 4:03 PM
anlambert accepted D6586: Deprecate /origin/releases route.

Looks good to me, thanks !

Oct 29 2021, 4:00 PM
borisbaldassari added a comment to D6395: lister: Add new maven lister.

[1] scheduler: D6520
[2] docker container: D6543 (to deploy ^ in the docker world)

So those landed. So now you should rebuild the containers [1] to have the most recent
scheduler version (after the listing is done, the scheduling of the maven origins should
happen).

First, make sure to reference in the the loader configuration [2] the new maven task
[3], override your service containers (lister and loader) with the right override and
you should be good to go [4].

Now that we've made some progress (and changes) on the lister I'll re-try that, with the proper loader conf. Thanks for the hint.
I'll keep you posted (probably next week though).

Oct 29 2021, 3:58 PM
swh-public-ci added a comment to D6586: Deprecate /origin/releases route.

Build is green

Oct 29 2021, 3:56 PM
olasd requested review of D6587: Improve docs rendering for recurrent visits scheduler.
Oct 29 2021, 3:48 PM
jayeshv updated the diff for D6586: Deprecate /origin/releases route.

Missing tests

Oct 29 2021, 3:43 PM
anlambert added a comment to T3694: Investigate svn loading failure.

@posenato, I did some deeper analysis on why your svn repository could not be loaded anymore into the archive
and it turned out @ardumont is right, the commit log of your repository got altered between two save code now requests.

Oct 29 2021, 3:41 PM · SVN Loader
swh-public-ci added a comment to D6395: lister: Add new maven lister.

Build is green

Oct 29 2021, 3:32 PM
jayeshv retitled D6586: Deprecate /origin/releases route from [WIP] Deprecate /origin/releases route to Deprecate /origin/releases route.
Oct 29 2021, 3:30 PM
borisbaldassari updated the diff for D6395: lister: Add new maven lister.
  • maven-lister: Split parsing and processing
Oct 29 2021, 3:30 PM
jayeshv requested review of D6586: Deprecate /origin/releases route.
Oct 29 2021, 3:27 PM
jayeshv added a revision to T3608: Deprecate most of the /browse/origin/.* URLs: D6586: Deprecate /origin/releases route.
Oct 29 2021, 3:12 PM · Web app
jayeshv added a revision to T3688: Deprecate /browse/origin/releases URLs: D6586: Deprecate /origin/releases route.
Oct 29 2021, 3:12 PM · Web app
douardda abandoned D6554: [WIP] Add a (redis-based) validation error reporting facility.

used another approach, see D6571

Oct 29 2021, 2:14 PM
vlorentz added inline comments to D6554: [WIP] Add a (redis-based) validation error reporting facility.
Oct 29 2021, 1:58 PM
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Build is green

Oct 29 2021, 1:53 PM
douardda updated the diff for D6571: Add support for a redis-based reporting for invalid mirrorred objects.

apply vlorentz' comments/suggestions

Oct 29 2021, 1:45 PM
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Build is green

Oct 29 2021, 1:36 PM
douardda added inline comments to D6571: Add support for a redis-based reporting for invalid mirrorred objects.
Oct 29 2021, 1:36 PM
douardda updated the diff for D6571: Add support for a redis-based reporting for invalid mirrorred objects.

fix test_cli

Oct 29 2021, 1:28 PM
vlorentz added inline comments to D6571: Add support for a redis-based reporting for invalid mirrorred objects.
Oct 29 2021, 1:24 PM
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Build has FAILED

Oct 29 2021, 1:22 PM
Harbormaster failed remote builds in B24819: Diff 23942 for D6571: Add support for a redis-based reporting for invalid mirrorred objects!
Oct 29 2021, 12:40 PM
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Build has FAILED

Oct 29 2021, 12:40 PM
douardda updated the diff for D6571: Add support for a redis-based reporting for invalid mirrorred objects.

rebase and a few fixes

Oct 29 2021, 12:30 PM
jayeshv closed T3687: Deprecate /browse/origin/branches URLs, a subtask of T3608: Deprecate most of the /browse/origin/.* URLs, as Resolved.
Oct 29 2021, 12:21 PM · Web app
jayeshv closed T3687: Deprecate /browse/origin/branches URLs as Resolved by committing rDWAPPSa741222342b5: Deprecate /origin/branches route.
Oct 29 2021, 12:21 PM · Web app
jayeshv committed rDWAPPSa741222342b5: Deprecate /origin/branches route (authored by jayeshv).
Deprecate /origin/branches route
Oct 29 2021, 12:20 PM
swh-public-ci added a comment to D6541: Deprecate /origin/branches route.

Build is green

Oct 29 2021, 12:15 PM
jayeshv updated the diff for D6541: Deprecate /origin/branches route.

review changes

Oct 29 2021, 12:00 PM
jayeshv added inline comments to D6541: Deprecate /origin/branches route.
Oct 29 2021, 11:42 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO0ced1ee73916: Merge tag 'debian/0.39.0-1_swh1' into debian/buster-swh (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Merge tag 'debian/0.39.0-1_swh1' into debian/buster-swh
Oct 29 2021, 11:23 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTOf6af4b4dabe8: Updated backport on buster-swh from debian/0.39.0-1_swh1 (unstable-swh) (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Updated backport on buster-swh from debian/0.39.0-1_swh1 (unstable-swh)
Oct 29 2021, 11:23 AM
anlambert accepted D6541: Deprecate /origin/branches route.

Looks good to me ! I added a couple of nitpick comments to handle before landing this.

Oct 29 2021, 11:20 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTOa085f26951da: pristine-tar data for swh-storage_0.39.0.orig.tar.gz (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
pristine-tar data for swh-storage_0.39.0.orig.tar.gz
Oct 29 2021, 11:17 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO8c63421b5828: Update upstream source from tag 'debian/upstream/0.39.0' (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Update upstream source from tag 'debian/upstream/0.39.0'
Oct 29 2021, 11:17 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO467aa2773f89: Updated debian changelog for version 0.39.0 (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
Updated debian changelog for version 0.39.0
Oct 29 2021, 11:17 AM
Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org> committed rDSTO3ce052af209f: New upstream version 0.39.0 (authored by Jenkins for Software Heritage <jenkins@jenkins-debian1.internal.softwareheritage.org>).
New upstream version 0.39.0
Oct 29 2021, 11:17 AM
vlorentz added a comment to D6578: Add support to filter files a minimum size.

Do you mean something like this?

Oct 29 2021, 11:16 AM
swh-public-ci added a comment to D6541: Deprecate /origin/branches route.

Build is green

Oct 29 2021, 10:39 AM
Harbormaster failed remote builds in B24816: Diff 23940 for D6571: Add support for a redis-based reporting for invalid mirrorred objects!
Oct 29 2021, 10:35 AM
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Build has FAILED

Oct 29 2021, 10:35 AM
posenato added a comment to T3694: Investigate svn loading failure.

Hi,
if you need to have an SVN log from the SVN server, I can ask our sysadmins to have an extract of the log.

Oct 29 2021, 10:26 AM · SVN Loader
douardda updated the diff for D6571: Add support for a redis-based reporting for invalid mirrorred objects.

"fix" mypy

Oct 29 2021, 10:26 AM
jayeshv updated the diff for D6541: Deprecate /origin/branches route.

Minor correction in doc string

Oct 29 2021, 10:24 AM
douardda added a comment to D6570: Remove now useless revision date checker in fixer.

You checked on the whole journal, right?

not yet. The shouldn't be any left, but... I'll check as soon as I have credentials for the production kafka cluster.

Oct 29 2021, 10:12 AM
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Build has FAILED

Oct 29 2021, 9:48 AM
borisbaldassari added a comment to D6395: lister: Add new maven lister.

[1] scheduler: D6520
[2] docker container: D6543 (to deploy ^ in the docker world)

So those landed. So now you should rebuild the containers [1] to have the most recent
scheduler version (after the listing is done, the scheduling of the maven origins should
happen).

Thanks for the hint. I'll first refactor the parsing loop and then run the whole process as suggested.
Will keep you posted asap.

Oct 29 2021, 8:50 AM

Oct 28 2021

stsp requested review of D6585: in cvs loader tests, use f-strings to build repository URLs.
Oct 28 2021, 7:06 PM
stsp added inline comments to D6566: test checkout of file lacking trailing \n over pserver protocol.
Oct 28 2021, 7:06 PM
stsp closed D6566: test checkout of file lacking trailing \n over pserver protocol.
Oct 28 2021, 7:02 PM
stsp committed rDLDCVSbeb7fc8a023a: test checkout of file lacking trailing \n over pserver protocol (authored by stsp).
test checkout of file lacking trailing \n over pserver protocol
Oct 28 2021, 7:02 PM
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Build has FAILED

Oct 28 2021, 6:50 PM
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Build has FAILED

Oct 28 2021, 6:45 PM
Harbormaster failed remote builds in B24812: Diff 23936 for D6571: Add support for a redis-based reporting for invalid mirrorred objects!
Oct 28 2021, 6:41 PM
swh-public-ci added a comment to D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Build has FAILED

Oct 28 2021, 6:41 PM
douardda updated the diff for D6571: Add support for a redis-based reporting for invalid mirrorred objects.

Bump the dependency on swh-journal to 0.9

Oct 28 2021, 6:40 PM
douardda closed D6584: Make the test_client_with_deserializer reliable.
Oct 28 2021, 6:33 PM
douardda committed rDJNL79d28b51ae08: Make the test_client_with_deserializer reliable (authored by douardda).
Make the test_client_with_deserializer reliable
Oct 28 2021, 6:33 PM
jayeshv retitled D6541: Deprecate /origin/branches route from WIP-Deprecate /origin/branches route to Deprecate /origin/branches route.
Oct 28 2021, 5:57 PM
jayeshv requested review of D6541: Deprecate /origin/branches route.
Oct 28 2021, 5:55 PM
Harbormaster failed to build B24809: rDSTOa5bfe5b5145b: interface: Add origin_snapshot_get_all method for rDSTOa5bfe5b5145b: interface: Add origin_snapshot_get_all method!
Oct 28 2021, 5:31 PM
anlambert created P1212 (An Untitled Masterwork).
Oct 28 2021, 5:16 PM
ardumont closed T3667: Orchestrate origins scheduling according to scheduler metrics feedback, a subtask of T2345: Improve handling of recurrent loading tasks in scheduler, as Resolved.
Oct 28 2021, 5:13 PM · Sprint 2021 01, Archive coverage, Scheduling utilities
ardumont closed T3667: Orchestrate origins scheduling according to scheduler metrics feedback as Resolved.
Oct 28 2021, 5:13 PM · System administration, Sprint 2021 01, Archive coverage, Scheduling utilities
anlambert closed D6572: interface: Add origin_snapshot_get_all method.
Oct 28 2021, 5:10 PM
anlambert committed rDSTOa5bfe5b5145b: interface: Add origin_snapshot_get_all method (authored by anlambert).
interface: Add origin_snapshot_get_all method
Oct 28 2021, 5:10 PM
ardumont accepted D6584: Make the test_client_with_deserializer reliable.
Oct 28 2021, 4:54 PM
ardumont added a comment to D6395: lister: Add new maven lister.

[1] scheduler: D6520
[2] docker container: D6543 (to deploy ^ in the docker world)

Oct 28 2021, 4:52 PM