LabelMapBuilder: implementation using java quicksort + heap sort
ClosedPublic
Actions

Authored by seirl on Mar 30 2022, 6:18 PM.

Details

Reviewers

None

Group Reviewers

Commits

rDGRPH6788bc25064c: config: add max value for batch_size
rDGRPH1b8316e879c1: LabelMapBuilder: implementation using quicksort + heap sort, more efficient…
rDGRPHfcbf62c74989: test dataset: fix file/directory permissions
rDGRPH5458fc565d26: Add implementation of a parallel quick sort for 3 zipped long arrays

Summary

This is a lot more efficient in terms of disk space and processing speed than
piping nodes to GNU sort.

Diff Detail

Repository

rDGRPH Compressed graph representation

Branch

parallelquicksort3

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 28038
Build 43913: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 43912: arc lint + arc unit

Event Timeline

seirl created this revision.Mar 30 2022, 6:18 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptMar 30 2022, 6:18 PM

Build is green

Patch application report for D7475 (id=27120)

Rebasing onto 3563007e9a...

Current branch diff-target is up to date.

Changes applied before test

commit 6788bc25064cf67b4b6cb6bb2bdf80cc363b4b80
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Mar 23 02:18:51 2022 +0100

    config: add max value for batch_size

commit 1b8316e879c1d957b9c960ea1c2dc7fc84362a1a
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Fri Mar 11 21:05:37 2022 +0100

    LabelMapBuilder: implementation using quicksort + heap sort, more efficient space usage

commit fcbf62c74989f3080023f49743fddd250a174a8b
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Fri Mar 11 20:56:46 2022 +0100

    test dataset: fix file/directory permissions

commit 5458fc565d26f4259042dacfebc34b5080dec2bc
Author: Antoine Pietri <antoine.pietri1@gmail.com>
Date:   Wed Mar 9 15:16:23 2022 +0100

    Add implementation of a parallel quick sort for 3 zipped long arrays

See https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/175/ for more details.

Harbormaster completed remote builds in B28038: Diff 27120.Mar 30 2022, 6:21 PM

seirl requested review of this revision.Mar 30 2022, 6:21 PM

Going to merge this without review, it's more of a research thing at this point

This revision was not accepted when it landed; it landed in state Needs Review.Mar 30 2022, 6:36 PM

Closed by commit rDGRPH5458fc565d26: Add implementation of a parallel quick sort for 3 zipped long arrays (authored by seirl). · Explain Why

This revision was automatically updated to reflect the committed changes.

seirl added a commit: rDGRPH5458fc565d26: Add implementation of a parallel quick sort for 3 zipped long arrays.

seirl added a commit: rDGRPHfcbf62c74989: test dataset: fix file/directory permissions.

seirl added a commit: rDGRPH1b8316e879c1: LabelMapBuilder: implementation using quicksort + heap sort, more efficient….

seirl added a commit: rDGRPH6788bc25064c: config: add max value for batch_size.