Page MenuHomeSoftware Heritage
Differential D8931

Prevent incorrect warning from being printed to output files
ClosedPublic
Actions

Authored by vlorentz on Dec 7 2022, 11:40 AM.

Details

Reviewers
olasd
Group Reviewers
Reviewers
Commits
rDGRPH233b0508395a: Prevent incorrect warning from being printed to output files
Summary

I cannot find how to make WebGraph log to stderr instead of stdout,
so it seems to be the only way.

Diff Detail

Repository
rDGRPH Compressed graph representation
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Dec 7 2022, 11:40 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptDec 7 2022, 11:40 AM
swh-public-ci added a comment.Dec 7 2022, 11:45 AM

Build has FAILED

Patch application report for D8931 (id=32180)

Could not rebase; Attempt merge onto b8ddd6cead...

Updating b8ddd6c..16ec221
Fast-forward
 swh/graph/luigi/misc_datasets.py            |  3 ++-
 swh/graph/luigi/origin_contributors.py      |  4 +++-
 swh/graph/luigi/utils.py                    | 10 ++++++++++
 swh/graph/tests/test_origin_contributors.py |  2 +-
 4 files changed, 16 insertions(+), 3 deletions(-)
Changes applied before test
commit 16ec2214095dcd754d9c7e77ca6272caefb03d8d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 11:40:23 2022 +0100

    Prevent incorrect warning from being printed to output files
    
    I cannot find how to make WebGraph log to stderr instead of stdout,
    so it seems to be the only way.

commit 4f87b94a305a617ec3f8f66a7c3c6e2cef788442
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 10:54:26 2022 +0100

    Fix crash when the sensitive dataset directory does not exist

Link to build: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/320/
See console output for more information: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/320/console

Harbormaster returned this revision to the author for changes because remote builds failed.Dec 7 2022, 11:45 AM
Harbormaster failed remote builds in B33133: Diff 32180!
vlorentz updated this revision to Diff 32185.Dec 7 2022, 12:48 PM

rebase

swh-public-ci added a comment.Dec 7 2022, 12:54 PM

Build is green

Patch application report for D8931 (id=32185)

Could not rebase; Attempt merge onto 66253a872d...

Updating 66253a8..634d0e4
Fast-forward
 swh/graph/luigi/misc_datasets.py            |  3 ++-
 swh/graph/luigi/origin_contributors.py      |  4 +++-
 swh/graph/luigi/utils.py                    | 10 ++++++++++
 swh/graph/tests/test_origin_contributors.py |  2 +-
 4 files changed, 16 insertions(+), 3 deletions(-)
Changes applied before test
commit 634d0e4c56e219be20b6af112833df8d4945209a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 11:40:23 2022 +0100

    Prevent incorrect warning from being printed to output files
    
    I cannot find how to make WebGraph log to stderr instead of stdout,
    so it seems to be the only way.

commit 3f42f4fb0a4fd52291bc922de95dc09a86b4674d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 10:54:26 2022 +0100

    Fix crash when the sensitive dataset directory does not exist

See https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/325/ for more details.

Harbormaster completed remote builds in B33139: Diff 32185.Dec 7 2022, 12:54 PM
vlorentz requested review of this revision.Dec 7 2022, 12:54 PM
olasd accepted this revision.Dec 7 2022, 1:37 PM
olasd added a subscriber: olasd.

Why not just touch all the files?

I believe that webgraph uses slf4j with the logback backend, so setting the logback.configurationFile property to point to a proper logback.xml config file might work?

This revision is now accepted and ready to land.Dec 7 2022, 1:37 PM
vlorentz added a comment.Dec 7 2022, 2:43 PM
In D8931#232231, @olasd wrote:

Why not just touch all the files?

Because it would not guarantee that mtimes for .obl files is greater or equal to mtimes .offset files

Aaaand I just realized it will fail when the compressed graph is not writeable (and it often won't), so this isn't an acceptable fix

I believe that webgraph uses slf4j with the logback backend, so setting the logback.configurationFile property to point to a proper logback.xml config file might work?

I'll look into this a bit more...

vlorentz planned changes to this revision.Dec 7 2022, 2:44 PM
vlorentz updated this revision to Diff 32196.Dec 7 2022, 3:09 PM

less awful fix

This revision is now accepted and ready to land.Dec 7 2022, 3:09 PM
vlorentz updated this revision to Diff 32199.Dec 7 2022, 3:18 PM

remove useless function

swh-public-ci added a comment.Dec 7 2022, 3:19 PM

Build has FAILED

Patch application report for D8931 (id=32196)

Could not rebase; Attempt merge onto 100ccd1d1e...

Updating 100ccd1..2799da3
Fast-forward
 swh/graph/luigi/origin_contributors.py      |  1 +
 swh/graph/luigi/utils.py                    | 61 +++++++++++++++++++++++------
 swh/graph/tests/test_origin_contributors.py |  2 +-
 3 files changed, 52 insertions(+), 12 deletions(-)
Changes applied before test
commit 2799da31ccb3e3165336cbd6e4c53a5df2f09bd3
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 11:40:23 2022 +0100

    Prevent incorrect warning from being printed to output files
    
    I cannot find how to make WebGraph log to stderr instead of stdout,
    so it seems to be the only way.

commit 042af3adf5b674d99733ba8738be82fcf41151a2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 10:54:26 2022 +0100

    Fix crash when the sensitive dataset directory does not exist

Link to build: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/329/
See console output for more information: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/329/console

Harbormaster failed remote builds in B33150: Diff 32196!Dec 7 2022, 3:19 PM
vlorentz updated this revision to Diff 32203.Dec 7 2022, 3:24 PM

I'm tired

swh-public-ci added a comment.Dec 7 2022, 3:28 PM

Build has FAILED

Patch application report for D8931 (id=32199)

Could not rebase; Attempt merge onto 100ccd1d1e...

Updating 100ccd1..663a697
Fast-forward
 swh/graph/luigi/origin_contributors.py      |  1 +
 swh/graph/luigi/utils.py                    | 51 ++++++++++++++++++++++-------
 swh/graph/tests/test_origin_contributors.py |  2 +-
 3 files changed, 42 insertions(+), 12 deletions(-)
Changes applied before test
commit 663a6976b789c004ed77082f052116115d891477
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 11:40:23 2022 +0100

    Prevent incorrect warning from being printed to output files
    
    I cannot find how to make WebGraph log to stderr instead of stdout,
    so it seems to be the only way.

commit 042af3adf5b674d99733ba8738be82fcf41151a2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 10:54:26 2022 +0100

    Fix crash when the sensitive dataset directory does not exist

Link to build: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/332/
See console output for more information: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/332/console

Harbormaster failed remote builds in B33153: Diff 32199!Dec 7 2022, 3:28 PM
swh-public-ci added a comment.Dec 7 2022, 3:35 PM

Build has FAILED

Patch application report for D8931 (id=32203)

Could not rebase; Attempt merge onto 100ccd1d1e...

Updating 100ccd1..233b050
Fast-forward
 swh/graph/luigi/origin_contributors.py      |  1 +
 swh/graph/luigi/utils.py                    | 52 +++++++++++++++++++++++------
 swh/graph/tests/test_origin_contributors.py |  2 +-
 3 files changed, 43 insertions(+), 12 deletions(-)
Changes applied before test
commit 233b0508395a1f8f6c3088db5b5dc938682620cb
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 11:40:23 2022 +0100

    Prevent incorrect warning from being printed to output files
    
    I cannot find how to make WebGraph log to stderr instead of stdout,
    so it seems to be the only way.

commit 042af3adf5b674d99733ba8738be82fcf41151a2
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Dec 7 10:54:26 2022 +0100

    Fix crash when the sensitive dataset directory does not exist

Link to build: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/335/
See console output for more information: https://jenkins.softwareheritage.org/job/DGRPH/job/tests-on-diff/335/console

Harbormaster failed remote builds in B33157: Diff 32203!Dec 7 2022, 3:35 PM
This revision was landed with ongoing or failed builds.Dec 7 2022, 5:15 PM
Closed by commit rDGRPH233b0508395a: Prevent incorrect warning from being printed to output files (authored by vlorentz). · Explain Why
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDGRPH233b0508395a: Prevent incorrect warning from being printed to output files.

Revision Contents
Changeset List

PathSize
swh/
graph/
luigi/
52 lines

Diff 32214

View Options

swh/graph/luigi/utils.py

Loading...
About · Developer information · Legal