Page MenuHomeSoftware Heritage

replay: Add more debug logs
ClosedPublic

Authored by anlambert on Nov 25 2022, 4:15 PM.

Details

Summary

Add more debug logs to the replay module to ease detection of issues.
Nevertheless, as those are quite verbose, only display them when setting
debug parameter of the loader to True.

Diff Detail

Event Timeline

Build is green

Patch application report for D8886 (id=32034)

Rebasing onto bcd1e753c6...

Current branch diff-target is up to date.
Changes applied before test
commit e35f800a423a7fce94527bcbb148cfb16ef4bb38
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Nov 25 11:26:37 2022 +0100

    replay: Add more debug logs
    
    Add more debug logs to the replay module to ease detection of issues.
    Nevertheless, as those are quite verbose, only display them when setting
    debug parameter of the loader to True.

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/359/ for more details.

Could you explain why using loglevels would not work? It feels to me they are meant to solve the problem of selecting the appropriate amount of debug messages… Having a separate flag introduce quite some extra noise in the process.

In D8886#231281, @lunar wrote:

Could you explain why using loglevels would not work? It feels to me they are meant to solve the problem of selecting the appropriate amount of debug messages… Having a separate flag introduce quite some extra noise in the process.

Because these extra debug logs are VERY verbose (all svn operations are displayed for each replayed revision) and should only be used when debugging the loading of a repository that failed to load or was partially loaded.
We already have debug logs for that loader to track its global execution but as I said those extra ones are only of interest when debugging using the command:

$ swh -l DEBUG loader run svn <repo_url> debug=true
ardumont added a subscriber: ardumont.

Makes sense.

I'm wondering whether that should be named extra_debug or something (e.g. trace?).
But that'd make up for a lot more changes...

So fine as is i guess.

This revision is now accepted and ready to land.Nov 29 2022, 12:31 PM

@anlambert Looks like you should use a separate logger, so it integrates with the standard logging config, eg.

swh -l DEBUG -l swh.loader.svn.whatever:INFO loader run svn <repo_url>

to have all debug logs except from the swh.loader.svn.whatever logger

@anlambert Looks like you should use a separate logger, so it integrates with the standard logging config, eg.

swh -l DEBUG -l swh.loader.svn.whatever:INFO loader run svn <repo_url>

to have all debug logs except from the swh.loader.svn.whatever logger

I find it simpler to activate those extra logs through a loader parameter as they should not be activated by default but I want them to be debug logs.

This revision was automatically updated to reflect the committed changes.