Page MenuHomeSoftware Heritage

common configuration file for swh graph rpc-serve, compress, …
Closed, MigratedEdits Locked

Description

swh graph compress has a configuration file (specified as swh -C conf.yml graph compress), but other sub-commands, and in particular rpc-serve don't use it and hard-code options instead. We should have a common approach to configuring swh graph features, and possibly sharing commons settings and defaults (e.g., max_ram and java_tool_options) among different sub-commands.

For reference, what compress uses is in webgraph.py, function check_config.

All configuration file keys are optional, so one doesn't actually need a conffile, but here's an example with some overrides:

graph:
  compress:
    batch_size: 1000
    java_tool_options: -Dlogback.configurationFile={logback}

As you can see interpolation is also supported.

Event Timeline

zack triaged this task as Normal priority.Nov 8 2019, 3:03 PM
zack created this task.

this has been merged into master with commit 9cece1f6722ce836cec9353b928bb4bb4b7b77e6