Page MenuHomeSoftware Heritage
Paste P830

(An Untitled Masterwork)
ActivePublic

Authored by douardda on Oct 21 2020, 2:39 PM.
swh/objstorage/misc/bench_reqrep.py --help
Usage: bench_reqrep.py [OPTIONS] CFG_FROM [CFG_TO]
Copy from one objstorage to the other. This tool is dedicated to
stress testing swh-objstorage backends.
The obj storages arguments can be given a either a file name (the
standard obj storage config file in yaml) or as an url of the form:
<backend>://?<arg>=<value>&[...]
If this url is a standard http one, it is expected to be the url of a
running swh objstorage server.
If the destination obj storage is not given, objects are dropped (for
a read only stress test).
It will produce simple performance summary as output, but is will
also push statsd metrics on the UDP endpoint given by the
STATDS_HOST/STATSD_PORT environment variable (these default to
localhost:8125), if you want to see real time curves of how the copy
is performing.
Examples:
- Generate random objects and put them in a running objstorage
listening on port 5003:
swh-objstorage-copy random:// http://127.0.0.1:5003
- Same as above, but limit the generator to 1000 fixed size objects:
swh-objstorage-copy random://?total=1000&filesize=1024
http://127.0.0.1:5003
- Pull with 16 workers x 2 threads from a local pathslicer only for
2mn:
swh-objstorage-copy -t 120 -w 16 -c 2
pathslicing://?root=/tmp/objstorage&slicing=0:4/8:10
Options:
-f, --sha1-file TEXT
-t, --duration INTEGER
-w, --workers INTEGER
-c, --threads INTEGER
-l, --log-level [CRITICAL|FATAL|ERROR|WARN|WARNING|INFO|DEBUG|NOTSET]
--message TEXT
--help Show this message and exit.