bakfill script for sha1 based ranges
ActivePublic
Actions

Authored by vsellier on Aug 17 2021, 12:33 PM.

Tags

None

Subscribers

None

	#!/bin/zsh
	# From

	setopt shwordsplit

	objtype=extid

	PARTITIONS=32
	PARAMS="--dry-run"

	mkdir -p logs/$objtype

	for i in `seq 0 $PARTITIONS`; do

	CMD=$(python3 backfill_command.py $PARTITIONS $i)

	if [ -z "$CMD" ]; then
	continue
	fi

	CMD="${CMD} ${PARAMS}"

	echo "Starting ${objtype} $CMD"
	echo " $CMD"
	SWH_CONFIG_FILENAME=/etc/softwareheritage/journal/backfill.yml stdbuf -o0 $CMD 2>&1 \| stdbuf -o0 tee >(gzip -c > logs/$objtype/$i.log.gz) &
	done

	wait

to use with P1122

bakfill script for sha1 based rangesActivePublicActions