Page MenuHomeSoftware Heritage
Paste P1459

replication/05-earliest-revision.sh
ActivePublic

Authored by vlorentz on Sep 21 2022, 2:29 PM.
GRAPH_VERSION="1.0.1"
GRAPH_CLASSPATH="$HOME/swh-environment/swh-graph/java/target/swh-graph-$GRAPH_VERSION.jar"
PERF_OPTS="-Xmx100G -XX:PretenureSizeThreshold=512M -XX:MaxNewSize=4G -XX:+UseLargePages -XX:+UseTransparentHugePages -XX:+UseNUMA -XX:+UseTLAB -XX:+ResizeTLAB"
GRAPH_PATH="/dev/shm/swh-graph/default/graph"
zstdcat license-blobs.csv.zst \
| pv --line-mode --size $(zstdcat license-blobs.csv.zst | wc -l) \
| sed "s/,.*//" \
| tail -n +2 \
| uniq \
| java -cp $GRAPH_CLASSPATH $PERF_OPTS org.softwareheritage.graph.utils.FindEarliestRevision $GRAPH_PATH \
| zstdcat -19 \
> blobs-earliest.csv.zst