HomeSoftware Heritage

StreamingGraphView: Buffer lines before writing

Description

StreamingGraphView: Buffer lines before writing

Most of the time is spent maxing out the CPU in the Python process.
This change has two effects:

  1. lines are joined before being encoded (instead of encoding them one-by-one)
  2. larger network packets are sent, instead of a single packet per line

I don't know which affects the performance, but overall, this is
a consistent 25 to 35% speed-up to the overall run time of
SimpleTraversalView.

Details

Provenance
vlorentzAuthored on Aug 10 2021, 12:03 PM
vlorentzPushed on Aug 12 2021, 9:30 AM
Differential Revision
D6072: StreamingGraphView: Buffer lines before writing
Parents
rDGRPHa48b5be5847c: cli: Fix rpc-serve to actually used the path given as argument
Branches
Unknown
Tags
Unknown
Build Status
Buildable 22970
Build 35810: test-and-buildJenkins console · Jenkins