HomeSoftware Heritage

StreamingGraphView: Buffer lines before writing

This commit no longer exists in the repository. It may have been part of a branch which was deleted.

Description

StreamingGraphView: Buffer lines before writing

Most of the time is spent maxing out the CPU in the Python process.
This change has two effects:

  1. lines are joined before being encoded (instead of encoding them one-by-one)
  2. larger network packets are sent, instead of a single packet per line

I don't know which affects the performance, but overall, this is
a consistent 25 to 35% speed-up to the overall run time of
SimpleTraversalView.

Details

Provenance
vlorentzAuthored on Aug 10 2021, 12:03 PM
vlorentzPushed on Aug 12 2021, 9:30 AM
Differential Revision
D6072: StreamingGraphView: Buffer lines before writing
Build Status
Buildable 22970
Build 35810: test-and-buildJenkins console · Jenkins

Commit No Longer Exists

This commit no longer exists in the repository.