Page MenuHomeSoftware Heritage

Reduce RAM usage for generating mapping files
Open, NormalPublic

Description

Right now a temporary array is used to correctly order elements and then write sequentially to the file. However this uses 2TB of RAM on the entire graph. Other solutions were tried in D1802 but they all were way to slow to be doable in practice.

Event Timeline

haltode created this task.Sat, Aug 10, 9:22 AM
haltode created this object in space S1 Public.
haltode triaged this task as Normal priority.
zack renamed this task from Implement mapping files dumping with less RAM usage to Reduce RAM usage for generating mapping files.Sat, Aug 10, 3:45 PM