Page MenuHomeSoftware Heritage

make DB/FS transactions nest properly
Closed, ResolvedPublic

Description

As @olasd discovered in T1817, it might happen that contents get added to the SQL storage, created in the FS-based object storage, but left empty. The likely cause is a crash of the VM running the object storage before the kernel has the time to flush bytes to the created content file. In other words, our hypothesis that writes to the storage (as a whole) are transactional is currently false.

A likely fix is adding a call to fdatasync() where appropriate in the object storage, before the outer DB transaction commits.

It is unclear how this will impact performances though.