HomeSoftware Heritage

swh.indexer.language: Improve edge cases policy

Description

swh.indexer.language: Improve edge cases policy

Before this commit, the following situation could arise:

  • no encoding detected (we cannot do much after that yet)
  • no result after multiple tryouts to compute the lang

This resulted in exceptions at runtime.

So, for now, we check those situations and we reference the sha1 with
lang None. The purpose is to permit an ulterior scheduling of those
later on (e.g. when we improve the encoding policy reuse for example).

Details

Provenance
ardumontAuthored on Jun 7 2017, 10:57 AM
ardumontPushed on Jun 7 2017, 11:09 AM
Parents
rDCIDX6ea5daa123a6: language: Improve decoding policy on bad chunking sequence
Branches
Unknown
Tags
Unknown
References
tag: v0.0.33