Page MenuHomeSoftware Heritage

arch: Use tempfile module to create temporary directory
ClosedPublic

Authored by anlambert on Sep 13 2022, 12:09 PM.

Details

Summary

It ensures created temporary directories will be removed once they
are no longer needed.

I also added a commit setting log level to debug for URL requests.

Diff Detail

Repository
rDLS Listers
Branch
arch-lister-improvements
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 31490
Build 49259: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 49258: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8458 (id=30483)

Rebasing onto 67211adb60...

Current branch diff-target is up to date.
Changes applied before test
commit f1a1b30fd1fc3e06641b51740066257fdea48c5b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 13 12:09:13 2022 +0200

    arch: Set log level to debug for URL requests

commit a55f171ed5ddacbc6c76b1fb835ef91e18939759
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 13 12:08:02 2022 +0200

    arch: Use tempfile module to create temporary directory
    
    It ensures created temporary directories will be removed once they
    are no longer needed.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/657/ for more details.

Don't we actually want to keep it across runs? It's pretty big

Don't we actually want to keep it across runs? It's pretty big

For the listing of the official arch repositories, there is 3 tarballs referencing packages and It's about 419 MB uncompressed, so not so huge.

I also think that such lister should be stateless with regards to the data it processes in order to be accurate regarding the packages we discover at a given time.

So I would prefer to cleanup such data after lister processed them.

This revision is now accepted and ready to land.Sep 15 2022, 10:25 AM