arch: Use tempfile module to create temporary directory
ClosedPublic
Actions

Authored by anlambert on Sep 13 2022, 12:09 PM.

Details

Reviewers

Group Reviewers

Commits

rDLSf1a1b30fd1fc: arch: Set log level to debug for URL requests
rDLSa55f171ed5dd: arch: Use tempfile module to create temporary directory

Summary

It ensures created temporary directories will be removed once they
are no longer needed.

I also added a commit setting log level to debug for URL requests.

Diff Detail

Repository

rDLS Listers

Branch

arch-lister-improvements

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 31490
Build 49259: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 49258: arc lint + arc unit

Event Timeline

anlambert created this revision.Sep 13 2022, 12:09 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptSep 13 2022, 12:09 PM

Build is green

Patch application report for D8458 (id=30483)

Rebasing onto 67211adb60...

Current branch diff-target is up to date.

Changes applied before test

commit f1a1b30fd1fc3e06641b51740066257fdea48c5b
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 13 12:09:13 2022 +0200

    arch: Set log level to debug for URL requests

commit a55f171ed5ddacbc6c76b1fb835ef91e18939759
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 13 12:08:02 2022 +0200

    arch: Use tempfile module to create temporary directory
    
    It ensures created temporary directories will be removed once they
    are no longer needed.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/657/ for more details.

Harbormaster completed remote builds in B31490: Diff 30483.Sep 13 2022, 12:14 PM

anlambert requested review of this revision.Sep 13 2022, 12:14 PM

Don't we actually want to keep it across runs? It's pretty big

In D8458#219763, @vlorentz wrote:

Don't we actually want to keep it across runs? It's pretty big

For the listing of the official arch repositories, there is 3 tarballs referencing packages and It's about 419 MB uncompressed, so not so huge.

I also think that such lister should be stateless with regards to the data it processes in order to be accurate regarding the packages we discover at a given time.

So I would prefer to cleanup such data after lister processed them.

vlorentz accepted this revision.Sep 15 2022, 10:25 AM

This revision is now accepted and ready to land.Sep 15 2022, 10:25 AM

Closed by commit rDLSa55f171ed5dd: arch: Use tempfile module to create temporary directory (authored by anlambert). · Explain WhySep 15 2022, 10:49 AM

This revision was automatically updated to reflect the committed changes.

anlambert added a commit: rDLSa55f171ed5dd: arch: Use tempfile module to create temporary directory.

anlambert added a commit: rDLSf1a1b30fd1fc: arch: Set log level to debug for URL requests.