Page MenuHomeSoftware Heritage
Differential D6254

package/tests/test_utils: Remove code duplication
ClosedPublic
Actions

Authored by anlambert on Sep 14 2021, 2:00 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Commits
rDLDBASEf4fe1f30bbe7: package/tests/test_utils: Remove code duplication
Summary

Those tests implementation are quite similar so let's put generic
test code in a function and use some global variables.

Depends on D6252

Diff Detail

Repository
rDLDBASE Generic VCS/Package Loader
Branch
test-utils-remove-code-duplication
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 23663
Build 36938: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 36937: arc lint + arc unit

Event Timeline

anlambert created this revision.Sep 14 2021, 2:00 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptSep 14 2021, 2:00 PM
swh-public-ci added a comment.Sep 14 2021, 2:03 PM

Build is green

Patch application report for D6254 (id=22643)

Could not rebase; Attempt merge onto d5e54a5eea...

Updating d5e54a5..6195214
Fast-forward
 swh/loader/package/tests/test_utils.py | 116 ++++++++++++++++-----------------
 swh/loader/package/utils.py            |   8 +++
 2 files changed, 66 insertions(+), 58 deletions(-)
Changes applied before test
commit 619521480d514f73c5827d59927de560de81a61d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:38:34 2021 +0200

    package/tests/test_utils: Remove code duplication

commit a60ba533470fd4d46a753f8bc765ad706b071384
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:22:37 2021 +0200

    package/utils: Try to extract download filename from response headers
    
    Some URLs for downloading a file do not contain any filename but
    rather provide it in the "content-disposition" response header.
    
    So ensure to extract the filename from that response header when
    available to avoid possible file processing issues afterwards.

commit 0e9cd127168e3fdd7b3ee78ccb6ea55cceb3f94f
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 12:08:53 2021 +0200

    package/utils: Use download response URL to extract filename
    
    requests follows URL redirection by default for GET requests so
    update input URL to response one to ensure correct filename will
    be extracted from it.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/547/ for more details.

Harbormaster completed remote builds in B23579: Diff 22643.Sep 14 2021, 2:03 PM
anlambert requested review of this revision.Sep 14 2021, 2:03 PM
anlambert updated this revision to Diff 22647.Sep 14 2021, 2:28 PM
anlambert mentioned this in D6252: package/utils: Improve downloaded filename extraction.

Rebase

swh-public-ci added a comment.Sep 14 2021, 2:32 PM

Build is green

Patch application report for D6254 (id=22647)

Could not rebase; Attempt merge onto d5e54a5eea...

Updating d5e54a5..88fc559
Fast-forward
 swh/loader/package/tests/test_utils.py | 117 +++++++++++++++++----------------
 swh/loader/package/utils.py            |  14 ++++
 2 files changed, 73 insertions(+), 58 deletions(-)
Changes applied before test
commit 88fc559c3ce8280282f3ccab46f39ea9e8405c8f
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:38:34 2021 +0200

    package/tests/test_utils: Remove code duplication
    
    Those tests implementation are quite similar so let's put generic
    test code in a function and use some global variables.

commit a11695f9cebc49cdbe1972de8ed89b98b5fd1311
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:22:37 2021 +0200

    package/utils: Try to extract download filename from response headers
    
    Some URLs for downloading a file do not contain any filename but
    rather provide it in the "content-disposition" response header.
    
    So ensure to extract the filename from that response header when
    available to avoid possible file processing issues afterwards.

commit 0e9cd127168e3fdd7b3ee78ccb6ea55cceb3f94f
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 12:08:53 2021 +0200

    package/utils: Use download response URL to extract filename
    
    requests follows URL redirection by default for GET requests so
    update input URL to response one to ensure correct filename will
    be extracted from it.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/549/ for more details.

Harbormaster completed remote builds in B23583: Diff 22647.Sep 14 2021, 2:32 PM
ardumont accepted this revision.Sep 14 2021, 6:49 PM
This revision is now accepted and ready to land.Sep 14 2021, 6:49 PM
anlambert updated this revision to Diff 22673.Sep 15 2021, 11:01 AM

Rebase

swh-public-ci added a comment.Sep 15 2021, 11:03 AM

Build is green

Patch application report for D6254 (id=22673)

Could not rebase; Attempt merge onto d5e54a5eea...

Updating d5e54a5..fa9ee78
Fast-forward
 swh/loader/package/tests/test_utils.py | 140 +++++++++++++++++++--------------
 swh/loader/package/utils.py            |  24 ++++++
 2 files changed, 107 insertions(+), 57 deletions(-)
Changes applied before test
commit fa9ee788d28884f2d52e63ca003c2f286498c0c9
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:38:34 2021 +0200

    package/tests/test_utils: Remove code duplication
    
    Those tests implementation are quite similar so let's put generic
    test code in a function and use some global variables.

commit 716951dc7284968a49ed028bd33697da6b818675
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:22:37 2021 +0200

    package/utils: Try to extract download filename from response headers
    
    Some URLs for downloading a file do not contain any filename but
    rather provide it in the "content-disposition" response header.
    
    So ensure to extract the filename from that response header when
    available to avoid possible file processing issues afterwards.

commit 0e9cd127168e3fdd7b3ee78ccb6ea55cceb3f94f
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 12:08:53 2021 +0200

    package/utils: Use download response URL to extract filename
    
    requests follows URL redirection by default for GET requests so
    update input URL to response one to ensure correct filename will
    be extracted from it.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/551/ for more details.

Harbormaster completed remote builds in B23606: Diff 22673.Sep 15 2021, 11:03 AM
anlambert updated this revision to Diff 22686.Sep 15 2021, 1:46 PM

Rebase

swh-public-ci added a comment.Sep 15 2021, 1:52 PM

Build is green

Patch application report for D6254 (id=22686)

Could not rebase; Attempt merge onto d5e54a5eea...

Updating d5e54a5..78bdbaa
Fast-forward
 swh/loader/package/tests/test_utils.py | 140 +++++++++++++++++++--------------
 swh/loader/package/utils.py            |  24 ++++++
 2 files changed, 107 insertions(+), 57 deletions(-)
Changes applied before test
commit 78bdbaa471e21943310d57a157fd78562dca4fe6
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:38:34 2021 +0200

    package/tests/test_utils: Remove code duplication
    
    Those tests implementation are quite similar so let's put generic
    test code in a function and use some global variables.

commit cc73c630bd9be291705e34938cd146696106af29
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:22:37 2021 +0200

    package/utils: Try to extract download filename from response headers
    
    Some URLs for downloading a file do not contain any filename but
    rather provide it in the "content-disposition" response header.
    
    So ensure to extract the filename from that response header when
    available to avoid possible file processing issues afterwards.

commit 0e9cd127168e3fdd7b3ee78ccb6ea55cceb3f94f
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 12:08:53 2021 +0200

    package/utils: Use download response URL to extract filename
    
    requests follows URL redirection by default for GET requests so
    update input URL to response one to ensure correct filename will
    be extracted from it.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/553/ for more details.

Harbormaster completed remote builds in B23619: Diff 22686.Sep 15 2021, 1:52 PM
anlambert updated this revision to Diff 22731.Sep 16 2021, 10:43 AM

Rebase

swh-public-ci added a comment.Sep 16 2021, 10:46 AM

Build is green

Patch application report for D6254 (id=22731)

Could not rebase; Attempt merge onto 7329998421...

Updating 7329998..f4fe1f3
Fast-forward
 swh/loader/package/tests/test_utils.py | 140 +++++++++++++++++++--------------
 swh/loader/package/utils.py            |  24 ++++++
 2 files changed, 107 insertions(+), 57 deletions(-)
Changes applied before test
commit f4fe1f30bbe7e3f005cc4e31db91e2ab96b76025
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:38:34 2021 +0200

    package/tests/test_utils: Remove code duplication
    
    Those tests implementation are quite similar so let's put generic
    test code in a function and use some global variables.

commit 1bd1827fa27466365bee22d9e10d759667242990
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 13:22:37 2021 +0200

    package/utils: Try to extract download filename from response headers
    
    Some URLs for downloading a file do not contain any filename but
    rather provide it in the "content-disposition" response header.
    
    So ensure to extract the filename from that response header when
    available to avoid possible file processing issues afterwards.

commit 049c41f6144d9cf80462c82e4f03f2a25fc45ebb
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 14 12:08:53 2021 +0200

    package/utils: Use download response URL to extract filename
    
    requests follows URL redirection by default for GET requests so
    update input URL to response one to ensure correct filename will
    be extracted from it.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/558/ for more details.

Harbormaster completed remote builds in B23663: Diff 22731.Sep 16 2021, 10:46 AM
Closed by commit rDLDBASEf4fe1f30bbe7: package/tests/test_utils: Remove code duplication (authored by anlambert). · Explain WhySep 16 2021, 10:48 AM
This revision was automatically updated to reflect the committed changes.
anlambert added a commit: rDLDBASEf4fe1f30bbe7: package/tests/test_utils: Remove code duplication.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
loader/
package/
tests/
176 lines100%
Loading...

Diff 22731

View Options

swh/loader/package/tests/test_utils.py

Loading...
About · Developer information · Legal