Paths

Table of Contentst

Differential D2133

pytest_plugin: Add support for http request (debian lister related)
ClosedPublic
Actions

Authored by ardumont on Oct 12 2019, 5:12 AM.

Tags

None

Subscribers

Details

Reviewers

douardda
olasd

Group Reviewers

Commits

rDCOREb89a97dddc99: pytest_plugin: Add support for http request (debian lister related)

Summary

Add support for both http and https support.
Now a request resolves to a local file in <https?>_<hostname>, for example:

http://pypi.org ~> http_pypi.org
https://files.pythonhosted.org ~> https_files.pythonhosted.org

Related T2032

Test Plan

tox

Diff Detail

Repository

rDCORE Foundations and core functionalities

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Oct 12 2019, 5:12 AM

ardumont mentioned this in D2134: debian.lister: Add integration test which checks scheduled tasks.

Build is green
See https://jenkins.softwareheritage.org/job/DCORE/job/tox/321/ for more details.

Harbormaster completed remote builds in B8379: Diff 7165.Oct 12 2019, 5:15 AM

ardumont edited the summary of this revision. (Show Details)Oct 12 2019, 5:07 PM

ardumont mentioned this in D2135: package.debian: Re-implement debian loader within the package loader mechanism.Oct 12 2019, 7:58 PM

I'm a bit worried that this pulls http and https from the same set of files without distinction; This looks like a good catalyst for subtle single-character bugs.

Could we make the top-level directory <domain_name>_http or http_<domain_name> for http-only requests? (And maybe add support for https_<domain_name> for symmetry)

swh/core/pytest_plugin.py
172	Please use `s?` rather than the very verbose `[s]{0,1}`.

This revision now requires changes to proceed.Oct 14 2019, 4:02 PM

Could we make the top-level directory <domain_name>_http or http_<domain_name> for http-only requests?

yes, totally

(And maybe add support for https_<domain_name> for symmetry)

I'm for symmetry but I'm a bit worried about the amount of work on my pile of diffs that this implies...
(for the https part, for http only debian code is impacted so fine)

I'd rather change that after merging the package-loader branch if you don't mind.

swh/core/pytest_plugin.py
172	sure.

I have no strong opinion on this. olsad's idea might be an acceptable solution.

In D2133#50040, @ardumont wrote:

(And maybe add support for https_<domain_name> for symmetry)

I'm for symmetry but I'm a bit worried about the amount of work on my pile of diffs that this implies...
(for the https part, for http only debian code is impacted so fine)

I'd rather change that after merging the package-loader branch if you don't mind.

It's really just a mv of the top level directory; But I was really suggesting adding support, not replacing the current implementation.

It's really just a mv of the top level directory; But I was really suggesting adding support, not replacing the current implementation.

yes, I meant in terms of rebase but mostly arc diff --update... mess.

Update according to the right suggestion of not mixing http/https data folder

Build is green
See https://jenkins.softwareheritage.org/job/DCORE/job/tox/322/ for more details.

Harbormaster completed remote builds in B8444: Diff 7228.Oct 14 2019, 4:25 PM

olasd accepted this revision.Oct 14 2019, 4:52 PM

This revision is now accepted and ready to land.Oct 14 2019, 4:52 PM

I'd rather change that after merging the package-loader branch if you don't mind.

oh well, after some bike thinking, might as well do it now.

ardumont edited the summary of this revision. (Show Details)Oct 14 2019, 5:47 PM

Update to look for file in top-level directory as <http_scheme>_<hostname>

Build is green
See https://jenkins.softwareheritage.org/job/DCORE/job/tox/323/ for more details.

Harbormaster completed remote builds in B8445: Diff 7229.Oct 14 2019, 5:51 PM

Closed by commit rDCOREb89a97dddc99: pytest_plugin: Add support for http request (debian lister related) (authored by ardumont). · Explain WhyOct 14 2019, 5:52 PM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDCOREb89a97dddc99: pytest_plugin: Add support for http request (debian lister related).

ardumont mentioned this in rDLDBASE58c532fd62c5: debian: Implement debian loader with package loader mechanism.Oct 17 2019, 1:28 PM

Revision Contents
Changeset List

Path

Size

swh/

core/

pytest_plugin.py

22 lines

tests/

data/

example.com/

file.json,name=doe,firstname=jane

file.json_visit1

http_example.com/

1 line

	https_example.com/
	example.com/

file.json,name=doe,firstname=jane

file.json_visit1

fixture/

data/

example.com/

	https_example.com/
	example.com/

test_pytest_plugin.py

4 lines

Diff 7230

swh/core/pytest_plugin.py

Loading...

swh/core/tests/data/example.com/file.json

Loading...

swh/core/tests/data/example.com/file.json,name=doe,firstname=jane

Loading...

swh/core/tests/data/example.com/file.json_visit1

Loading...

swh/core/tests/data/example.com/other.json

Loading...

swh/core/tests/data/http_example.com/something.json

Loading...

swh/core/tests/data/https_example.com/file.json

Loading...

swh/core/tests/data/https_example.com/file.json,name=doe,firstname=jane

Loading...

swh/core/tests/data/https_example.com/file.json_visit1

Loading...

swh/core/tests/data/https_example.com/other.json

Loading...

swh/core/tests/fixture/data/example.com/file.json

Loading...

swh/core/tests/fixture/data/https_example.com/file.json

Loading...

swh/core/tests/test_pytest_plugin.py

Loading...