Page MenuHomeSoftware Heritage
Differential D2074

gnu.tests: Check gnu lister
ClosedPublic
Actions

Authored by ardumont on Oct 4 2019, 6:30 PM.

Details

Reviewers
douardda
Group Reviewers
Reviewers
Commits
rDLS4f30cb50e36c: tox.ini: Use tests installed files instead of working directory
Summary

This triggers a gnu listing and checks the output from the scheduler.

For this, a new swh-listers fixture got added.
This allows to retrieve a test ready lister from its name (e.g gnu).
Those listers have then access to a scheduler fixture to be able to
check the lister's output in the scheduler backend.

Related T2023

Test Plan

tox

Diff Detail

Repository
rDLS Listers
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 8121
Build 11706: tox-on-jenkinsJenkins
Build 11705: arc lint + arc unit

Event Timeline

ardumont created this revision.Oct 4 2019, 6:30 PM
swh-public-ci added a comment.Oct 4 2019, 6:32 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DLS/job/tox/381/
See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tox/381/console

Harbormaster failed remote builds in B8121: Diff 6969!Oct 4 2019, 6:32 PM
ardumont updated this revision to Diff 6970.Oct 4 2019, 6:47 PM

Fix

swh-public-ci added a comment.Oct 4 2019, 6:50 PM

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/382/ for more details.

Harbormaster completed remote builds in B8122: Diff 6970.Oct 4 2019, 6:50 PM
ardumont updated this revision to Diff 6972.Oct 5 2019, 12:50 PM
  • gnu.tests: Checks lister output from scheduler
  • core.models: Fix typo
swh-public-ci added a comment.Oct 5 2019, 12:53 PM

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/383/ for more details.

Harbormaster completed remote builds in B8126: Diff 6972.Oct 5 2019, 12:53 PM
ardumont retitled this revision from gnu.tests: Add basic listing check from a tree.json.gz to gnu.tests: Check gnu listing.Oct 5 2019, 12:56 PM
ardumont edited the summary of this revision. (Show Details)
ardumont added projects: Lister, Origin-GNU.
ardumont retitled this revision from gnu.tests: Check gnu listing to gnu.tests: Check gnu lister.
ardumont edited the summary of this revision. (Show Details)Oct 5 2019, 2:25 PM
ardumont added a child revision: D2075: simple_lister: Extract common behavior in base class.Oct 5 2019, 4:51 PM
ardumont added a child revision: D2077: cgit.tests: Check the tasks from the scheduler.Oct 5 2019, 6:25 PM
douardda requested changes to this revision.Oct 7 2019, 10:24 AM
douardda added inline comments.
swh/lister/gnu/tests/test_lister.py
18–25

wouldn't it be the right moment to (at least) move this in swh/lister/core/tests/conftest.py (or so)?

This revision now requires changes to proceed.Oct 7 2019, 10:24 AM
ardumont added inline comments.Oct 7 2019, 10:33 AM
swh/lister/gnu/tests/test_lister.py
18–25

I entertained the idea then i did not really know how to "properly" handle the DATADIR which is different between listers.

ardumont added inline comments.Oct 7 2019, 2:28 PM
swh/lister/gnu/tests/test_lister.py
18–25

wip in swh-core

ardumont mentioned this in T2032: listers: Add proper integration tests to lister.Oct 8 2019, 9:06 AM
ardumont mentioned this in D2082: Add pytest fixture to allow mocking requests with data file.Oct 8 2019, 10:44 AM
ardumont added inline comments.Oct 8 2019, 11:08 AM
swh/lister/gnu/tests/test_lister.py
18–25

no longer wip, it's happening -> D2082

douardda accepted this revision.Oct 8 2019, 1:39 PM
This revision is now accepted and ready to land.Oct 8 2019, 1:39 PM
ardumont updated this revision to Diff 7040.Oct 9 2019, 2:46 PM

Update diff?

Harbormaster failed remote builds in B8218: Diff 7040!Oct 9 2019, 2:46 PM
swh-public-ci added a comment.Oct 9 2019, 2:46 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DLS/job/tox/397/
See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tox/397/console

ardumont updated this revision to Diff 7041.Oct 9 2019, 2:47 PM

Include the tox.ini fix.

  • tox.ini: Use tests installed files instead of working directory
Harbormaster failed remote builds in B8219: Diff 7041!Oct 9 2019, 2:48 PM
swh-public-ci added a comment.Oct 9 2019, 2:48 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DLS/job/tox/398/
See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tox/398/console

This revision was landed with ongoing or failed builds.Oct 9 2019, 5:39 PM
Closed by commit rDLS4f30cb50e36c: tox.ini: Use tests installed files instead of working directory (authored by ardumont). · Explain Why
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDLS4f30cb50e36c: tox.ini: Use tests installed files instead of working directory.
ardumont removed a child revision: D2077: cgit.tests: Check the tasks from the scheduler.Oct 9 2019, 6:16 PM

Revision Contents
Changeset List

PathSize
swh/
lister/
gnu/
tests/
data/
ftp.gnu.org/
34 lines

Diff 6969

View Options

swh/lister/gnu/tests/data/ftp.gnu.org/tree.json.gz

Loading...
View Options

swh/lister/gnu/tests/test_lister.py

Loading...
About · Developer information · Legal