Page MenuHomeSoftware Heritage
Differential D8048

Move mapping-specific tests to a new directory
ClosedPublic
Actions

Authored by vlorentz on Jun 29 2022, 1:47 PM.

Details

Reviewers
douardda
Group Reviewers
Reviewers
Commits
rDCIDX3074268b1b7d: Move mapping-specific tests to a new directory
Summary

We have many of those now; and keeping them all their tests in the same file
is messy

This causes these tests to run after Celery tests, which breaks them;
so this commit also renames Celery tests to make them run last.

Diff Detail

Repository
rDCIDX Metadata indexer
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Jun 29 2022, 1:47 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptJun 29 2022, 1:47 PM
vlorentz updated this revision to Diff 29036.Jun 29 2022, 1:48 PM

uncomment test

swh-public-ci added a comment.Jun 29 2022, 1:53 PM

Build is green

Patch application report for D8048 (id=29035)

Could not rebase; Attempt merge onto 1be4e184d4...

Updating 1be4e18..72f76f6
Fast-forward
 swh/indexer/metadata.py                            |   41 +-
 swh/indexer/metadata_detector.py                   |    8 +-
 swh/indexer/metadata_dictionary/base.py            |   26 +-
 swh/indexer/metadata_dictionary/npm.py             |    4 +-
 swh/indexer/metadata_dictionary/ruby.py            |    7 +-
 swh/indexer/tests/metadata_dictionary/__init__.py  |    0
 swh/indexer/tests/metadata_dictionary/test_cff.py  |  220 ++++
 .../tests/metadata_dictionary/test_codemeta.py     |  175 +++
 .../tests/metadata_dictionary/test_maven.py        |  365 ++++++
 swh/indexer/tests/metadata_dictionary/test_npm.py  |  322 +++++
 .../tests/metadata_dictionary/test_python.py       |  114 ++
 swh/indexer/tests/metadata_dictionary/test_ruby.py |  134 ++
 swh/indexer/tests/test_metadata.py                 | 1277 --------------------
 swh/indexer/tests/zz_celery/README                 |    2 +
 swh/indexer/tests/zz_celery/__init__.py            |    0
 swh/indexer/tests/{ => zz_celery}/test_tasks.py    |    2 +
 16 files changed, 1385 insertions(+), 1312 deletions(-)
 create mode 100644 swh/indexer/tests/metadata_dictionary/__init__.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_cff.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_codemeta.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_maven.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_npm.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_python.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_ruby.py
 create mode 100644 swh/indexer/tests/zz_celery/README
 create mode 100644 swh/indexer/tests/zz_celery/__init__.py
 rename swh/indexer/tests/{ => zz_celery}/test_tasks.py (99%)
Changes applied before test
commit 72f76f63f204bb09c8c433c64b622df3d11f5dc6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 13:43:14 2022 +0200

    Move mapping-specific tests to a new directory
    
    We have many of those now; and keeping them all their tests in the same file
    is messy
    
    This causes these tests to run after Celery tests, which breaks them;
    so this commit also renames Celery tests to make them run last.

commit e002b2ee66b305c98a153cc2b57088c179a3fc68
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 12:19:00 2022 +0200

    Remove given/when/then comments

commit 65edef32831949de7b8e14846ecd4fa43bc619ee
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 12:08:49 2022 +0200

    Remove SingleFileMapping from JsonMapping's base classes
    
    Extrinsic metadata indexers will not use a 'file' as input,
    but will typically use RawExtrinsicMetadata containing formats
    in JSON.

commit f7a4bf4e04b3ac4c2fa89cf9b8a5c22e5f0c4d12
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 11:01:35 2022 +0200

    Add typing to detect_metadata() and related functions

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/289/ for more details.

Harbormaster completed remote builds in B30122: Diff 29035.Jun 29 2022, 1:53 PM
swh-public-ci added a comment.Jun 29 2022, 1:54 PM

Build is green

Patch application report for D8048 (id=29036)

Could not rebase; Attempt merge onto 1be4e184d4...

Updating 1be4e18..244bf36
Fast-forward
 swh/indexer/metadata.py                            |   41 +-
 swh/indexer/metadata_detector.py                   |    8 +-
 swh/indexer/metadata_dictionary/base.py            |   26 +-
 swh/indexer/metadata_dictionary/npm.py             |    4 +-
 swh/indexer/metadata_dictionary/ruby.py            |    7 +-
 swh/indexer/tests/metadata_dictionary/__init__.py  |    0
 swh/indexer/tests/metadata_dictionary/test_cff.py  |  220 ++++
 .../tests/metadata_dictionary/test_codemeta.py     |  175 +++
 .../tests/metadata_dictionary/test_maven.py        |  365 ++++++
 swh/indexer/tests/metadata_dictionary/test_npm.py  |  322 +++++
 .../tests/metadata_dictionary/test_python.py       |  114 ++
 swh/indexer/tests/metadata_dictionary/test_ruby.py |  134 ++
 swh/indexer/tests/test_metadata.py                 | 1277 --------------------
 swh/indexer/tests/zz_celery/README                 |    2 +
 swh/indexer/tests/zz_celery/__init__.py            |    0
 swh/indexer/tests/{ => zz_celery}/test_tasks.py    |    0
 16 files changed, 1383 insertions(+), 1312 deletions(-)
 create mode 100644 swh/indexer/tests/metadata_dictionary/__init__.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_cff.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_codemeta.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_maven.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_npm.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_python.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_ruby.py
 create mode 100644 swh/indexer/tests/zz_celery/README
 create mode 100644 swh/indexer/tests/zz_celery/__init__.py
 rename swh/indexer/tests/{ => zz_celery}/test_tasks.py (100%)
Changes applied before test
commit 244bf36f55fb919f9b9da8503db309a6d816fd30
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 13:43:14 2022 +0200

    Move mapping-specific tests to a new directory
    
    We have many of those now; and keeping them all their tests in the same file
    is messy
    
    This causes these tests to run after Celery tests, which breaks them;
    so this commit also renames Celery tests to make them run last.

commit e002b2ee66b305c98a153cc2b57088c179a3fc68
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 12:19:00 2022 +0200

    Remove given/when/then comments

commit 65edef32831949de7b8e14846ecd4fa43bc619ee
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 12:08:49 2022 +0200

    Remove SingleFileMapping from JsonMapping's base classes
    
    Extrinsic metadata indexers will not use a 'file' as input,
    but will typically use RawExtrinsicMetadata containing formats
    in JSON.

commit f7a4bf4e04b3ac4c2fa89cf9b8a5c22e5f0c4d12
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 11:01:35 2022 +0200

    Add typing to detect_metadata() and related functions

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/290/ for more details.

Harbormaster completed remote builds in B30123: Diff 29036.Jun 29 2022, 1:54 PM
vlorentz requested review of this revision.Jun 29 2022, 1:54 PM
vlorentz added a child revision: D8053: Add minimal GitHub metadata mapping.Jun 29 2022, 6:08 PM
douardda accepted this revision.Jul 4 2022, 11:47 AM
douardda added a subscriber: douardda.

Don't like the zz-celery hack, but well, that's life. Did not check the new test files actually correspond to old test_metadata.py content. If this only test being moved or are there other (eg new tests) coming along? If so, please specify it in the commit message.

This revision is now accepted and ready to land.Jul 4 2022, 11:47 AM
vlorentz added a comment.Jul 4 2022, 11:48 AM
In D8048#209747, @douardda wrote:

If this only test being moved or are there other (eg new tests) coming along? If so, please specify it in the commit message.

only moved

vlorentz updated this revision to Diff 29115.Jul 4 2022, 12:07 PM

rebase

swh-public-ci added a comment.Jul 4 2022, 12:13 PM

Build is green

Patch application report for D8048 (id=29115)

Could not rebase; Attempt merge onto 1be4e184d4...

Updating 1be4e18..f000287
Fast-forward
 swh/indexer/metadata.py                            |   44 +-
 swh/indexer/metadata_detector.py                   |    8 +-
 swh/indexer/metadata_dictionary/base.py            |   26 +-
 swh/indexer/metadata_dictionary/npm.py             |    4 +-
 swh/indexer/metadata_dictionary/ruby.py            |    7 +-
 swh/indexer/tests/metadata_dictionary/__init__.py  |    0
 swh/indexer/tests/metadata_dictionary/test_cff.py  |  220 ++++
 .../tests/metadata_dictionary/test_codemeta.py     |  175 +++
 .../tests/metadata_dictionary/test_maven.py        |  365 ++++++
 swh/indexer/tests/metadata_dictionary/test_npm.py  |  322 +++++
 .../tests/metadata_dictionary/test_python.py       |  114 ++
 swh/indexer/tests/metadata_dictionary/test_ruby.py |  134 ++
 swh/indexer/tests/test_metadata.py                 | 1277 --------------------
 swh/indexer/tests/zz_celery/README                 |    2 +
 swh/indexer/tests/zz_celery/__init__.py            |    0
 swh/indexer/tests/{ => zz_celery}/test_tasks.py    |    0
 16 files changed, 1384 insertions(+), 1314 deletions(-)
 create mode 100644 swh/indexer/tests/metadata_dictionary/__init__.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_cff.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_codemeta.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_maven.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_npm.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_python.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_ruby.py
 create mode 100644 swh/indexer/tests/zz_celery/README
 create mode 100644 swh/indexer/tests/zz_celery/__init__.py
 rename swh/indexer/tests/{ => zz_celery}/test_tasks.py (100%)
Changes applied before test
commit f00028724a986a8557c2d361d76764f2df599a37
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 13:43:14 2022 +0200

    Move mapping-specific tests to a new directory
    
    We have many of those now; and keeping them all their tests in the same file
    is messy
    
    This causes these tests to run after Celery tests, which breaks them;
    so this commit also renames Celery tests to make them run last.

commit 1801806843e41352e0099042f116fae1f4611625
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 12:19:00 2022 +0200

    Remove given/when/then comments

commit 62932303c8b8a2a551697f562b0406c03b457712
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 12:08:49 2022 +0200

    Remove SingleFileMapping from JsonMapping's base classes
    
    Extrinsic metadata indexers will not use a 'file' as input,
    but will typically use RawExtrinsicMetadata containing formats
    in JSON.

commit dc035a669329272713b0b977ba2f685501e80ba0
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 11:01:35 2022 +0200

    Add typing to detect_metadata() and related functions
    
    This also moves the call to `detect_metadata()` to
    `translate_directory_intrinsic_metadata` so type annotations make more
    sense; and remove a dead/broken code branch in
    `DirectoryMetadataIndexer.index()` that was detected by mypy.

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/310/ for more details.

Harbormaster completed remote builds in B30206: Diff 29115.Jul 4 2022, 12:13 PM
vlorentz updated this revision to Diff 29118.Jul 4 2022, 12:21 PM

rebase

This revision was landed with ongoing or failed builds.Jul 4 2022, 12:29 PM
Closed by commit rDCIDX3074268b1b7d: Move mapping-specific tests to a new directory (authored by vlorentz). · Explain Why
This revision was automatically updated to reflect the committed changes.
vlorentz added a commit: rDCIDX3074268b1b7d: Move mapping-specific tests to a new directory.
swh-public-ci added a comment.Jul 4 2022, 12:33 PM

Build was aborted

Patch application report for D8048 (id=29118)

Could not rebase; Attempt merge onto 1be4e184d4...

Updating 1be4e18..3074268
Fast-forward
 swh/indexer/metadata.py                            |   52 +-
 swh/indexer/metadata_detector.py                   |    8 +-
 swh/indexer/metadata_dictionary/base.py            |   26 +-
 swh/indexer/metadata_dictionary/npm.py             |    4 +-
 swh/indexer/metadata_dictionary/ruby.py            |    7 +-
 swh/indexer/tests/metadata_dictionary/__init__.py  |    0
 swh/indexer/tests/metadata_dictionary/test_cff.py  |  220 ++++
 .../tests/metadata_dictionary/test_codemeta.py     |  175 +++
 .../tests/metadata_dictionary/test_maven.py        |  365 ++++++
 swh/indexer/tests/metadata_dictionary/test_npm.py  |  322 +++++
 .../tests/metadata_dictionary/test_python.py       |  114 ++
 swh/indexer/tests/metadata_dictionary/test_ruby.py |  134 ++
 swh/indexer/tests/test_metadata.py                 | 1277 --------------------
 swh/indexer/tests/zz_celery/README                 |    2 +
 swh/indexer/tests/zz_celery/__init__.py            |    0
 swh/indexer/tests/{ => zz_celery}/test_tasks.py    |    0
 16 files changed, 1387 insertions(+), 1319 deletions(-)
 create mode 100644 swh/indexer/tests/metadata_dictionary/__init__.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_cff.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_codemeta.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_maven.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_npm.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_python.py
 create mode 100644 swh/indexer/tests/metadata_dictionary/test_ruby.py
 create mode 100644 swh/indexer/tests/zz_celery/README
 create mode 100644 swh/indexer/tests/zz_celery/__init__.py
 rename swh/indexer/tests/{ => zz_celery}/test_tasks.py (100%)
Changes applied before test
commit 3074268b1b7d5409dc162a6125db2eb325b96add
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 13:43:14 2022 +0200

    Move mapping-specific tests to a new directory
    
    We have many of those now; and keeping them all their tests in the same file
    is messy
    
    This causes these tests to run after Celery tests, which breaks them;
    so this commit also renames Celery tests to make them run last.

commit f4d1ce835ca392213a0a9bda1cc210d2a4525a5d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 12:19:00 2022 +0200

    Remove given/when/then comments

commit e6b03db91c994fc1d680b76bf0798054a8120b79
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 12:08:49 2022 +0200

    Remove SingleFileMapping from JsonMapping's base classes
    
    Extrinsic metadata indexers will not use a 'file' as input,
    but will typically use RawExtrinsicMetadata containing formats
    in JSON.

commit 8c5a84459d4436af7b4b751e48091a99bce0a7d1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Jun 29 11:01:35 2022 +0200

    Add typing to detect_metadata() and related functions
    
    This also moves the call to `detect_metadata()` to
    `translate_directory_intrinsic_metadata` so type annotations make more
    sense; and remove a dead/broken code branch in
    `DirectoryMetadataIndexer.index()` that was detected by mypy.

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/313/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/313/console

Harbormaster failed remote builds in B30209: Diff 29118!Jul 4 2022, 12:33 PM

Revision Contents
Changeset List

PathSize
swh/
indexer/
tests/
metadata_dictionary/
220 lines
175 lines
365 lines
322 lines
114 lines
134 lines
1243 lines
zz_celery/
2 lines
zz_celery/

Diff 29121

View Options

swh/indexer/tests/metadata_dictionary/__init__.py

Loading...
View Options

swh/indexer/tests/metadata_dictionary/test_cff.py

Loading...
View Options

swh/indexer/tests/metadata_dictionary/test_codemeta.py

Loading...
View Options

swh/indexer/tests/metadata_dictionary/test_maven.py

Loading...
View Options

swh/indexer/tests/metadata_dictionary/test_npm.py

Loading...
View Options

swh/indexer/tests/metadata_dictionary/test_python.py

Loading...
View Options

swh/indexer/tests/metadata_dictionary/test_ruby.py

Loading...
View Options

swh/indexer/tests/test_metadata.py

Loading...
View Options

swh/indexer/tests/test_tasks.py

Loading...
View Options

swh/indexer/tests/zz_celery/README

Loading...
View Options

swh/indexer/tests/zz_celery/__init__.py

Loading...
View Options

swh/indexer/tests/zz_celery/test_tasks.py

Loading...
About · Developer information · Legal