Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Maniphest Tasks

T2833: cpan.loader - archive Perl modules from CPAN

Commits

rDLS05cd1de1cde7: cpan: Fix module version extraction for some edge cases

Summary

CPAN API can return versions that are not of str type: either
int or float.

When version equals 0, it means that version failed to be parsed
by CPAN so we try to extract it from release name in that case.

Otherwise we ensure to convert the version to str type.

Related to T2833

Depends on D8615

Diff Detail

Repository

rDLS Listers

Branch

cpan-fix-version-parsing

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 32205
Build 50437: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 50436: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	4 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.cpan.lister::swh.lister.cpan.lister.get_field_value 025 026 Splits ``field_name`` on ``.``, and use it as path in the nested ``entry`` 027 dictionary. If a value does not exist, returns None.
	3 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.arch.lister::swh.lister.arch.lister.size_to_bytes
	2,016 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.arch.tests.test_lister::test_arch_lister
	504 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.arch.tests.test_tasks::test_arch_lister
	4,688 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.lister.arch.tests.test_tasks::test_arch_ping
		View Full Test Results (1 Failed · 333 Passed)

Event Timeline

anlambert created this revision.Oct 10 2022, 4:26 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptOct 10 2022, 4:26 PM

anlambert added a child revision: D8649: cpan: Use a fake origin URL instead of an HTTP one.Oct 10 2022, 4:32 PM

Build is green

Patch application report for D8648 (id=31231)

Could not rebase; Attempt merge onto 108816f232...

Updating 108816f..8d26db1
Fast-forward
 swh/lister/cpan/__init__.py                        |   8 +-
 swh/lister/cpan/lister.py                          | 144 ++++++++++--
 ...TU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw== |  50 -----
 ...NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1 |  16 --
 .../v1__search_scroll_page1                        | 247 +++++++++++++++++++++
 .../v1__search_scroll_page2                        |  39 ++++
 .../v1__search_scroll_page3                        |  85 +++++++
 .../v1__search_scroll_page4                        | 131 +++++++++++
 ...ibution__search,fields=name,size=1000,scroll=1m |  52 -----
 .../https_fastapi.metacpan.org/v1_release__search  | 246 ++++++++++++++++++++
 swh/lister/cpan/tests/test_lister.py               | 166 ++++++++++++--
 11 files changed, 1025 insertions(+), 159 deletions(-)
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page2
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page3
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page4
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_distribution__search,fields=name,size=1000,scroll=1m
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_release__search

Changes applied before test

commit 8d26db1cf78bddfb005addd2bc41fdca44fc19f4
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Oct 10 15:55:54 2022 +0200

    cpan: Fix module version extraction for some edge cases
    
    CPAN API can return versions that are not of str type: either
    int or float.
    
    When version equals 0, it means that version failed to be parsed
    by CPAN so we try to extract it from release name in that case.
    
    Otherwise we ensure to convert the version to str type.
    
    Related to T2833

commit 2177ac9f5a08c2bd276f494b2aa4c8f0d4239e65
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 27 16:34:38 2022 +0200

    cpan: Improve listing process by querying the metacpan release endpoint
    
    Instead of querying the metacpan distribution endpoint to list origins,
    prefer to use the release endpoint instead enabling to list all artifacts
    associated to CPAN packages by scrolling results.
    
    Compared to previous implementation, it enables to compute a last_update
    date for all CPAN packages but also to obtain artifact sha256 checksums
    that will be used by the CPAN loader to check downloads integrity.
    
    As the multiple versions of a module are spread across multiple pages
    from the CPAN API, origins are sent to the scheduler once all pages
    processed, it is also faster to proceed that way.
    
    Also compute extrinsic metadata URL for each perl module versions in
    order for the cpan loader to query it.
    
    Related to T2833

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/775/ for more details.

Harbormaster completed remote builds in B32184: Diff 31231.Oct 10 2022, 4:32 PM

anlambert requested review of this revision.Oct 10 2022, 4:32 PM

vlorentz accepted this revision.Oct 11 2022, 9:55 AM

vlorentz added a subscriber: vlorentz.

vlorentz added inline comments.

swh/lister/cpan/lister.py
31	or `module_version = release_name.replace(prefix, "", 1)`, to avoid accidentally replacing more than once
32–33	redundant

This revision is now accepted and ready to land.Oct 11 2022, 9:55 AM

Rebase

Build has FAILED

Patch application report for D8648 (id=31253)

Could not rebase; Attempt merge onto 108816f232...

Updating 108816f..2777809
Fast-forward
 swh/lister/cpan/__init__.py                        |   8 +-
 swh/lister/cpan/lister.py                          | 158 ++++++++++---
 ...TU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw== |  50 -----
 ...NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1 |  16 --
 .../v1__search_scroll_page1                        | 247 +++++++++++++++++++++
 .../v1__search_scroll_page2                        |  39 ++++
 .../v1__search_scroll_page3                        |  85 +++++++
 .../v1__search_scroll_page4                        | 131 +++++++++++
 ...ibution__search,fields=name,size=1000,scroll=1m |  52 -----
 .../https_fastapi.metacpan.org/v1_release__search  | 246 ++++++++++++++++++++
 swh/lister/cpan/tests/test_lister.py               | 165 ++++++++++++--
 11 files changed, 1037 insertions(+), 160 deletions(-)
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page2
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page3
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page4
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_distribution__search,fields=name,size=1000,scroll=1m
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_release__search

Changes applied before test

commit 27778090c535fa473ea08bd6f5a9e0a491de573a
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Oct 10 15:55:54 2022 +0200

    cpan: Fix module version extraction for some edge cases
    
    CPAN API can return versions that are not of str type: either
    int or float.
    
    When version equals 0, it means that version failed to be parsed
    by CPAN so we try to extract it from release name in that case.
    
    Otherwise we ensure to convert the version to str type.
    
    Related to T2833

commit 5042a43e31c091d186a7e38c36df0235f6cd65e7
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 27 16:34:38 2022 +0200

    cpan: Improve listing process by querying the metacpan release endpoint
    
    Instead of querying the metacpan distribution endpoint to list origins,
    prefer to use the release endpoint instead enabling to list all artifacts
    associated to CPAN packages by scrolling results.
    
    Compared to previous implementation, it enables to compute a last_update
    date for all CPAN packages but also to obtain artifact sha256 checksums
    that will be used by the CPAN loader to check downloads integrity.
    
    As the multiple versions of a module are spread across multiple pages
    from the CPAN API, origins are sent to the scheduler once all pages
    processed, it is also faster to proceed that way.
    
    Related to T2833

Link to build: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/778/
See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/778/console

Harbormaster failed remote builds in B32205: Diff 31253!Oct 11 2022, 2:23 PM

Rebase

Build has FAILED

Patch application report for D8648 (id=31259)

Could not rebase; Attempt merge onto 108816f232...

Updating 108816f..729d9b6
Fast-forward
 swh/lister/cpan/__init__.py                        |   8 +-
 swh/lister/cpan/lister.py                          | 158 ++++++++++---
 ...TU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw== |  50 -----
 ...NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1 |  16 --
 .../v1__search_scroll_page1                        | 247 +++++++++++++++++++++
 .../v1__search_scroll_page2                        |  39 ++++
 .../v1__search_scroll_page3                        |  85 +++++++
 .../v1__search_scroll_page4                        | 131 +++++++++++
 ...ibution__search,fields=name,size=1000,scroll=1m |  52 -----
 .../https_fastapi.metacpan.org/v1_release__search  | 246 ++++++++++++++++++++
 swh/lister/cpan/tests/test_lister.py               | 165 ++++++++++++--
 11 files changed, 1037 insertions(+), 160 deletions(-)
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page2
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page3
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page4
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_distribution__search,fields=name,size=1000,scroll=1m
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_release__search

Changes applied before test

commit 729d9b64da81df1ef2d81034b96a16b16a8d9544
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Oct 10 15:55:54 2022 +0200

    cpan: Fix module version extraction for some edge cases
    
    CPAN API can return versions that are not of str type: either
    int or float.
    
    When version equals 0, it means that version failed to be parsed
    by CPAN so we try to extract it from release name in that case.
    
    Otherwise we ensure to convert the version to str type.
    
    Related to T2833

commit 5121157ce326d32411e32f9f984f9a1f6e8710ae
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 27 16:34:38 2022 +0200

    cpan: Improve listing process by querying the metacpan release endpoint
    
    Instead of querying the metacpan distribution endpoint to list origins,
    prefer to use the release endpoint instead enabling to list all artifacts
    associated to CPAN packages by scrolling results.
    
    Compared to previous implementation, it enables to compute a last_update
    date for all CPAN packages but also to obtain artifact sha256 checksums
    that will be used by the CPAN loader to check downloads integrity.
    
    As the multiple versions of a module are spread across multiple pages
    from the CPAN API, origins are sent to the scheduler once all pages
    processed, it is also faster to proceed that way.
    
    Related to T2833

Link to build: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/780/
See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/780/console

Harbormaster failed remote builds in B32211: Diff 31259!Oct 11 2022, 3:08 PM

rebase

Build is green

Patch application report for D8648 (id=31261)

Could not rebase; Attempt merge onto 108816f232...

Updating 108816f..a64077d
Fast-forward
 swh/lister/cpan/__init__.py                        |   8 +-
 swh/lister/cpan/lister.py                          | 159 ++++++++++---
 ...TU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw== |  50 -----
 ...NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1 |  16 --
 .../v1__search_scroll_page1                        | 247 +++++++++++++++++++++
 .../v1__search_scroll_page2                        |  39 ++++
 .../v1__search_scroll_page3                        |  85 +++++++
 .../v1__search_scroll_page4                        | 131 +++++++++++
 ...ibution__search,fields=name,size=1000,scroll=1m |  52 -----
 .../https_fastapi.metacpan.org/v1_release__search  | 246 ++++++++++++++++++++
 swh/lister/cpan/tests/test_lister.py               | 165 ++++++++++++--
 11 files changed, 1038 insertions(+), 160 deletions(-)
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page2
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page3
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page4
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_distribution__search,fields=name,size=1000,scroll=1m
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_release__search

Changes applied before test

commit a64077d2251605f4aced9c2a35d649d8b7a56ef7
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Oct 10 15:55:54 2022 +0200

    cpan: Fix module version extraction for some edge cases
    
    CPAN API can return versions that are not of str type: either
    int or float.
    
    When version equals 0, it means that version failed to be parsed
    by CPAN so we try to extract it from release name in that case.
    
    Otherwise we ensure to convert the version to str type.
    
    Related to T2833

commit e09a31c4c0072ff93453215aa772a7cfcabec5f1
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 27 16:34:38 2022 +0200

    cpan: Improve listing process by querying the metacpan release endpoint
    
    Instead of querying the metacpan distribution endpoint to list origins,
    prefer to use the release endpoint instead enabling to list all artifacts
    associated to CPAN packages by scrolling results.
    
    Compared to previous implementation, it enables to compute a last_update
    date for all CPAN packages but also to obtain artifact sha256 checksums
    that will be used by the CPAN loader to check downloads integrity.
    
    As the multiple versions of a module are spread across multiple pages
    from the CPAN API, origins are sent to the scheduler once all pages
    processed, it is also faster to proceed that way.
    
    Related to T2833

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/782/ for more details.

Harbormaster completed remote builds in B32213: Diff 31261.Oct 11 2022, 3:26 PM

Build is green

Patch application report for D8648 (id=31263)

Could not rebase; Attempt merge onto 108816f232...

Updating 108816f..05cd1de
Fast-forward
 swh/lister/cpan/__init__.py                        |   8 +-
 swh/lister/cpan/lister.py                          | 158 ++++++++++---
 ...TU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw== |  50 -----
 ...NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1 |  16 --
 .../v1__search_scroll_page1                        | 247 +++++++++++++++++++++
 .../v1__search_scroll_page2                        |  39 ++++
 .../v1__search_scroll_page3                        |  85 +++++++
 .../v1__search_scroll_page4                        | 131 +++++++++++
 ...ibution__search,fields=name,size=1000,scroll=1m |  52 -----
 .../https_fastapi.metacpan.org/v1_release__search  | 246 ++++++++++++++++++++
 swh/lister/cpan/tests/test_lister.py               | 165 ++++++++++++--
 11 files changed, 1037 insertions(+), 160 deletions(-)
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll,scroll=1m,scroll_id=cXVlcnlUaGVuRmV0Y2g7Mzs5NTU1MTQ1NTk6eXptdmszQUNUam1XbVJjRjRkRk9Udzs5NTQ5NjQ5NjI6ZHZIZWxCb3BUZi1Cb3NwRDB5NmRQUTs5NTU1MTQ1NjA6eXptdmszQUNUam1XbVJjRjRkRk9UdzswOw==_visit1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page1
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page2
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page3
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page4
 delete mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_distribution__search,fields=name,size=1000,scroll=1m
 create mode 100644 swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1_release__search

Changes applied before test

commit 05cd1de1cde7ed26ca46d970e4635ba142af9031
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Oct 10 15:55:54 2022 +0200

    cpan: Fix module version extraction for some edge cases
    
    CPAN API can return versions that are not of str type: either
    int or float.
    
    When version equals 0, it means that version failed to be parsed
    by CPAN so we try to extract it from release name in that case.
    
    Otherwise we ensure to convert the version to str type.
    
    Related to T2833

commit f57b8f3a2c49080ae9bc11217b8d6ef4ed8c564e
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Tue Sep 27 16:34:38 2022 +0200

    cpan: Improve listing process by querying the metacpan release endpoint
    
    Instead of querying the metacpan distribution endpoint to list origins,
    prefer to use the release endpoint instead enabling to list all artifacts
    associated to CPAN packages by scrolling results.
    
    Compared to previous implementation, it enables to compute a last_update
    date for all CPAN packages but also to obtain artifact sha256 checksums
    that will be used by the CPAN loader to check downloads integrity.
    
    As the multiple versions of a module are spread across multiple pages
    from the CPAN API, origins are sent to the scheduler once all pages
    processed, it is also faster to proceed that way.
    
    Related to T2833

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/784/ for more details.

Harbormaster completed remote builds in B32215: Diff 31263.Oct 11 2022, 3:31 PM

Closed by commit rDLS05cd1de1cde7: cpan: Fix module version extraction for some edge cases (authored by anlambert). · Explain WhyOct 11 2022, 5:16 PM

This revision was automatically updated to reflect the committed changes.

anlambert added a commit: rDLS05cd1de1cde7: cpan: Fix module version extraction for some edge cases.

cpan: Fix module version extraction for some edge cases
ClosedPublic
Actions

Details

Diff Detail

Unit TestsFailed
View All

Event Timeline

Patch application report for D8648 (id=31231)

Changes applied before test

Patch application report for D8648 (id=31253)

Changes applied before test

Patch application report for D8648 (id=31259)

Changes applied before test

Patch application report for D8648 (id=31261)

Changes applied before test

Patch application report for D8648 (id=31263)

Changes applied before test

Revision Contents
Changeset List

Diff 31253

swh/lister/cpan/lister.py

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page1

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page2

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page3

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page4

swh/lister/cpan/tests/test_lister.py

cpan: Fix module version extraction for some edge casesClosedPublicActions

Details

Diff Detail

Unit TestsFailedView All

Event Timeline

Patch application report for D8648 (id=31231)

Changes applied before test

Patch application report for D8648 (id=31253)

Changes applied before test

Patch application report for D8648 (id=31259)

Changes applied before test

Patch application report for D8648 (id=31261)

Changes applied before test

Patch application report for D8648 (id=31263)

Changes applied before test

Revision ContentsChangeset List

Diff 31253

swh/lister/cpan/lister.py

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page1

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page2

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page3

swh/lister/cpan/tests/data/https_fastapi.metacpan.org/v1__search_scroll_page4

swh/lister/cpan/tests/test_lister.py

cpan: Fix module version extraction for some edge cases
ClosedPublic
Actions

Unit TestsFailed
View All

Revision Contents
Changeset List