Page MenuHomeSoftware Heritage
Differential D7164

Adapt indexer to swh.core 2.0.0
ClosedPublic
Actions

Authored by douardda on Feb 11 2022, 5:08 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T3894: Add a 'swh db upgrade' cli
Commits
rDCIDX9e89b83be455: Adapt to swh.core 2.0.0
Summary
  • add the get_datastore function in swh.indexer
  • add the get_current_version method in IndexerBackend,
  • remove dbversion management from sql init script
  • update tests accordingly

Depends on D7155
Related to T3894

Diff Detail

Unit TestsFailed
View All

TimeTest
4 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.indexer.tests.test_metadata.Metadata::test_compute_metadata_maven_invalid_encoding
self = <swh.indexer.tests.test_metadata.Metadata testMethod=test_compute_metadata_maven_invalid_encoding> def test_compute_metadata_maven_invalid_encoding(self):
4 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.indexer.codemeta::swh.indexer.codemeta.merge_values
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.indexer.metadata_dictionary.maven::swh.indexer.metadata_dictionary.maven.MavenMapping.normalize_groupId
2 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.indexer.metadata_dictionary.maven::swh.indexer.metadata_dictionary.maven.MavenMapping.parse_licenses
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.indexer.metadata_dictionary.maven::swh.indexer.metadata_dictionary.maven.MavenMapping.parse_repositories
View Full Test Results (1 Failed · 332 Passed · 15 Skipped)

Event Timeline

douardda created this revision.Feb 11 2022, 5:08 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptFeb 11 2022, 5:08 PM
swh-public-ci added a comment.Feb 11 2022, 5:09 PM

Build has FAILED

Patch application report for D7164 (id=25966)

Rebasing onto 38bae9837b...

Current branch diff-target is up to date.
Changes applied before test
commit c2758e6547ede817b1d7cf2bd05c3aa6798b56b5
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Feb 8 14:57:30 2022 +0100

    Adapt to swh.core  2.0.0
    
    - add the `get_datastore` function in `swh.indexer`
    - add the `get_current_version` method in `IndexerBackend`,
    - remove dbversion management from sql init script
    - update tests accordingly

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/189/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/189/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 11 2022, 5:09 PM
Harbormaster failed remote builds in B26842: Diff 25966!
douardda retitled this revision from Adapt to swh.core 2.0.0 to Adapt indexer to swh.core 2.0.0.Feb 11 2022, 5:13 PM
swh-public-ci added a comment.Feb 23 2022, 11:29 AM

Build has FAILED

Patch application report for D7164 (id=25966)

Rebasing onto 38bae9837b...

Current branch diff-target is up to date.
Changes applied before test
commit c2758e6547ede817b1d7cf2bd05c3aa6798b56b5
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Feb 8 14:57:30 2022 +0100

    Adapt to swh.core  2.0.0
    
    - add the `get_datastore` function in `swh.indexer`
    - add the `get_current_version` method in `IndexerBackend`,
    - remove dbversion management from sql init script
    - update tests accordingly

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/190/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/190/console

douardda updated this revision to Diff 26205.Feb 23 2022, 2:32 PM

small flake8 and mypy fixes

swh-public-ci added a comment.Feb 23 2022, 2:36 PM

Build has FAILED

Patch application report for D7164 (id=26205)

Rebasing onto 38bae9837b...

Current branch diff-target is up to date.
Changes applied before test
commit d400d5161b2c447636b6485d86f2e4d1ff5b1af0
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Feb 8 14:57:30 2022 +0100

    Adapt to swh.core  2.0.0
    
    - add the `get_datastore` function in `swh.indexer`
    - add the `get_current_version` method in `IndexerBackend`,
    - remove dbversion management from sql init script
    - update tests accordingly

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/191/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/191/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 23 2022, 2:36 PM
Harbormaster failed remote builds in B27078: Diff 26205!
douardda updated this revision to Diff 26207.Feb 23 2022, 2:52 PM

forgot to commit requirements file

swh-public-ci added a comment.Feb 23 2022, 2:56 PM

Build has FAILED

Patch application report for D7164 (id=26207)

Rebasing onto 38bae9837b...

Current branch diff-target is up to date.
Changes applied before test
commit 8cb650cab81c4067be24fcaeb41150ff81558fe7
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Feb 8 14:57:30 2022 +0100

    Adapt to swh.core  2.0.0
    
    - add the `get_datastore` function in `swh.indexer`
    - add the `get_current_version` method in `IndexerBackend`,
    - remove dbversion management from sql init script
    - update tests accordingly

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/192/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/192/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 23 2022, 2:56 PM
Harbormaster failed remote builds in B27080: Diff 26207!
douardda updated this revision to Diff 26236.Feb 24 2022, 11:13 AM

rebase

swh-public-ci added a comment.Feb 24 2022, 11:18 AM

Build is green

Patch application report for D7164 (id=26236)

Rebasing onto 12c3f4cf87...

Current branch diff-target is up to date.
Changes applied before test
commit 9fb219a65d01b32c421808d9904373f5d07ba7db
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Feb 8 14:57:30 2022 +0100

    Adapt to swh.core  2.0.0
    
    - add the `get_datastore` function in `swh.indexer`
    - add the `get_current_version` method in `IndexerBackend`,
    - remove dbversion management from sql init script
    - update tests accordingly

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/195/ for more details.

Harbormaster completed remote builds in B27109: Diff 26236.Feb 24 2022, 11:18 AM
douardda requested review of this revision.Feb 24 2022, 11:18 AM
vlorentz added a subscriber: vlorentz.Feb 24 2022, 11:21 AM
vlorentz added inline comments.
requirements-test.txt
10 ↗(On Diff #26236)

why?

ardumont accepted this revision.Feb 24 2022, 12:04 PM
ardumont added a subscriber: ardumont.

lgtm

This revision is now accepted and ready to land.Feb 24 2022, 12:04 PM
douardda added inline comments.Feb 24 2022, 12:05 PM
requirements-test.txt
10 ↗(On Diff #26236)

why?

because mypy told me so (did not dig further, just did what I was told)

douardda updated this revision to Diff 26285.Feb 24 2022, 5:23 PM

remove the dep on types-toml

seems not needed after all.

douardda marked an inline comment as done.Feb 24 2022, 5:23 PM
swh-public-ci added a comment.Feb 24 2022, 5:26 PM

Build is green

Patch application report for D7164 (id=26285)

Rebasing onto 12c3f4cf87...

Current branch diff-target is up to date.
Changes applied before test
commit 9e89b83be45582d7b709f192a83d14c7433641f2
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Feb 8 14:57:30 2022 +0100

    Adapt to swh.core  2.0.0
    
    - add the `get_datastore` function in `swh.indexer`
    - add the `get_current_version` method in `IndexerBackend`,
    - remove dbversion management from sql init script
    - update tests accordingly

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/196/ for more details.

Harbormaster completed remote builds in B27157: Diff 26285.Feb 24 2022, 5:27 PM
Closed by commit rDCIDX9e89b83be455: Adapt to swh.core 2.0.0 (authored by douardda). · Explain WhyFeb 24 2022, 5:33 PM
This revision was automatically updated to reflect the committed changes.
douardda added a commit: rDCIDX9e89b83be455: Adapt to swh.core 2.0.0.

Revision Contents
Changeset List

PathSize
sql/
upgrades/
swh/
indexer/
9 lines
sql/
13 lines
storage/
8 lines
1 line
tests/
23 lines
storage/
2 lines
6 lines
swh/
indexer/
sql/
upgrades/
sql/
upgrades/
42 lines
81 lines
11 lines
13 lines
86 lines
61 lines
10 lines
9 lines
103 lines
9 lines
41 lines
42 lines
63 lines
9 lines
119 lines
42 lines
223 lines
45 lines
184 lines

Diff 26205

View Options

sql/upgrades/115.sql

Loading...
View Options

sql/upgrades/116.sql

Loading...
View Options

sql/upgrades/117.sql

Loading...
View Options

sql/upgrades/118.sql

Loading...
View Options

sql/upgrades/119.sql

Loading...
View Options

sql/upgrades/120.sql

Loading...
View Options

sql/upgrades/121.sql

Loading...
View Options

sql/upgrades/122.sql

Loading...
View Options

sql/upgrades/123.sql

Loading...
View Options

sql/upgrades/124.sql

Loading...
View Options

sql/upgrades/125.sql

Loading...
View Options

sql/upgrades/126.sql

Loading...
View Options

sql/upgrades/127.sql

Loading...
View Options

sql/upgrades/128.sql

Loading...
View Options

sql/upgrades/129.sql

Loading...
View Options

sql/upgrades/130.sql

Loading...
View Options

sql/upgrades/131.sql

Loading...
View Options

sql/upgrades/132.sql

Loading...
View Options

sql/upgrades/133.sql

Loading...
View Options

swh/indexer/__init__.py

Loading...
View Options

swh/indexer/sql/30-schema.sql

Loading...
View Options

swh/indexer/sql/upgrades/115.sql

Loading...
View Options

swh/indexer/sql/upgrades/116.sql

Loading...
View Options

swh/indexer/sql/upgrades/117.sql

Loading...
View Options

swh/indexer/sql/upgrades/118.sql

Loading...
View Options

swh/indexer/sql/upgrades/119.sql

Loading...
View Options

swh/indexer/sql/upgrades/120.sql

Loading...
View Options

swh/indexer/sql/upgrades/121.sql

Loading...
View Options

swh/indexer/sql/upgrades/122.sql

Loading...
View Options

swh/indexer/sql/upgrades/123.sql

Loading...
View Options

swh/indexer/sql/upgrades/124.sql

Loading...
View Options

swh/indexer/sql/upgrades/125.sql

Loading...
View Options

swh/indexer/sql/upgrades/126.sql

Loading...
View Options

swh/indexer/sql/upgrades/127.sql

Loading...
View Options

swh/indexer/sql/upgrades/128.sql

Loading...
View Options

swh/indexer/sql/upgrades/129.sql

Loading...
View Options

swh/indexer/sql/upgrades/130.sql

Loading...
View Options

swh/indexer/sql/upgrades/131.sql

Loading...
View Options

swh/indexer/sql/upgrades/132.sql

Loading...
View Options

swh/indexer/sql/upgrades/133.sql

Loading...
View Options

swh/indexer/storage/__init__.py

Loading...
View Options

swh/indexer/storage/db.py

Loading...
View Options

swh/indexer/tests/conftest.py

Loading...
View Options

swh/indexer/tests/storage/__init__.py

Loading...
View Options

swh/indexer/tests/storage/conftest.py

Loading...
About · Developer information · Legal