Differential D8158

Drop decomissioned language indexer storage endpoints
ClosedPublic
Actions

Authored by ardumont on Jul 22 2022, 4:42 PM.

Details

Reviewers

None

Group Reviewers

Reviewers

Maniphest Tasks

T4273: Rewrite indexers as journal clients when relevant

Commits

rDCIDXb8be8197ef58: Drop decomissioned language indexer storage endpoints

Summary

The indexer language has been no longer running in production for years (and the related
indexer code has been pruned years ago as well). Those related endpoints are not
consumed by anyone. So we can drop those.

That will ease code maintenance and make the ci gain some time when running the overall
tests.

Related to T4273

Diff Detail

Repository

rDCIDX Metadata indexer

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Jul 22 2022, 4:42 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJul 22 2022, 4:42 PM

commit message should mention you are only removing the idx-storage parts, not the indexer itself

swh/indexer/tests/test_fossology_license.py
70	heh

Build has FAILED

Patch application report for D8158 (id=29459)

Could not rebase; Attempt merge onto a3253a0be4...

Updating a3253a0..026d269
Fast-forward
 swh/indexer/ctags.py                            | 153 -------------
 swh/indexer/storage/__init__.py                 | 122 +---------
 swh/indexer/storage/converters.py               |  94 +-------
 swh/indexer/storage/db.py                       | 107 +--------
 swh/indexer/storage/in_memory.py                |  58 +----
 swh/indexer/storage/interface.py                | 117 +---------
 swh/indexer/storage/model.py                    |  29 +--
 swh/indexer/tasks.py                            |   8 +-
 swh/indexer/tests/storage/generate_data_test.py |  10 +-
 swh/indexer/tests/storage/test_converters.py    | 107 +--------
 swh/indexer/tests/storage/test_storage.py       | 282 +-----------------------
 swh/indexer/tests/test_ctags.py                 | 171 --------------
 swh/indexer/tests/test_fossology_license.py     |   2 +-
 swh/indexer/tests/utils.py                      |  28 ---
 swh/indexer/tests/zz_celery/test_tasks.py       |  20 +-
 15 files changed, 20 insertions(+), 1288 deletions(-)
 delete mode 100644 swh/indexer/ctags.py
 delete mode 100644 swh/indexer/tests/test_ctags.py

Changes applied before test

commit 026d269cd009521f4395a56c1dbf961f0bac40b0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jul 22 16:41:30 2022 +0200

    cleanup: Drop decomissioned language indexer related code
    
    It's been no longer running in production for years. That will ease code maintenance.
    
    Related to T4273

commit 230e89d8eeb8b11069655aa9468e37d9d7fe8ee5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jul 22 16:20:41 2022 +0200

    cleanup: Drop decomissioned ctags indexer
    
    It's been no longer running in production for years. That will ease code maintenance.
    
    Related to T4273

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/382/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/382/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 22 2022, 4:51 PM

Harbormaster failed remote builds in B30535: Diff 29459!

ardumont retitled this revision from cleanup: Drop decomissioned language indexer related code to cleanup: Drop decomissioned language indexer storage endpoint.Jul 22 2022, 5:00 PM

ardumont edited the summary of this revision. (Show Details)

ardumont retitled this revision from cleanup: Drop decomissioned language indexer storage endpoint to Drop decomissioned language indexer storage endpoints.

ardumont edited the summary of this revision. (Show Details)

Adapt according to suggestions

Build is green

Patch application report for D8158 (id=29460)

Could not rebase; Attempt merge onto a3253a0be4...

Updating a3253a0..b8be819
Fast-forward
 swh/indexer/ctags.py                            | 153 -------------
 swh/indexer/storage/__init__.py                 | 127 +----------
 swh/indexer/storage/converters.py               |  94 +-------
 swh/indexer/storage/db.py                       | 107 +--------
 swh/indexer/storage/in_memory.py                |  58 +----
 swh/indexer/storage/interface.py                | 117 +---------
 swh/indexer/storage/model.py                    |  29 +--
 swh/indexer/tasks.py                            |   8 +-
 swh/indexer/tests/storage/generate_data_test.py |  10 +-
 swh/indexer/tests/storage/test_converters.py    | 107 +--------
 swh/indexer/tests/storage/test_storage.py       | 282 +-----------------------
 swh/indexer/tests/test_ctags.py                 | 171 --------------
 swh/indexer/tests/test_fossology_license.py     |   2 +-
 swh/indexer/tests/utils.py                      |  28 ---
 swh/indexer/tests/zz_celery/test_tasks.py       |  20 +-
 15 files changed, 23 insertions(+), 1290 deletions(-)
 delete mode 100644 swh/indexer/ctags.py
 delete mode 100644 swh/indexer/tests/test_ctags.py

Changes applied before test

commit b8be8197ef586b4c1c50ce3d0323d5ea0f837884
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jul 22 16:41:30 2022 +0200

    Drop decomissioned language indexer storage endpoints
    
    The indexer language has been no longer running in production for years (and the related
    indexer code has been pruned years ago as well). Those related endpoints are not
    consumed by anyone. So we can drop those.
    
    That will ease code maintenance and make the ci gain some time when running the overall
    tests.
    
    Related to T4273

commit 230e89d8eeb8b11069655aa9468e37d9d7fe8ee5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jul 22 16:20:41 2022 +0200

    cleanup: Drop decomissioned ctags indexer
    
    It's been no longer running in production for years. That will ease code maintenance.
    
    Related to T4273

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/383/ for more details.

Harbormaster completed remote builds in B30536: Diff 29460.Jul 22 2022, 5:12 PM

ardumont requested review of this revision.Jul 22 2022, 5:12 PM

ardumont mentioned this in T4409: Drop dead code regarding decomissioned Indexers.Jul 25 2022, 9:47 AM

This revision was not accepted when it landed; it landed in state Needs Review.Jul 25 2022, 4:15 PM

Closed by commit rDCIDXb8be8197ef58: Drop decomissioned language indexer storage endpoints (authored by ardumont). · Explain Why

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDCIDXb8be8197ef58: Drop decomissioned language indexer storage endpoints.