Page MenuHomeSoftware Heritage

Drop decomissioned language indexer storage endpoints
ClosedPublic

Authored by ardumont on Fri, Jul 22, 4:42 PM.

Details

Summary

The indexer language has been no longer running in production for years (and the related
indexer code has been pruned years ago as well). Those related endpoints are not
consumed by anyone. So we can drop those.

That will ease code maintenance and make the ci gain some time when running the overall
tests.

Related to T4273

Diff Detail

Repository
rDCIDX Metadata indexer
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

commit message should mention you are only removing the idx-storage parts, not the indexer itself

swh/indexer/tests/test_fossology_license.py
70

heh

Build has FAILED

Patch application report for D8158 (id=29459)

Could not rebase; Attempt merge onto a3253a0be4...

Updating a3253a0..026d269
Fast-forward
 swh/indexer/ctags.py                            | 153 -------------
 swh/indexer/storage/__init__.py                 | 122 +---------
 swh/indexer/storage/converters.py               |  94 +-------
 swh/indexer/storage/db.py                       | 107 +--------
 swh/indexer/storage/in_memory.py                |  58 +----
 swh/indexer/storage/interface.py                | 117 +---------
 swh/indexer/storage/model.py                    |  29 +--
 swh/indexer/tasks.py                            |   8 +-
 swh/indexer/tests/storage/generate_data_test.py |  10 +-
 swh/indexer/tests/storage/test_converters.py    | 107 +--------
 swh/indexer/tests/storage/test_storage.py       | 282 +-----------------------
 swh/indexer/tests/test_ctags.py                 | 171 --------------
 swh/indexer/tests/test_fossology_license.py     |   2 +-
 swh/indexer/tests/utils.py                      |  28 ---
 swh/indexer/tests/zz_celery/test_tasks.py       |  20 +-
 15 files changed, 20 insertions(+), 1288 deletions(-)
 delete mode 100644 swh/indexer/ctags.py
 delete mode 100644 swh/indexer/tests/test_ctags.py
Changes applied before test
commit 026d269cd009521f4395a56c1dbf961f0bac40b0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jul 22 16:41:30 2022 +0200

    cleanup: Drop decomissioned language indexer related code
    
    It's been no longer running in production for years. That will ease code maintenance.
    
    Related to T4273

commit 230e89d8eeb8b11069655aa9468e37d9d7fe8ee5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jul 22 16:20:41 2022 +0200

    cleanup: Drop decomissioned ctags indexer
    
    It's been no longer running in production for years. That will ease code maintenance.
    
    Related to T4273

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/382/
See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/382/console

Harbormaster returned this revision to the author for changes because remote builds failed.Fri, Jul 22, 4:51 PM
Harbormaster failed remote builds in B30535: Diff 29459!
ardumont retitled this revision from cleanup: Drop decomissioned language indexer related code to cleanup: Drop decomissioned language indexer storage endpoint.Fri, Jul 22, 5:00 PM
ardumont edited the summary of this revision. (Show Details)
ardumont retitled this revision from cleanup: Drop decomissioned language indexer storage endpoint to Drop decomissioned language indexer storage endpoints.
ardumont edited the summary of this revision. (Show Details)
ardumont edited the summary of this revision. (Show Details)

Adapt according to suggestions

Build is green

Patch application report for D8158 (id=29460)

Could not rebase; Attempt merge onto a3253a0be4...

Updating a3253a0..b8be819
Fast-forward
 swh/indexer/ctags.py                            | 153 -------------
 swh/indexer/storage/__init__.py                 | 127 +----------
 swh/indexer/storage/converters.py               |  94 +-------
 swh/indexer/storage/db.py                       | 107 +--------
 swh/indexer/storage/in_memory.py                |  58 +----
 swh/indexer/storage/interface.py                | 117 +---------
 swh/indexer/storage/model.py                    |  29 +--
 swh/indexer/tasks.py                            |   8 +-
 swh/indexer/tests/storage/generate_data_test.py |  10 +-
 swh/indexer/tests/storage/test_converters.py    | 107 +--------
 swh/indexer/tests/storage/test_storage.py       | 282 +-----------------------
 swh/indexer/tests/test_ctags.py                 | 171 --------------
 swh/indexer/tests/test_fossology_license.py     |   2 +-
 swh/indexer/tests/utils.py                      |  28 ---
 swh/indexer/tests/zz_celery/test_tasks.py       |  20 +-
 15 files changed, 23 insertions(+), 1290 deletions(-)
 delete mode 100644 swh/indexer/ctags.py
 delete mode 100644 swh/indexer/tests/test_ctags.py
Changes applied before test
commit b8be8197ef586b4c1c50ce3d0323d5ea0f837884
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jul 22 16:41:30 2022 +0200

    Drop decomissioned language indexer storage endpoints
    
    The indexer language has been no longer running in production for years (and the related
    indexer code has been pruned years ago as well). Those related endpoints are not
    consumed by anyone. So we can drop those.
    
    That will ease code maintenance and make the ci gain some time when running the overall
    tests.
    
    Related to T4273

commit 230e89d8eeb8b11069655aa9468e37d9d7fe8ee5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jul 22 16:20:41 2022 +0200

    cleanup: Drop decomissioned ctags indexer
    
    It's been no longer running in production for years. That will ease code maintenance.
    
    Related to T4273

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/383/ for more details.

This revision was not accepted when it landed; it landed in state Needs Review.Mon, Jul 25, 4:15 PM
This revision was automatically updated to reflect the committed changes.