Page MenuHomeSoftware Heritage
Differential D6068

origin/search: Add support for search query language
ClosedPublic
Actions

Authored by KShivendu on Aug 6 2021, 3:22 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Commits
rDWAPPSc5259fce4f20: origin/search: Add support for search query language
Summary

This diff introduces a new config parameter "enable_ql"

If it's set to yes/True, the swh-search query language can be used for the archive search (if the "use query language" checkbox is enabled)

While using the /origin/search API endpoint, "use_ql=true" can be passed to use the url_pattern as the search query language input

Diff Detail

Repository
rDWAPPS Web applications
Branch
search-language-config
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 22957
Build 35794: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 35793: arc lint + arc unit

Unit TestsFailed
View All

TimeTest
7 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.api.views.test_origin::Tests / Python tests / test_api_origin_search_use_ql
api_client = <rest_framework.test.APIClient object at 0x7ff15b1e6588> mocker = <pytest_mock.plugin.MockerFixture object at 0x7ff15b1e68d0>
1,598 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.common.test_archive::Tests / Python tests / test_search_origin_use_ql
mocker = <pytest_mock.plugin.MockerFixture object at 0x7ff15a7827f0> @given(origin())
216 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::Tests / Python tests / test_accept_pending_save_request
259 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::Tests / Python tests / test_add_authorized_origin_url
116 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::Tests / Python tests / test_add_unauthorized_origin_url
View Full Test Results (2 Failed · 648 Passed · 5 Skipped)

Event Timeline

KShivendu created this revision.Aug 6 2021, 3:22 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptAug 6 2021, 3:22 PM
swh-public-ci added a comment.Aug 6 2021, 3:43 PM

Build has FAILED

Patch application report for D6068 (id=21963)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit c6f8efdacbf47262fb78622c00b639b1477c298f
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    archive: Add configuration to use search query language
    
    This diff introduces a new config parameter `use_query_language`
    
    If it's set to yes/True, the swh-search query language is used for the archive search.

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/985/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/985/console

Harbormaster returned this revision to the author for changes because remote builds failed.Aug 6 2021, 3:43 PM
Harbormaster failed remote builds in B22926: Diff 21963!
vlorentz added a subscriber: vlorentz.Aug 9 2021, 1:22 PM
vlorentz added inline comments.
swh/web/config.py
137

group it with metadata_search_backend.

And make it default to False, this is what we'll want while testing the feature. (and it will make CI go green, at least until you add tests)

KShivendu updated this revision to Diff 21983.Aug 9 2021, 4:11 PM
  • Rename use_query_language to enable_query_language
  • Group enable_query_language with metadata_search_backend
  • Set default value of use_query_language to False
swh-public-ci added a comment.Aug 9 2021, 4:26 PM

Build is green

Patch application report for D6068 (id=21983)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit aaaf936cce4769c425aa0a63fe714009ce512d88
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Mon Aug 9 19:39:05 2021 +0530

    archive: Group enable_query_language with metadata_search_backend

commit 418ca1f12b80ad5fa4527196f12ea71f043165dc
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    archive: Add configuration to use search query language
    
    Summary:
    This diff introduces a new config parameter `use_query_language`
    
    If it's set to yes/True, the swh-search query language is used for the archive search.
    
    Reviewers: #reviewers
    
    Differential Revision: https://forge.softwareheritage.org/D6068

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/986/ for more details.

Harbormaster completed remote builds in B22945: Diff 21983.Aug 9 2021, 4:26 PM
KShivendu requested review of this revision.Aug 9 2021, 4:26 PM
vlorentz added inline comments.Aug 10 2021, 9:52 AM
swh/web/config.py
136

the comment should be on backend

vlorentz added inline comments.Aug 10 2021, 9:53 AM
swh/web/config.py
134–135

and since we're already changing this, rename metadata_search_backend to search, for consistency with the other keys (plus, we're not just using it for origin metadata anymore)

KShivendu updated this revision to Diff 21989.Aug 10 2021, 12:50 PM
  • Add checkbox to use query langauge (Visible only if enable_ql config param = True/yes)
  • Rename metadata_search_backend to search_config
KShivendu retitled this revision from archive: Add configuration to use search query language to origin/search: Add support for search query language.Aug 10 2021, 12:50 PM
KShivendu edited the summary of this revision. (Show Details)
KShivendu edited the summary of this revision. (Show Details)
KShivendu edited the summary of this revision. (Show Details)
swh-public-ci added a comment.Aug 10 2021, 1:11 PM

Build was aborted

Patch application report for D6068 (id=21989)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit 0c5c3d15e35b5b5376271a56e79822eef95ded62
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/987/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/987/console

Harbormaster failed remote builds in B22953: Diff 21989!Aug 10 2021, 1:11 PM
KShivendu updated this revision to Diff 21994.Aug 11 2021, 8:41 AM
KShivendu edited the summary of this revision. (Show Details)
  • tests: Add tests for origin search with ql
KShivendu updated this revision to Diff 21995.Aug 11 2021, 8:58 AM
  • Fix failing tests (by fixing mocker namespaces)
swh-public-ci added a comment.Aug 11 2021, 9:03 AM

Build has FAILED

Patch application report for D6068 (id=21994)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit bf1e5890d99f373ad08b3f11ae9b94518b70032c
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Wed Aug 11 12:10:48 2021 +0530

    tests: Add tests for origin search with ql

commit 0c5c3d15e35b5b5376271a56e79822eef95ded62
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/988/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/988/console

Harbormaster failed remote builds in B22957: Diff 21994!Aug 11 2021, 9:03 AM
swh-public-ci added a comment.Aug 11 2021, 9:24 AM

Build was aborted

Patch application report for D6068 (id=21995)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit f74974e501a47d66f74bba1035e488aa176e354e
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Wed Aug 11 12:10:48 2021 +0530

    tests: Add tests for origin search with ql

commit 0c5c3d15e35b5b5376271a56e79822eef95ded62
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/989/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/989/console

Harbormaster failed remote builds in B22958: Diff 21995!Aug 11 2021, 9:24 AM
vlorentz added inline comments.Aug 11 2021, 10:09 AM
swh/web/config.py
59–62
swh/web/templates/includes/origin-search-form.html
55

Few users will understand what this mean. Can you think of a better way to phrase it?

swh-public-ci added a comment.Aug 11 2021, 1:28 PM

Build was aborted

Patch application report for D6068 (id=21995)

Rebasing onto 70fb9a97c5...

First, rewinding head to replay your work on top of it...
Applying: origin/search: Add support for search query language
Applying: tests: Add tests for origin search with ql
Changes applied before test
commit 88fe66bf25e50ace70973f0771c96ddc05a3fb57
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Wed Aug 11 12:10:48 2021 +0530

    tests: Add tests for origin search with ql

commit e8d9e0f20cef06a513e669a6c19b071d0cb08b2f
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/990/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/990/console

KShivendu updated this revision to Diff 22000.Aug 11 2021, 3:50 PM
  • Fix failing cypress tests
  • Squash
swh-public-ci added a comment.Aug 11 2021, 4:04 PM

Build is green

Patch application report for D6068 (id=22000)

Rebasing onto 70fb9a97c5...

First, rewinding head to replay your work on top of it...
Applying: origin/search: Add support for search query language
Changes applied before test
commit 53c036229d1c137ef077fbc063d5f7f5f154433b
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/991/ for more details.

Harbormaster completed remote builds in B22963: Diff 22000.Aug 11 2021, 4:04 PM
KShivendu updated this revision to Diff 22028.Aug 12 2021, 7:30 PM
  • Changes suggested by vlorentz
swh-public-ci added a comment.Aug 12 2021, 7:44 PM

Build is green

Patch application report for D6068 (id=22028)

Rebasing onto 70fb9a97c5...

First, rewinding head to replay your work on top of it...
Applying: origin/search: Add support for search query language
Applying: Changes suggested by val
Changes applied before test
commit 94eb8d0362a9328ca8bb9c08e051a26b5c2b9b03
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Thu Aug 12 22:58:31 2021 +0530

    Changes suggested by val

commit 62d10325d75db60e9383e188d1f20f95f5265690
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/995/ for more details.

Harbormaster completed remote builds in B22985: Diff 22028.Aug 12 2021, 7:44 PM
vlorentz accepted this revision.Aug 13 2021, 12:13 AM

Thanks! Don't forget to squash the commits before landing

This revision is now accepted and ready to land.Aug 13 2021, 12:13 AM
KShivendu updated this revision to Diff 22029.Aug 13 2021, 6:38 AM
  • Rebase and Squash
swh-public-ci added a comment.Aug 13 2021, 6:52 AM

Build is green

Patch application report for D6068 (id=22029)

Rebasing onto 70fb9a97c5...

Current branch diff-target is up to date.
Changes applied before test
commit c5259fce4f2034131843780eb0b2c430b90cc2b5
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/996/ for more details.

Harbormaster completed remote builds in B22986: Diff 22029.Aug 13 2021, 6:52 AM
Closed by commit rDWAPPSc5259fce4f20: origin/search: Add support for search query language (authored by KShivendu). · Explain WhyAug 13 2021, 7:17 AM
This revision was automatically updated to reflect the committed changes.
KShivendu added a commit: rDWAPPSc5259fce4f20: origin/search: Add support for search query language.

Revision Contents
Changeset List

PathSize
assets/
src/
bundles/
browse/
8 lines
swh/
web/
api/
views/
3 lines
browse/
8 lines
common/
27 lines
13 lines
templates/
includes/
9 lines
tests/
api/
views/
31 lines
common/
32 lines

Diff 21994

View Options

assets/src/bundles/browse/origin-search.js

Loading...
View Options

swh/web/api/views/origin.py

Loading...
View Options

swh/web/browse/urls.py

Loading...
View Options

swh/web/common/archive.py

Loading...
View Options

swh/web/config.py

Loading...
View Options

swh/web/templates/includes/origin-search-form.html

Loading...
View Options

swh/web/tests/api/views/test_origin.py

Loading...
View Options

swh/web/tests/common/test_archive.py

Loading...
About · Developer information · Legal