Page MenuHomeSoftware Heritage

origin/search: Add support for search query language
ClosedPublic

Authored by KShivendu on Aug 6 2021, 3:22 PM.

Details

Summary

This diff introduces a new config parameter "enable_ql"

If it's set to yes/True, the swh-search query language can be used for the archive search (if the "use query language" checkbox is enabled)

While using the /origin/search API endpoint, "use_ql=true" can be passed to use the url_pattern as the search query language input

Diff Detail

Repository
rDWAPPS Web applications
Branch
search-language-config
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 22953
Build 35788: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 35787: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D6068 (id=21963)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit c6f8efdacbf47262fb78622c00b639b1477c298f
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    archive: Add configuration to use search query language
    
    This diff introduces a new config parameter `use_query_language`
    
    If it's set to yes/True, the swh-search query language is used for the archive search.

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/985/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/985/console

Harbormaster returned this revision to the author for changes because remote builds failed.Aug 6 2021, 3:43 PM
Harbormaster failed remote builds in B22926: Diff 21963!
vlorentz added inline comments.
swh/web/config.py
137

group it with metadata_search_backend.

And make it default to False, this is what we'll want while testing the feature. (and it will make CI go green, at least until you add tests)

  • Rename use_query_language to enable_query_language
  • Group enable_query_language with metadata_search_backend
  • Set default value of use_query_language to False

Build is green

Patch application report for D6068 (id=21983)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit aaaf936cce4769c425aa0a63fe714009ce512d88
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Mon Aug 9 19:39:05 2021 +0530

    archive: Group enable_query_language with metadata_search_backend

commit 418ca1f12b80ad5fa4527196f12ea71f043165dc
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    archive: Add configuration to use search query language
    
    Summary:
    This diff introduces a new config parameter `use_query_language`
    
    If it's set to yes/True, the swh-search query language is used for the archive search.
    
    Reviewers: #reviewers
    
    Differential Revision: https://forge.softwareheritage.org/D6068

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/986/ for more details.

swh/web/config.py
136

the comment should be on backend

swh/web/config.py
134–135

and since we're already changing this, rename metadata_search_backend to search, for consistency with the other keys (plus, we're not just using it for origin metadata anymore)

  • Add checkbox to use query langauge (Visible only if enable_ql config param = True/yes)
  • Rename metadata_search_backend to search_config
KShivendu retitled this revision from archive: Add configuration to use search query language to origin/search: Add support for search query language.Aug 10 2021, 12:50 PM
KShivendu edited the summary of this revision. (Show Details)
KShivendu edited the summary of this revision. (Show Details)
KShivendu edited the summary of this revision. (Show Details)

Build was aborted

Patch application report for D6068 (id=21989)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit 0c5c3d15e35b5b5376271a56e79822eef95ded62
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/987/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/987/console

KShivendu edited the summary of this revision. (Show Details)
  • tests: Add tests for origin search with ql
  • Fix failing tests (by fixing mocker namespaces)

Build has FAILED

Patch application report for D6068 (id=21994)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit bf1e5890d99f373ad08b3f11ae9b94518b70032c
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Wed Aug 11 12:10:48 2021 +0530

    tests: Add tests for origin search with ql

commit 0c5c3d15e35b5b5376271a56e79822eef95ded62
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/988/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/988/console

Build was aborted

Patch application report for D6068 (id=21995)

Rebasing onto 481647f863...

Current branch diff-target is up to date.
Changes applied before test
commit f74974e501a47d66f74bba1035e488aa176e354e
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Wed Aug 11 12:10:48 2021 +0530

    tests: Add tests for origin search with ql

commit 0c5c3d15e35b5b5376271a56e79822eef95ded62
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/989/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/989/console

swh/web/config.py
59–62
swh/web/templates/includes/origin-search-form.html
55

Few users will understand what this mean. Can you think of a better way to phrase it?

Build was aborted

Patch application report for D6068 (id=21995)

Rebasing onto 70fb9a97c5...

First, rewinding head to replay your work on top of it...
Applying: origin/search: Add support for search query language
Applying: tests: Add tests for origin search with ql
Changes applied before test
commit 88fe66bf25e50ace70973f0771c96ddc05a3fb57
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Wed Aug 11 12:10:48 2021 +0530

    tests: Add tests for origin search with ql

commit e8d9e0f20cef06a513e669a6c19b071d0cb08b2f
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/990/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/990/console

  • Fix failing cypress tests
  • Squash

Build is green

Patch application report for D6068 (id=22000)

Rebasing onto 70fb9a97c5...

First, rewinding head to replay your work on top of it...
Applying: origin/search: Add support for search query language
Changes applied before test
commit 53c036229d1c137ef077fbc063d5f7f5f154433b
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/991/ for more details.

  • Changes suggested by vlorentz

Build is green

Patch application report for D6068 (id=22028)

Rebasing onto 70fb9a97c5...

First, rewinding head to replay your work on top of it...
Applying: origin/search: Add support for search query language
Applying: Changes suggested by val
Changes applied before test
commit 94eb8d0362a9328ca8bb9c08e051a26b5c2b9b03
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Thu Aug 12 22:58:31 2021 +0530

    Changes suggested by val

commit 62d10325d75db60e9383e188d1f20f95f5265690
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/995/ for more details.

Thanks! Don't forget to squash the commits before landing

This revision is now accepted and ready to land.Aug 13 2021, 12:13 AM

Build is green

Patch application report for D6068 (id=22029)

Rebasing onto 70fb9a97c5...

Current branch diff-target is up to date.
Changes applied before test
commit c5259fce4f2034131843780eb0b2c430b90cc2b5
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Fri Aug 6 18:37:45 2021 +0530

    origin/search: Add support for search query language
    
    This diff introduces a new config parameter "enable_ql"
    
    If it's set to yes/True, the swh-search query language can be used for the archive search
    
    While using the /origin/search API endpoint, "use_ql=true" can be
    passed to use the url_pattern as the search query language input

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/996/ for more details.