diff --git a/PKG-INFO b/PKG-INFO index 35f2d91..b9d6de0 100644 --- a/PKG-INFO +++ b/PKG-INFO @@ -1,90 +1,90 @@ Metadata-Version: 2.1 Name: swh.search -Version: 0.12.1 +Version: 0.13.0 Summary: Software Heritage search service Home-page: https://forge.softwareheritage.org/diffusion/DSEA Author: Software Heritage developers Author-email: swh-devel@inria.fr License: UNKNOWN Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest Project-URL: Funding, https://www.softwareheritage.org/donate Project-URL: Source, https://forge.softwareheritage.org/source/swh-search Project-URL: Documentation, https://docs.softwareheritage.org/devel/swh-search/ Platform: UNKNOWN Classifier: Programming Language :: Python :: 3 Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3) Classifier: Operating System :: OS Independent Classifier: Development Status :: 3 - Alpha Requires-Python: >=3.7 Description-Content-Type: text/markdown Provides-Extra: testing License-File: LICENSE License-File: AUTHORS swh-search ========== Search service for the Software Heritage archive. It is similar to swh-storage in what it contains, but provides different ways to query it: while swh-storage is mostly a key-value store that returns an object from a primary key, swh-search is focused on reverse indices, to allow finding objects that match some criteria; for example full-text search. Currently uses ElasticSearch, and provides only origin search (by URL and metadata) ## Dependencies - Python tests for this module include tests that cannot be run without a local ElasticSearch instance, so you need the ElasticSearch server executable on your machine (no need to have a running ElasticSearch server). - Debian-like host The elasticsearch package is required. As it's not part of debian-stable, [another debian repository is required to be configured](https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html#deb-repo) - Non Debian-like host The tests expect: - `/usr/share/elasticsearch/jdk/bin/java` to exist. - `org.elasticsearch.bootstrap.Elasticsearch` to be in java's classpath. - Emscripten is required for generating tree-sitter WASM module. The following commands need to be executed for the setup: ```bash cd /opt && git clone https://github.com/emscripten-core/emsdk.git && cd emsdk && \ ./emsdk install latest && ./emsdk activate latest PATH="${PATH}:/opt/emsdk/upstream/emscripten" ``` **Note:** If emsdk isn't found in the PATH, the tree-sitter cli automatically pulls `emscripten/emsdk` image from docker hub when `make ts-build-wasm` or `make ts-build` is used. ## Make targets Below is the list of available make targets that can be executed from the root directory of swh-search in order to build and/or execute the swh-search under various configurations: * **ts-install**: Install node_modules and emscripten SDK required for TreeSitter * **ts-generate**: Generate parser files(C and JSON) from the grammar * **ts-repl**: Starts a web based playground for the TreeSitter grammar. It's the recommended way for developing TreeSitter grammar. * **ts-dev**: Parse the `query_language/sample_query` and print the corresponding syntax expression along with the start and end positions of all the nodes. * **ts-dev sanitize=1**: Same as **ts-dev** but without start and end position of the nodes. This format is expected by TreeSitter's native test command. `sanitize=1` cleans the output of **ts-dev** using `sed` to achieve the desired format. * **ts-test**: executes TreeSitter's native tests * **ts-build-so**: Generates `swh_ql.so` file from the previously generated parser using py-tree-sitter * **ts-build-so**: Generates `swh_ql.wasm` file from the previously generated parser using emscripten * **ts-build**: Executes both **ts-build-so** and **ts-build-so** diff --git a/docs/query-language.rst b/docs/query-language.rst index ed6623a..11dba04 100644 --- a/docs/query-language.rst +++ b/docs/query-language.rst @@ -1,190 +1,190 @@ Search Query Language ===================== Every query is composed of filters separated by ``and`` or ``or``. These filters have 3 components in the order : ``Name Operator Value`` Some of the examples are : - * ``origin = django and language in [python] and visits >= 5`` + * ``origin : plasma and language in [python] and visits >= 5`` * ``last_revision > 2020-01-01 and limit = 10`` * ``last_visit > 2021-01-01 or last_visit < 2020-01-01`` - * ``visited = false and metadata = "kubernetes" or origin = "minikube"`` + * ``visited = false and metadata = "kubernetes" or origin : "minikube"`` * ``keyword in ["orchestration", "kubectl"] and language in ["go", "rust"]`` - * ``(origin = debian or visit_type = ["deb"]) and license in ["GPL-3"]`` + * ``(origin : debian or visit_type = ["deb"]) and license in ["GPL-3"]`` **Note**: * Whitespaces are optional between the three components of a filter. * The conjunction operators have left precedence. Therefore ``foo and bar and baz`` means ``(foo and bar) and baz`` * ``and`` has higher precedence than ``or``. Therefore ``foo or bar and baz`` means ``foo or (bar and baz)`` * Precedence can be overridden using parentheses: ``(`` and ``)``. For example, you can override the default precedence in the previous query as: ``(foo or bar) and baz`` * To actually search for ``and`` or ``or`` as strings, just put them within quotes. Example : ``metadata : "vcs history and metadata"``, or even just ``metadata : "and"`` to search for the string ``and`` in the metadata The filters have been classified based on the type of value that they expects. Pattern filters --------------- Returns origins having the given keywords in their url or intrinsic metadata * Name: * ``origin``: Keywords from the origin url * ``metadata``: Keywords from all the intrinsic metadata fields - * Operator: ``=`` + * Operator: ``:`` * Value: String wrapped in quotation marks(``"`` or ``'``) **Note:** If a string has no whitespace then the quotation marks become optional. **Examples:** - * ``origin = https://github.com/Django/django`` - * ``origin = kubernetes`` - * ``origin = "github python"`` - * ``metadata = orchestration`` - * ``metadata = "javascript language"`` + * ``origin : https://github.com/Django/django`` + * ``origin : kubernetes`` + * ``origin : "github python"`` + * ``metadata : orchestration`` + * ``metadata : "javascript language"`` Boolean filters --------------- Returns origins having their boolean type values equal to given values * Name: ``visited`` : Whether the origin has been visited * Operator: ``=`` * Value: ``true`` or ``false`` **Examples:** * ``visited = true`` * ``visited = false`` Numeric filters --------------- Returns origins having their numeric type values in the given range * Name: ``visits`` : Number of visits of an origin * Operator: ``<`` ``<=`` ``=`` ``!=`` ``>`` ``>=`` * Value: Positive integer **Examples:** * ``visits > 2`` * ``visits = 5`` * ``visits <= 10`` Un-bounded List filters ----------------------- Returns origins that satisfy the criteria based on a given list * Name: * ``language`` : Programming languages used * ``license`` : License used * ``keyword`` : keywords (often same as tags) or description (includes README) from the metadata * Operator: ``in`` ``not in`` * Value: Array of strings **Note:** * If a string has no whitespace then the quotation marks become optional. * The ``keyword`` filter gives more priority to the keywords field of intrinsic metadata than the description field. So origins having the queried term in their intrinsic metadata keyword will appear first. **Examples:** * ``language in [python, js]`` * ``license in ["GPL 3.0 or later", MIT]`` * ``keyword in ["Software Heritage", swh]`` Bounded List filters -------------------- Returns origins that satisfy the criteria based on a list of fixed options **visit_type** * Name: ``visit_type`` : Returns only origins with at least one of the specified visit types * Operator: ``=`` * Value: Array of the following values ``any`` ``cran`` ``deb`` ``deposit`` ``ftp`` ``hg`` ``git`` ``nixguix`` ``npm`` ``pypi`` ``svn`` ``tar`` **sort_by** * Name: ``sort_by`` : Sorts origins based on the given list of origin attributes * Operator: ``=`` * Value: Array of the following values ``visits`` ``last_visit`` ``last_eventful_visit`` ``last_revision`` ``last_release`` ``created`` ``modified`` ``published`` **Examples:** * ``visit_type = [svn, npm]`` * ``visit_type = [nixguix, "ftp"]`` * ``sort_by = ["last_visit", created]`` * ``sort_by = [visits, modified]`` Date filters ------------ Returns origins having their date type values in the given range * Name: * ``last_visit`` : Latest visit date * ``last_eventful_visit`` : Latest visit date where a new snapshot was detected * ``last_revision`` : Latest commit date * ``last_release`` : Latest release date * ``created`` Creation date * ``modified`` Modification date * ``published`` Published date * Operator: ``<`` ``<=`` ``=`` ``!=`` ``>`` ``>=`` * Value: Date in ``Standard ISO`` format **Note:** The last three date filters are based on metadata that has to be manually entered by the repository authors. So they might not be correct or up-to-date. **Examples:** * ``last_visit > 2001-01-01 and last_visit < 2101-01-01`` * ``last_revision = "2000-01-01 18:35Z"`` * ``last_release != "2021-07-17T18:35:00Z"`` * ``created <= "2021-07-17 18:35"`` Limit filter ------------ Limits the number of results to at most N * Name: ``limit`` * Operator: ``=`` * Value: Positive Integer **Note:** The default value of the limit is 50 **Examples:** * ``limit = 1`` * ``limit = 15`` diff --git a/setup.py b/setup.py index 0661ba5..2126810 100755 --- a/setup.py +++ b/setup.py @@ -1,188 +1,206 @@ #!/usr/bin/env python3 # Copyright (C) 2015-2020 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU General Public License version 3, or any later version # See top-level LICENSE file for more information from distutils.cmd import Command from distutils.command.build import build +import glob from io import open import os import shutil import subprocess from setuptools import find_packages, setup from setuptools.command.develop import develop from setuptools.command.sdist import sdist here = os.path.abspath(os.path.dirname(__file__)) # Get the long description from the README file with open(os.path.join(here, "README.md"), encoding="utf-8") as f: long_description = f.read() def parse_requirements(name=None): if name: reqf = "requirements-%s.txt" % name else: reqf = "requirements.txt" requirements = [] if not os.path.exists(reqf): return requirements with open(reqf) as f: for line in f.readlines(): line = line.strip() if not line or line.startswith("#"): continue requirements.append(line) return requirements +def needs_regen(dest, sources) -> bool: + """Returns whether any of the 'sources' files was modified after 'dst'.""" + if not os.path.exists(dest): + return True + + dest_mtime = os.stat(dest).st_mtime + + for source in sources: + if os.stat(source).st_mtime > dest_mtime: + return True + + return False + + yarn = os.environ.get("YARN", "yarnpkg" if shutil.which("yarnpkg") else "yarn") class TSCommand(Command): user_options = [] def initialize_options(self): pass def finalize_options(self): pass class TSInstallCommand(TSCommand): description = "Installs node_modules related to query language" def run(self): subprocess.run([yarn, "install"], check=True) class TSBuildSoCommand(TSCommand): description = "Builds swh_ql.so" def initialize_options(self): self.build_lib = None super().initialize_options() def finalize_options(self): self.set_undefined_options("build", ("build_lib", "build_lib")) super().finalize_options() def run(self): ql_dir = os.path.join(self.build_lib, "swh/search/query_language") copy_ql_tree(ql_dir) - if not os.path.exists(os.path.join(ql_dir, "src/parser.c")): + if needs_regen( + os.path.join(ql_dir, "src/parser.c"), + glob.glob("swh/search/query_language/**/*"), + ): print("parser.c missing from build dir.") self.run_command("ts_install") generate_parser(ql_dir) static_dir = os.path.join(self.build_lib, "swh/search/static") os.makedirs(static_dir, exist_ok=True) # This import cannot be toplevel, as setuptools installs it after the script # starts running from tree_sitter import Language Language.build_library(os.path.join(static_dir, "swh_ql.so"), [ql_dir]) print("swh_ql.so file generated") class TSBuildCommand(TSCommand): description = "Builds swh_ql.so" def run(self): self.run_command("ts_build_so") class custom_build(build): def run(self): super().run() if not self.dry_run: self.run_command("ts_build") class custom_sdist(sdist): def make_release_tree(self, base_dir, files): super().make_release_tree(base_dir, files) dist_ql_path = os.path.join(base_dir, "swh/search/query_language") if not self.dry_run: self.run_command("ts_install") copy_ql_tree(dist_ql_path) generate_parser(dist_ql_path) class custom_develop(develop): def run(self): super().run() if not self.dry_run: self.run_command("ts_install") generate_parser("swh/search/query_language") def copy_ql_tree(dest_path): # FIXME: setuptools should copy this itself... print("Copying parser files") if os.path.exists(dest_path): shutil.rmtree(dest_path) shutil.copytree("swh/search/query_language", dest_path) def generate_parser(dest_path): print("Getting path") path = subprocess.check_output([yarn, "bin"]).decode().strip() env = {**os.environ, "PATH": os.pathsep.join([path, os.environ["PATH"]])} print("Generating") subprocess.run(["tree-sitter", "generate", "--no-bindings"], cwd=dest_path, env=env) setup( name="swh.search", description="Software Heritage search service", long_description=long_description, long_description_content_type="text/markdown", python_requires=">=3.7", author="Software Heritage developers", author_email="swh-devel@inria.fr", url="https://forge.softwareheritage.org/diffusion/DSEA", packages=find_packages(), # packages's modules install_requires=parse_requirements() + parse_requirements("swh"), tests_require=parse_requirements("test"), entry_points=""" [swh.cli.subcommands] search=swh.search.cli """, setup_requires=["setuptools-scm", "tree-sitter"], use_scm_version=True, extras_require={"testing": parse_requirements("test")}, include_package_data=True, classifiers=[ "Programming Language :: Python :: 3", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Operating System :: OS Independent", "Development Status :: 3 - Alpha", ], project_urls={ "Bug Reports": "https://forge.softwareheritage.org/maniphest", "Funding": "https://www.softwareheritage.org/donate", "Source": "https://forge.softwareheritage.org/source/swh-search", "Documentation": "https://docs.softwareheritage.org/devel/swh-search/", }, cmdclass={ "build": custom_build, "sdist": custom_sdist, "develop": custom_develop, "ts_install": TSInstallCommand, "ts_build_so": TSBuildSoCommand, "ts_build": TSBuildCommand, }, zip_safe=False, ) diff --git a/swh.search.egg-info/PKG-INFO b/swh.search.egg-info/PKG-INFO index 35f2d91..b9d6de0 100644 --- a/swh.search.egg-info/PKG-INFO +++ b/swh.search.egg-info/PKG-INFO @@ -1,90 +1,90 @@ Metadata-Version: 2.1 Name: swh.search -Version: 0.12.1 +Version: 0.13.0 Summary: Software Heritage search service Home-page: https://forge.softwareheritage.org/diffusion/DSEA Author: Software Heritage developers Author-email: swh-devel@inria.fr License: UNKNOWN Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest Project-URL: Funding, https://www.softwareheritage.org/donate Project-URL: Source, https://forge.softwareheritage.org/source/swh-search Project-URL: Documentation, https://docs.softwareheritage.org/devel/swh-search/ Platform: UNKNOWN Classifier: Programming Language :: Python :: 3 Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3) Classifier: Operating System :: OS Independent Classifier: Development Status :: 3 - Alpha Requires-Python: >=3.7 Description-Content-Type: text/markdown Provides-Extra: testing License-File: LICENSE License-File: AUTHORS swh-search ========== Search service for the Software Heritage archive. It is similar to swh-storage in what it contains, but provides different ways to query it: while swh-storage is mostly a key-value store that returns an object from a primary key, swh-search is focused on reverse indices, to allow finding objects that match some criteria; for example full-text search. Currently uses ElasticSearch, and provides only origin search (by URL and metadata) ## Dependencies - Python tests for this module include tests that cannot be run without a local ElasticSearch instance, so you need the ElasticSearch server executable on your machine (no need to have a running ElasticSearch server). - Debian-like host The elasticsearch package is required. As it's not part of debian-stable, [another debian repository is required to be configured](https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html#deb-repo) - Non Debian-like host The tests expect: - `/usr/share/elasticsearch/jdk/bin/java` to exist. - `org.elasticsearch.bootstrap.Elasticsearch` to be in java's classpath. - Emscripten is required for generating tree-sitter WASM module. The following commands need to be executed for the setup: ```bash cd /opt && git clone https://github.com/emscripten-core/emsdk.git && cd emsdk && \ ./emsdk install latest && ./emsdk activate latest PATH="${PATH}:/opt/emsdk/upstream/emscripten" ``` **Note:** If emsdk isn't found in the PATH, the tree-sitter cli automatically pulls `emscripten/emsdk` image from docker hub when `make ts-build-wasm` or `make ts-build` is used. ## Make targets Below is the list of available make targets that can be executed from the root directory of swh-search in order to build and/or execute the swh-search under various configurations: * **ts-install**: Install node_modules and emscripten SDK required for TreeSitter * **ts-generate**: Generate parser files(C and JSON) from the grammar * **ts-repl**: Starts a web based playground for the TreeSitter grammar. It's the recommended way for developing TreeSitter grammar. * **ts-dev**: Parse the `query_language/sample_query` and print the corresponding syntax expression along with the start and end positions of all the nodes. * **ts-dev sanitize=1**: Same as **ts-dev** but without start and end position of the nodes. This format is expected by TreeSitter's native test command. `sanitize=1` cleans the output of **ts-dev** using `sed` to achieve the desired format. * **ts-test**: executes TreeSitter's native tests * **ts-build-so**: Generates `swh_ql.so` file from the previously generated parser using py-tree-sitter * **ts-build-so**: Generates `swh_ql.wasm` file from the previously generated parser using emscripten * **ts-build**: Executes both **ts-build-so** and **ts-build-so** diff --git a/swh/search/elasticsearch.py b/swh/search/elasticsearch.py index a0ca953..5cc0451 100644 --- a/swh/search/elasticsearch.py +++ b/swh/search/elasticsearch.py @@ -1,555 +1,555 @@ # Copyright (C) 2019-2021 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU General Public License version 3, or any later version # See top-level LICENSE file for more information import base64 from collections import Counter import logging import pprint from textwrap import dedent from typing import Any, Dict, Iterable, List, Optional from elasticsearch import Elasticsearch, helpers import msgpack from swh.indexer import codemeta from swh.model import model from swh.model.hashutil import hash_to_hex from swh.search.interface import ( SORT_BY_OPTIONS, MinimalOriginDict, OriginDict, PagedResult, ) from swh.search.metrics import send_metric, timed from swh.search.translator import Translator from swh.search.utils import escape, get_expansion, parse_and_format_date logger = logging.getLogger(__name__) INDEX_NAME_PARAM = "index" READ_ALIAS_PARAM = "read_alias" WRITE_ALIAS_PARAM = "write_alias" ORIGIN_DEFAULT_CONFIG = { INDEX_NAME_PARAM: "origin", READ_ALIAS_PARAM: "origin-read", WRITE_ALIAS_PARAM: "origin-write", } def _sanitize_origin(origin): origin = origin.copy() # Whitelist fields to be saved in Elasticsearch res = {"url": origin.pop("url")} for field_name in ( "blocklisted", "has_visits", "intrinsic_metadata", "visit_types", "nb_visits", "snapshot_id", "last_visit_date", "last_eventful_visit_date", "last_revision_date", "last_release_date", ): if field_name in origin: res[field_name] = origin.pop(field_name) # Run the JSON-LD expansion algorithm # # to normalize the Codemeta metadata. # This is required as Elasticsearch will needs each field to have a consistent # type across documents to be searchable; and non-expanded JSON-LD documents # can have various types in the same field. For example, all these are # equivalent in JSON-LD: # * {"author": "Jane Doe"} # * {"author": ["Jane Doe"]} # * {"author": {"@value": "Jane Doe"}} # * {"author": [{"@value": "Jane Doe"}]} # and JSON-LD expansion will convert them all to the last one. if "intrinsic_metadata" in res: intrinsic_metadata = res["intrinsic_metadata"] for date_field in ["dateCreated", "dateModified", "datePublished"]: if date_field in intrinsic_metadata: date = intrinsic_metadata[date_field] # If date{Created,Modified,Published} value isn't parsable # It gets rejected and isn't stored (unlike other fields) formatted_date = parse_and_format_date(date) if formatted_date is None: intrinsic_metadata.pop(date_field) else: intrinsic_metadata[date_field] = formatted_date res["intrinsic_metadata"] = codemeta.expand(intrinsic_metadata) return res def token_encode(index_to_tokenize: Dict[bytes, Any]) -> str: """Tokenize as string an index page result from a search""" page_token = base64.b64encode(msgpack.dumps(index_to_tokenize)) return page_token.decode() def token_decode(page_token: str) -> Dict[bytes, Any]: """Read the page_token""" return msgpack.loads(base64.b64decode(page_token.encode()), raw=True) class ElasticSearch: def __init__(self, hosts: List[str], indexes: Dict[str, Dict[str, str]] = {}): self._backend = Elasticsearch(hosts=hosts) self._translator = Translator() # Merge current configuration with default values origin_config = indexes.get("origin", {}) self.origin_config = {**ORIGIN_DEFAULT_CONFIG, **origin_config} def _get_origin_index(self) -> str: return self.origin_config[INDEX_NAME_PARAM] def _get_origin_read_alias(self) -> str: return self.origin_config[READ_ALIAS_PARAM] def _get_origin_write_alias(self) -> str: return self.origin_config[WRITE_ALIAS_PARAM] @timed def check(self): return self._backend.ping() def deinitialize(self) -> None: """Removes all indices from the Elasticsearch backend""" self._backend.indices.delete(index="*") def initialize(self) -> None: """Declare Elasticsearch indices, aliases and mappings""" if not self._backend.indices.exists(index=self._get_origin_index()): self._backend.indices.create(index=self._get_origin_index()) if not self._backend.indices.exists_alias(name=self._get_origin_read_alias()): self._backend.indices.put_alias( index=self._get_origin_index(), name=self._get_origin_read_alias() ) if not self._backend.indices.exists_alias(name=self._get_origin_write_alias()): self._backend.indices.put_alias( index=self._get_origin_index(), name=self._get_origin_write_alias() ) self._backend.indices.put_mapping( index=self._get_origin_index(), body={ "dynamic_templates": [ { "booleans_as_string": { # All fields stored as string in the metadata # even the booleans "match_mapping_type": "boolean", "path_match": "intrinsic_metadata.*", "mapping": {"type": "keyword"}, } } ], "date_detection": False, "properties": { # sha1 of the URL; used as the document id "sha1": {"type": "keyword", "doc_values": True,}, # Used both to search URLs, and as the result to return # as a response to queries "url": { "type": "text", # To split URLs into token on any character # that is not alphanumerical "analyzer": "simple", # 2-gram and partial-3-gram search (ie. with the end of the # third word potentially missing) "fields": { "as_you_type": { "type": "search_as_you_type", "analyzer": "simple", } }, }, "visit_types": {"type": "keyword"}, # used to filter out origins that were never visited "has_visits": {"type": "boolean",}, "nb_visits": {"type": "integer"}, "snapshot_id": {"type": "keyword"}, "last_visit_date": {"type": "date"}, "last_eventful_visit_date": {"type": "date"}, "last_release_date": {"type": "date"}, "last_revision_date": {"type": "date"}, "intrinsic_metadata": { "type": "nested", "properties": { "@context": { # don't bother indexing tokens in these URIs, as the # are used as namespaces "type": "keyword", }, "http://schema": { "properties": { "org/dateCreated": { "properties": {"@value": {"type": "date",}} }, "org/dateModified": { "properties": {"@value": {"type": "date",}} }, "org/datePublished": { "properties": {"@value": {"type": "date",}} }, } }, }, }, # Has this origin been taken down? "blocklisted": {"type": "boolean",}, }, }, ) @timed def flush(self) -> None: self._backend.indices.refresh(index=self._get_origin_write_alias()) @timed def origin_update(self, documents: Iterable[OriginDict]) -> None: write_index = self._get_origin_write_alias() documents = map(_sanitize_origin, documents) documents_with_sha1 = ( (hash_to_hex(model.Origin(url=document["url"]).id), document) for document in documents ) # painless script that will be executed when updating an origin document update_script = dedent( """ // utility function to get and parse date ZonedDateTime getDate(def ctx, String date_field) { String default_date = "0001-01-01T00:00:00Z"; String date = ctx._source.getOrDefault(date_field, default_date); return ZonedDateTime.parse(date); } // backup current visit_types field value List visit_types = ctx._source.getOrDefault("visit_types", []); int nb_visits = ctx._source.getOrDefault("nb_visits", 0); ZonedDateTime last_visit_date = getDate(ctx, "last_visit_date"); String snapshot_id = ctx._source.getOrDefault("snapshot_id", ""); ZonedDateTime last_eventful_visit_date = getDate(ctx, "last_eventful_visit_date"); ZonedDateTime last_revision_date = getDate(ctx, "last_revision_date"); ZonedDateTime last_release_date = getDate(ctx, "last_release_date"); // update origin document with new field values ctx._source.putAll(params); // restore previous visit types after visit_types field overriding if (ctx._source.containsKey("visit_types")) { for (int i = 0; i < visit_types.length; ++i) { if (!ctx._source.visit_types.contains(visit_types[i])) { ctx._source.visit_types.add(visit_types[i]); } } } // Undo overwrite if incoming nb_visits is smaller if (ctx._source.containsKey("nb_visits")) { int incoming_nb_visits = ctx._source.getOrDefault("nb_visits", 0); if(incoming_nb_visits < nb_visits){ ctx._source.nb_visits = nb_visits; } } // Undo overwrite if incoming last_visit_date is older if (ctx._source.containsKey("last_visit_date")) { ZonedDateTime incoming_last_visit_date = getDate(ctx, "last_visit_date"); int difference = // returns -1, 0 or 1 incoming_last_visit_date.compareTo(last_visit_date); if(difference < 0){ ctx._source.last_visit_date = last_visit_date; } } // Undo update of last_eventful_date and snapshot_id if // snapshot_id hasn't changed OR incoming_last_eventful_visit_date is older if (ctx._source.containsKey("snapshot_id")) { String incoming_snapshot_id = ctx._source.getOrDefault("snapshot_id", ""); ZonedDateTime incoming_last_eventful_visit_date = getDate(ctx, "last_eventful_visit_date"); int difference = // returns -1, 0 or 1 incoming_last_eventful_visit_date.compareTo(last_eventful_visit_date); if(snapshot_id == incoming_snapshot_id || difference < 0){ ctx._source.snapshot_id = snapshot_id; ctx._source.last_eventful_visit_date = last_eventful_visit_date; } } // Undo overwrite if incoming last_revision_date is older if (ctx._source.containsKey("last_revision_date")) { ZonedDateTime incoming_last_revision_date = getDate(ctx, "last_revision_date"); int difference = // returns -1, 0 or 1 incoming_last_revision_date.compareTo(last_revision_date); if(difference < 0){ ctx._source.last_revision_date = last_revision_date; } } // Undo overwrite if incoming last_release_date is older if (ctx._source.containsKey("last_release_date")) { ZonedDateTime incoming_last_release_date = getDate(ctx, "last_release_date"); // returns -1, 0 or 1 int difference = incoming_last_release_date.compareTo(last_release_date); if(difference < 0){ ctx._source.last_release_date = last_release_date; } } """ # noqa ) actions = [ { "_op_type": "update", "_id": sha1, "_index": write_index, "scripted_upsert": True, "upsert": {**document, "sha1": sha1,}, "retry_on_conflict": 10, "script": { "source": update_script, "lang": "painless", "params": document, }, } for (sha1, document) in documents_with_sha1 ] indexed_count, errors = helpers.bulk(self._backend, actions, index=write_index) assert isinstance(errors, List) # Make mypy happy send_metric("document:index", count=indexed_count, method_name="origin_update") send_metric( "document:index_error", count=len(errors), method_name="origin_update" ) @timed def origin_search( self, *, query: str = "", url_pattern: Optional[str] = None, metadata_pattern: Optional[str] = None, with_visit: bool = False, visit_types: Optional[List[str]] = None, min_nb_visits: int = 0, min_last_visit_date: str = "", min_last_eventful_visit_date: str = "", min_last_revision_date: str = "", min_last_release_date: str = "", min_date_created: str = "", min_date_modified: str = "", min_date_published: str = "", programming_languages: Optional[List[str]] = None, licenses: Optional[List[str]] = None, keywords: Optional[List[str]] = None, sort_by: Optional[List[str]] = None, page_token: Optional[str] = None, limit: int = 50, ) -> PagedResult[MinimalOriginDict]: query_clauses: List[Dict[str, Any]] = [] query_filters = [] if url_pattern: - query_filters.append(f"origin = {escape(url_pattern)}") + query_filters.append(f"origin : {escape(url_pattern)}") if metadata_pattern: - query_filters.append(f"metadata = {escape(metadata_pattern)}") + query_filters.append(f"metadata : {escape(metadata_pattern)}") # if not query_clauses: # raise ValueError( # "At least one of url_pattern and metadata_pattern must be provided." # ) if with_visit: query_filters.append(f"visited = {'true' if with_visit else 'false'}") if min_nb_visits: query_filters.append(f"visits >= {min_nb_visits}") if min_last_visit_date: query_filters.append( f"last_visit >= {min_last_visit_date.replace('Z', '+00:00')}" ) if min_last_eventful_visit_date: query_filters.append( "last_eventful_visit >= " f"{min_last_eventful_visit_date.replace('Z', '+00:00')}" ) if min_last_revision_date: query_filters.append( f"last_revision >= {min_last_revision_date.replace('Z', '+00:00')}" ) if min_last_release_date: query_filters.append( f"last_release >= {min_last_release_date.replace('Z', '+00:00')}" ) if keywords: query_filters.append(f"keyword in {escape(keywords)}") if licenses: query_filters.append(f"license in {escape(licenses)}") if programming_languages: query_filters.append(f"language in {escape(programming_languages)}") if min_date_created: query_filters.append( f"created >= {min_date_created.replace('Z', '+00:00')}" ) if min_date_modified: query_filters.append( f"modified >= {min_date_modified.replace('Z', '+00:00')}" ) if min_date_published: query_filters.append( f"published >= {min_date_published.replace('Z', '+00:00')}" ) if visit_types is not None: query_filters.append(f"visit_type = {escape(visit_types)}") combined_filters = " and ".join(query_filters) if combined_filters and query: query = f"{combined_filters} and {query}" else: query = combined_filters or query parsed_query = self._translator.parse_query(query) query_clauses.append(parsed_query["filters"]) field_map = { "visits": "nb_visits", "last_visit": "last_visit_date", "last_eventful_visit": "last_eventful_visit_date", "last_revision": "last_revision_date", "last_release": "last_release_date", "created": "date_created", "modified": "date_modified", "published": "date_published", } if "sortBy" in parsed_query: if sort_by is None: sort_by = [] for sort_by_option in parsed_query["sortBy"]: if sort_by_option[0] == "-": sort_by.append("-" + field_map[sort_by_option[1:]]) else: sort_by.append(field_map[sort_by_option]) if parsed_query.get("limit", 0): limit = parsed_query["limit"] sorting_params: List[Dict[str, Any]] = [] if sort_by: for field in sort_by: order = "asc" if field and field[0] == "-": field = field[1:] order = "desc" if field in ["date_created", "date_modified", "date_published"]: sorting_params.append( { get_expansion(field, "."): { "nested_path": "intrinsic_metadata", "order": order, } } ) elif field in SORT_BY_OPTIONS: sorting_params.append({field: order}) sorting_params.extend( [{"_score": "desc"}, {"sha1": "asc"},] ) body = { "query": { "bool": { "must": query_clauses, "must_not": [{"term": {"blocklisted": True}}], } }, "sort": sorting_params, } if page_token: # TODO: use ElasticSearch's scroll API? page_token_content = token_decode(page_token) body["search_after"] = [ page_token_content[b"score"], page_token_content[b"sha1"].decode("ascii"), ] if logger.isEnabledFor(logging.DEBUG): formatted_body = pprint.pformat(body) logger.debug("Search query body: %s", formatted_body) res = self._backend.search( index=self._get_origin_read_alias(), body=body, size=limit ) hits = res["hits"]["hits"] next_page_token: Optional[str] = None if len(hits) == limit: # There are more results after this page; return a pagination token # to get them in a future query last_hit = hits[-1] next_page_token_content = { b"score": last_hit["_score"], b"sha1": last_hit["_source"]["sha1"], } next_page_token = token_encode(next_page_token_content) assert len(hits) <= limit return PagedResult( results=[{"url": hit["_source"]["url"]} for hit in hits], next_page_token=next_page_token, ) def visit_types_count(self) -> Counter: body = { "aggs": { "not_blocklisted": { "filter": {"bool": {"must_not": [{"term": {"blocklisted": True}}]}}, "aggs": { "visit_types": {"terms": {"field": "visit_types", "size": 1000}} }, } } } res = self._backend.search( index=self._get_origin_read_alias(), body=body, size=0 ) buckets = ( res.get("aggregations", {}) .get("not_blocklisted", {}) .get("visit_types", {}) .get("buckets", []) ) return Counter({bucket["key"]: bucket["doc_count"] for bucket in buckets}) diff --git a/swh/search/query_language/grammar.js b/swh/search/query_language/grammar.js index 594a934..7dca22d 100644 --- a/swh/search/query_language/grammar.js +++ b/swh/search/query_language/grammar.js @@ -1,192 +1,193 @@ // Copyright (C) 2021 The Software Heritage developers // See the AUTHORS file at the top-level directory of this distribution // License: GNU General Public License version 3, or any later version // See top-level LICENSE file for more information const { visitTypeField, sortByField, limitField } = require("./tokens.js"); const { patternFields, booleanFields, numericFields, listFields, dateFields } = require("./tokens.js"); -const { equalOp, rangeOp, choiceOp } = require("./tokens.js"); +const { equalOp, containOp, rangeOp, choiceOp } = require("./tokens.js"); const { sortByOptions, visitTypeOptions } = require("./tokens.js"); const { OR, AND, TRUE, FALSE } = require("./tokens.js"); const PRECEDENCE = { or: 2, and: 3, bracket: 4, } module.exports = grammar({ name: 'swh_search_ql', rules: { query: $ => seq( $.filters, optional(seq( optional($.and), choice( seq($.sortBy, optional($.and), optional($.limit)), seq($.limit, optional($.and), optional($.sortBy)), ), )) ), filters: $ => choice( prec.left(PRECEDENCE.and, seq( field('left', $.filters), field('operator', $.and), field('right', $.filters), ) ), prec.left(PRECEDENCE.or, seq( field('left', $.filters), field('operator', $.or), field('right', $.filters), ) ), prec.left(PRECEDENCE.bracket, seq("(", $.filters, ")"), ), $.filter ), sortBy: $ => annotateFilter($.sortByField, $.sortByOp, $.sortByVal), sortByField: $ => token(sortByField), sortByOp: $ => $.equalOp, sortByVal: $ => createArray(optionalWrapWith($.sortByOptions, ["'", '"'])), sortByOptions: $ => seq( optional('-'), choice(...sortByOptions) ), limit: $ => annotateFilter($.limitField, $.equalOp, $.number), limitField: $ => token(limitField), filter: $ => field('category', choice( $.patternFilter, $.booleanFilter, $.numericFilter, $.boundedListFilter, $.unboundedListFilter, $.dateFilter )), patternFilter: $ => annotateFilter($.patternField, $.patternOp, $.patternVal), patternField: $ => token(choice(...patternFields)), - patternOp: $ => $.equalOp, + patternOp: $ => $.containOp, patternVal: $ => $.string, booleanFilter: $ => annotateFilter($.booleanField, $.booleanOp, $.booleanVal), booleanField: $ => token(choice(...booleanFields)), booleanOp: $ => $.equalOp, booleanVal: $ => choice($.booleanTrue, $.booleanFalse), numericFilter: $ => annotateFilter($.numericField, $.numericOp, $.numberVal), numericField: $ => token(choice(...numericFields)), numericOp: $ => $.rangeOp, numberVal: $ => $.number, // Array members must be from the given options boundedListFilter: $ => choice($.visitTypeFilter), visitTypeFilter: $ => annotateFilter($.visitTypeField, $.visitTypeOp, $.visitTypeVal), visitTypeField: $ => token(visitTypeField), visitTypeOp: $ => $.equalOp, visitTypeVal: $ => createArray(optionalWrapWith($.visitTypeOptions, ["'", '"'])), visitTypeOptions: $ => choice(...visitTypeOptions), // TODO: fetch visitTypeOptions choices dynamically from other swh services? // Array members can be any string unboundedListFilter: $ => annotateFilter($.listField, $.listOp, $.listVal), listField: $ => token(choice(...listFields)), listOp: $ => $.choiceOp, listVal: $ => createArray($.string), dateFilter: $ => annotateFilter($.dateField, $.dateOp, $.dateVal), dateField: $ => token(choice(...dateFields)), dateOp: $ => $.rangeOp, dateVal: $ => $.isoDateTime, rangeOp: $ => token(choice(...rangeOp)), equalOp: $ => token(choice(...equalOp)), + containOp: $ => token(choice(...containOp)), choiceOp: $ => token(choice(...choiceOp)), isoDateTime: $ => { const dateRegex = (/\d{4}[-]\d{2}[-]\d{2}/).source const dateTimeSepRegex = (/(\s|T)*/).source - const timeRegex = (/(\d{2}:\d{2}(:\d{2}(\.\d{6})?)?)?/).source - const timezoneRegex = (/(\+\d{2}:\d{2}|Z)?/).source - return new RegExp(dateRegex + dateTimeSepRegex + timeRegex + timezoneRegex) + const timeRegex = (/\d{2}:\d{2}(:\d{2}(\.\d{6})?)?/).source + const timezoneRegex = (/\+\d{2}:\d{2}|Z/).source + return new RegExp(`${dateRegex}(${dateTimeSepRegex}${timeRegex}(${timezoneRegex})?)?`) }, string: $ => choice(wrapWith($.stringContent, ["'", '"']), $.singleWord), number: $ => /\d+/, booleanTrue: $ => TRUE, booleanFalse: $ => FALSE, or: $ => OR, and: $ => AND, singleWord: $ => /[^\s"'\[\]\(\),]+/, // Based on tree-sitter-json grammar: stringContent: $ => repeat1(choice( token.immediate(/[^\\'"\n]+/), $.escape_sequence )), escape_sequence: $ => token.immediate(seq( '\\', /(\"|\'|\\|\/|b|n|r|t|u)/ )), } }); function joinBySep1(rule, sep) { // At least one repetition of the rule separated by `sep` return seq(rule, repeat(seq(sep, optional(rule)))) } function joinBySep(rule, sep = ",") { // Any number of repetitions of the rule separated by `sep` return optional(joinBySep1(rule, sep)) } function createArray(rule) { // An array having `rule` as its member return seq( "[", joinBySep( field('array_member', rule), "," ), "]" ) } function wrapWith(rule, wrappers = ["'", '"']) { // The rule must be wrapped with one of the wrappers const wrappedRules = wrappers.map(wrapper => seq(wrapper, rule, wrapper)) return choice(...wrappedRules) } function optionalWrapWith(rule, wrappers = ["'", '"']) { // The rule may or may not be wrapped with the wrappers return choice(wrapWith(rule, wrappers), rule) } function annotateFilter(filterField, filterOp, filterVal) { return seq( field('field', filterField), field('op', filterOp), field('value', filterVal) ); } diff --git a/swh/search/query_language/sample_query b/swh/search/query_language/sample_query index 3d8c08d..1ea3ad5 100644 --- a/swh/search/query_language/sample_query +++ b/swh/search/query_language/sample_query @@ -1,6 +1,6 @@ -(origin = django/django and language in ["python"] or visits >= 5) or +(origin : django/django and language in ["python"] or visits >= 5) or (last_revision > 2020-01-01 and limit = 10) or (last_visit > 2021-01-01 or last_visit < 2020-01-01) or -(visited = false and metadata = "gitlab") or +(visited = false and metadata : "gitlab") or (keyword in ["orchestration", "kubectl"] and language in ["go", "rust"]) or (visit_type = [deb] and license in ["GPL-3"]) diff --git a/swh/search/query_language/src/grammar.json b/swh/search/query_language/src/grammar.json index 22ced21..61b54e7 100644 --- a/swh/search/query_language/src/grammar.json +++ b/swh/search/query_language/src/grammar.json @@ -1,1314 +1,1326 @@ { "name": "swh_search_ql", "rules": { "query": { "type": "SEQ", "members": [ { "type": "SYMBOL", "name": "filters" }, { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "CHOICE", "members": [ { "type": "SYMBOL", "name": "and" }, { "type": "BLANK" } ] }, { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "SYMBOL", "name": "sortBy" }, { "type": "CHOICE", "members": [ { "type": "SYMBOL", "name": "and" }, { "type": "BLANK" } ] }, { "type": "CHOICE", "members": [ { "type": "SYMBOL", "name": "limit" }, { "type": "BLANK" } ] } ] }, { "type": "SEQ", "members": [ { "type": "SYMBOL", "name": "limit" }, { "type": "CHOICE", "members": [ { "type": "SYMBOL", "name": "and" }, { "type": "BLANK" } ] }, { "type": "CHOICE", "members": [ { "type": "SYMBOL", "name": "sortBy" }, { "type": "BLANK" } ] } ] } ] } ] }, { "type": "BLANK" } ] } ] }, "filters": { "type": "CHOICE", "members": [ { "type": "PREC_LEFT", "value": 3, "content": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "left", "content": { "type": "SYMBOL", "name": "filters" } }, { "type": "FIELD", "name": "operator", "content": { "type": "SYMBOL", "name": "and" } }, { "type": "FIELD", "name": "right", "content": { "type": "SYMBOL", "name": "filters" } } ] } }, { "type": "PREC_LEFT", "value": 2, "content": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "left", "content": { "type": "SYMBOL", "name": "filters" } }, { "type": "FIELD", "name": "operator", "content": { "type": "SYMBOL", "name": "or" } }, { "type": "FIELD", "name": "right", "content": { "type": "SYMBOL", "name": "filters" } } ] } }, { "type": "PREC_LEFT", "value": 4, "content": { "type": "SEQ", "members": [ { "type": "STRING", "value": "(" }, { "type": "SYMBOL", "name": "filters" }, { "type": "STRING", "value": ")" } ] } }, { "type": "SYMBOL", "name": "filter" } ] }, "sortBy": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "field", "content": { "type": "SYMBOL", "name": "sortByField" } }, { "type": "FIELD", "name": "op", "content": { "type": "SYMBOL", "name": "sortByOp" } }, { "type": "FIELD", "name": "value", "content": { "type": "SYMBOL", "name": "sortByVal" } } ] }, "sortByField": { "type": "TOKEN", "content": { "type": "STRING", "value": "sort_by" } }, "sortByOp": { "type": "SYMBOL", "name": "equalOp" }, "sortByVal": { "type": "SEQ", "members": [ { "type": "STRING", "value": "[" }, { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "FIELD", "name": "array_member", "content": { "type": "CHOICE", "members": [ { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "STRING", "value": "'" }, { "type": "SYMBOL", "name": "sortByOptions" }, { "type": "STRING", "value": "'" } ] }, { "type": "SEQ", "members": [ { "type": "STRING", "value": "\"" }, { "type": "SYMBOL", "name": "sortByOptions" }, { "type": "STRING", "value": "\"" } ] } ] }, { "type": "SYMBOL", "name": "sortByOptions" } ] } }, { "type": "REPEAT", "content": { "type": "SEQ", "members": [ { "type": "STRING", "value": "," }, { "type": "CHOICE", "members": [ { "type": "FIELD", "name": "array_member", "content": { "type": "CHOICE", "members": [ { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "STRING", "value": "'" }, { "type": "SYMBOL", "name": "sortByOptions" }, { "type": "STRING", "value": "'" } ] }, { "type": "SEQ", "members": [ { "type": "STRING", "value": "\"" }, { "type": "SYMBOL", "name": "sortByOptions" }, { "type": "STRING", "value": "\"" } ] } ] }, { "type": "SYMBOL", "name": "sortByOptions" } ] } }, { "type": "BLANK" } ] } ] } } ] }, { "type": "BLANK" } ] }, { "type": "STRING", "value": "]" } ] }, "sortByOptions": { "type": "SEQ", "members": [ { "type": "CHOICE", "members": [ { "type": "STRING", "value": "-" }, { "type": "BLANK" } ] }, { "type": "CHOICE", "members": [ { "type": "STRING", "value": "visits" }, { "type": "STRING", "value": "last_visit" }, { "type": "STRING", "value": "last_eventful_visit" }, { "type": "STRING", "value": "last_revision" }, { "type": "STRING", "value": "last_release" }, { "type": "STRING", "value": "created" }, { "type": "STRING", "value": "modified" }, { "type": "STRING", "value": "published" } ] } ] }, "limit": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "field", "content": { "type": "SYMBOL", "name": "limitField" } }, { "type": "FIELD", "name": "op", "content": { "type": "SYMBOL", "name": "equalOp" } }, { "type": "FIELD", "name": "value", "content": { "type": "SYMBOL", "name": "number" } } ] }, "limitField": { "type": "TOKEN", "content": { "type": "STRING", "value": "limit" } }, "filter": { "type": "FIELD", "name": "category", "content": { "type": "CHOICE", "members": [ { "type": "SYMBOL", "name": "patternFilter" }, { "type": "SYMBOL", "name": "booleanFilter" }, { "type": "SYMBOL", "name": "numericFilter" }, { "type": "SYMBOL", "name": "boundedListFilter" }, { "type": "SYMBOL", "name": "unboundedListFilter" }, { "type": "SYMBOL", "name": "dateFilter" } ] } }, "patternFilter": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "field", "content": { "type": "SYMBOL", "name": "patternField" } }, { "type": "FIELD", "name": "op", "content": { "type": "SYMBOL", "name": "patternOp" } }, { "type": "FIELD", "name": "value", "content": { "type": "SYMBOL", "name": "patternVal" } } ] }, "patternField": { "type": "TOKEN", "content": { "type": "CHOICE", "members": [ { "type": "STRING", "value": "origin" }, { "type": "STRING", "value": "metadata" } ] } }, "patternOp": { "type": "SYMBOL", - "name": "equalOp" + "name": "containOp" }, "patternVal": { "type": "SYMBOL", "name": "string" }, "booleanFilter": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "field", "content": { "type": "SYMBOL", "name": "booleanField" } }, { "type": "FIELD", "name": "op", "content": { "type": "SYMBOL", "name": "booleanOp" } }, { "type": "FIELD", "name": "value", "content": { "type": "SYMBOL", "name": "booleanVal" } } ] }, "booleanField": { "type": "TOKEN", "content": { "type": "CHOICE", "members": [ { "type": "STRING", "value": "visited" } ] } }, "booleanOp": { "type": "SYMBOL", "name": "equalOp" }, "booleanVal": { "type": "CHOICE", "members": [ { "type": "SYMBOL", "name": "booleanTrue" }, { "type": "SYMBOL", "name": "booleanFalse" } ] }, "numericFilter": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "field", "content": { "type": "SYMBOL", "name": "numericField" } }, { "type": "FIELD", "name": "op", "content": { "type": "SYMBOL", "name": "numericOp" } }, { "type": "FIELD", "name": "value", "content": { "type": "SYMBOL", "name": "numberVal" } } ] }, "numericField": { "type": "TOKEN", "content": { "type": "CHOICE", "members": [ { "type": "STRING", "value": "visits" } ] } }, "numericOp": { "type": "SYMBOL", "name": "rangeOp" }, "numberVal": { "type": "SYMBOL", "name": "number" }, "boundedListFilter": { "type": "CHOICE", "members": [ { "type": "SYMBOL", "name": "visitTypeFilter" } ] }, "visitTypeFilter": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "field", "content": { "type": "SYMBOL", "name": "visitTypeField" } }, { "type": "FIELD", "name": "op", "content": { "type": "SYMBOL", "name": "visitTypeOp" } }, { "type": "FIELD", "name": "value", "content": { "type": "SYMBOL", "name": "visitTypeVal" } } ] }, "visitTypeField": { "type": "TOKEN", "content": { "type": "STRING", "value": "visit_type" } }, "visitTypeOp": { "type": "SYMBOL", "name": "equalOp" }, "visitTypeVal": { "type": "SEQ", "members": [ { "type": "STRING", "value": "[" }, { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "FIELD", "name": "array_member", "content": { "type": "CHOICE", "members": [ { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "STRING", "value": "'" }, { "type": "SYMBOL", "name": "visitTypeOptions" }, { "type": "STRING", "value": "'" } ] }, { "type": "SEQ", "members": [ { "type": "STRING", "value": "\"" }, { "type": "SYMBOL", "name": "visitTypeOptions" }, { "type": "STRING", "value": "\"" } ] } ] }, { "type": "SYMBOL", "name": "visitTypeOptions" } ] } }, { "type": "REPEAT", "content": { "type": "SEQ", "members": [ { "type": "STRING", "value": "," }, { "type": "CHOICE", "members": [ { "type": "FIELD", "name": "array_member", "content": { "type": "CHOICE", "members": [ { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "STRING", "value": "'" }, { "type": "SYMBOL", "name": "visitTypeOptions" }, { "type": "STRING", "value": "'" } ] }, { "type": "SEQ", "members": [ { "type": "STRING", "value": "\"" }, { "type": "SYMBOL", "name": "visitTypeOptions" }, { "type": "STRING", "value": "\"" } ] } ] }, { "type": "SYMBOL", "name": "visitTypeOptions" } ] } }, { "type": "BLANK" } ] } ] } } ] }, { "type": "BLANK" } ] }, { "type": "STRING", "value": "]" } ] }, "visitTypeOptions": { "type": "CHOICE", "members": [ { "type": "STRING", "value": "any" }, { "type": "STRING", "value": "bzr" }, { "type": "STRING", "value": "cran" }, { "type": "STRING", "value": "cvs" }, { "type": "STRING", "value": "deb" }, { "type": "STRING", "value": "deposit" }, { "type": "STRING", "value": "ftp" }, { "type": "STRING", "value": "hg" }, { "type": "STRING", "value": "git" }, { "type": "STRING", "value": "nixguix" }, { "type": "STRING", "value": "npm" }, { "type": "STRING", "value": "opam" }, { "type": "STRING", "value": "pypi" }, { "type": "STRING", "value": "svn" }, { "type": "STRING", "value": "tar" } ] }, "unboundedListFilter": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "field", "content": { "type": "SYMBOL", "name": "listField" } }, { "type": "FIELD", "name": "op", "content": { "type": "SYMBOL", "name": "listOp" } }, { "type": "FIELD", "name": "value", "content": { "type": "SYMBOL", "name": "listVal" } } ] }, "listField": { "type": "TOKEN", "content": { "type": "CHOICE", "members": [ { "type": "STRING", "value": "language" }, { "type": "STRING", "value": "license" }, { "type": "STRING", "value": "keyword" } ] } }, "listOp": { "type": "SYMBOL", "name": "choiceOp" }, "listVal": { "type": "SEQ", "members": [ { "type": "STRING", "value": "[" }, { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "FIELD", "name": "array_member", "content": { "type": "SYMBOL", "name": "string" } }, { "type": "REPEAT", "content": { "type": "SEQ", "members": [ { "type": "STRING", "value": "," }, { "type": "CHOICE", "members": [ { "type": "FIELD", "name": "array_member", "content": { "type": "SYMBOL", "name": "string" } }, { "type": "BLANK" } ] } ] } } ] }, { "type": "BLANK" } ] }, { "type": "STRING", "value": "]" } ] }, "dateFilter": { "type": "SEQ", "members": [ { "type": "FIELD", "name": "field", "content": { "type": "SYMBOL", "name": "dateField" } }, { "type": "FIELD", "name": "op", "content": { "type": "SYMBOL", "name": "dateOp" } }, { "type": "FIELD", "name": "value", "content": { "type": "SYMBOL", "name": "dateVal" } } ] }, "dateField": { "type": "TOKEN", "content": { "type": "CHOICE", "members": [ { "type": "STRING", "value": "last_visit" }, { "type": "STRING", "value": "last_eventful_visit" }, { "type": "STRING", "value": "last_revision" }, { "type": "STRING", "value": "last_release" }, { "type": "STRING", "value": "created" }, { "type": "STRING", "value": "modified" }, { "type": "STRING", "value": "published" } ] } }, "dateOp": { "type": "SYMBOL", "name": "rangeOp" }, "dateVal": { "type": "SYMBOL", "name": "isoDateTime" }, "rangeOp": { "type": "TOKEN", "content": { "type": "CHOICE", "members": [ { "type": "STRING", "value": "<" }, { "type": "STRING", "value": "<=" }, { "type": "STRING", "value": "=" }, { "type": "STRING", "value": "!=" }, { "type": "STRING", "value": ">=" }, { "type": "STRING", "value": ">" } ] } }, "equalOp": { "type": "TOKEN", "content": { "type": "CHOICE", "members": [ { "type": "STRING", "value": "=" } ] } }, + "containOp": { + "type": "TOKEN", + "content": { + "type": "CHOICE", + "members": [ + { + "type": "STRING", + "value": ":" + } + ] + } + }, "choiceOp": { "type": "TOKEN", "content": { "type": "CHOICE", "members": [ { "type": "STRING", "value": "in" }, { "type": "STRING", "value": "not in" } ] } }, "isoDateTime": { "type": "PATTERN", - "value": "\\d{4}[-]\\d{2}[-]\\d{2}(\\s|T)*(\\d{2}:\\d{2}(:\\d{2}(\\.\\d{6})?)?)?(\\+\\d{2}:\\d{2}|Z)?" + "value": "\\d{4}[-]\\d{2}[-]\\d{2}((\\s|T)*\\d{2}:\\d{2}(:\\d{2}(\\.\\d{6})?)?(\\+\\d{2}:\\d{2}|Z)?)?" }, "string": { "type": "CHOICE", "members": [ { "type": "CHOICE", "members": [ { "type": "SEQ", "members": [ { "type": "STRING", "value": "'" }, { "type": "SYMBOL", "name": "stringContent" }, { "type": "STRING", "value": "'" } ] }, { "type": "SEQ", "members": [ { "type": "STRING", "value": "\"" }, { "type": "SYMBOL", "name": "stringContent" }, { "type": "STRING", "value": "\"" } ] } ] }, { "type": "SYMBOL", "name": "singleWord" } ] }, "number": { "type": "PATTERN", "value": "\\d+" }, "booleanTrue": { "type": "STRING", "value": "true" }, "booleanFalse": { "type": "STRING", "value": "false" }, "or": { "type": "STRING", "value": "or" }, "and": { "type": "STRING", "value": "and" }, "singleWord": { "type": "PATTERN", "value": "[^\\s\"'\\[\\]\\(\\),]+" }, "stringContent": { "type": "REPEAT1", "content": { "type": "CHOICE", "members": [ { "type": "IMMEDIATE_TOKEN", "content": { "type": "PATTERN", "value": "[^\\\\'\"\\n]+" } }, { "type": "SYMBOL", "name": "escape_sequence" } ] } }, "escape_sequence": { "type": "IMMEDIATE_TOKEN", "content": { "type": "SEQ", "members": [ { "type": "STRING", "value": "\\" }, { "type": "PATTERN", "value": "(\\\"|\\'|\\\\|\\/|b|n|r|t|u)" } ] } } }, "extras": [ { "type": "PATTERN", "value": "\\s" } ], "conflicts": [], "precedences": [], "externals": [], "inline": [], "supertypes": [] } diff --git a/swh/search/query_language/src/node-types.json b/swh/search/query_language/src/node-types.json index 71d51a5..aab470f 100644 --- a/swh/search/query_language/src/node-types.json +++ b/swh/search/query_language/src/node-types.json @@ -1,899 +1,903 @@ [ { "type": "booleanFilter", "named": true, "fields": { "field": { "multiple": false, "required": true, "types": [ { "type": "booleanField", "named": true } ] }, "op": { "multiple": false, "required": true, "types": [ { "type": "booleanOp", "named": true } ] }, "value": { "multiple": false, "required": true, "types": [ { "type": "booleanVal", "named": true } ] } } }, { "type": "booleanOp", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "equalOp", "named": true } ] } }, { "type": "booleanVal", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "booleanFalse", "named": true }, { "type": "booleanTrue", "named": true } ] } }, { "type": "boundedListFilter", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "visitTypeFilter", "named": true } ] } }, { "type": "dateFilter", "named": true, "fields": { "field": { "multiple": false, "required": true, "types": [ { "type": "dateField", "named": true } ] }, "op": { "multiple": false, "required": true, "types": [ { "type": "dateOp", "named": true } ] }, "value": { "multiple": false, "required": true, "types": [ { "type": "dateVal", "named": true } ] } } }, { "type": "dateOp", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "rangeOp", "named": true } ] } }, { "type": "dateVal", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "isoDateTime", "named": true } ] } }, { "type": "filter", "named": true, "fields": { "category": { "multiple": false, "required": true, "types": [ { "type": "booleanFilter", "named": true }, { "type": "boundedListFilter", "named": true }, { "type": "dateFilter", "named": true }, { "type": "numericFilter", "named": true }, { "type": "patternFilter", "named": true }, { "type": "unboundedListFilter", "named": true } ] } } }, { "type": "filters", "named": true, "fields": { "left": { "multiple": false, "required": false, "types": [ { "type": "filters", "named": true } ] }, "operator": { "multiple": false, "required": false, "types": [ { "type": "and", "named": true }, { "type": "or", "named": true } ] }, "right": { "multiple": false, "required": false, "types": [ { "type": "filters", "named": true } ] } }, "children": { "multiple": false, "required": false, "types": [ { "type": "filter", "named": true }, { "type": "filters", "named": true } ] } }, { "type": "limit", "named": true, "fields": { "field": { "multiple": false, "required": true, "types": [ { "type": "limitField", "named": true } ] }, "op": { "multiple": false, "required": true, "types": [ { "type": "equalOp", "named": true } ] }, "value": { "multiple": false, "required": true, "types": [ { "type": "number", "named": true } ] } } }, { "type": "listOp", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "choiceOp", "named": true } ] } }, { "type": "listVal", "named": true, "fields": { "array_member": { "multiple": true, "required": false, "types": [ { "type": "string", "named": true } ] } } }, { "type": "numberVal", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "number", "named": true } ] } }, { "type": "numericFilter", "named": true, "fields": { "field": { "multiple": false, "required": true, "types": [ { "type": "numericField", "named": true } ] }, "op": { "multiple": false, "required": true, "types": [ { "type": "numericOp", "named": true } ] }, "value": { "multiple": false, "required": true, "types": [ { "type": "numberVal", "named": true } ] } } }, { "type": "numericOp", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "rangeOp", "named": true } ] } }, { "type": "patternFilter", "named": true, "fields": { "field": { "multiple": false, "required": true, "types": [ { "type": "patternField", "named": true } ] }, "op": { "multiple": false, "required": true, "types": [ { "type": "patternOp", "named": true } ] }, "value": { "multiple": false, "required": true, "types": [ { "type": "patternVal", "named": true } ] } } }, { "type": "patternOp", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { - "type": "equalOp", + "type": "containOp", "named": true } ] } }, { "type": "patternVal", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "string", "named": true } ] } }, { "type": "query", "named": true, "fields": {}, "children": { "multiple": true, "required": true, "types": [ { "type": "and", "named": true }, { "type": "filters", "named": true }, { "type": "limit", "named": true }, { "type": "sortBy", "named": true } ] } }, { "type": "sortBy", "named": true, "fields": { "field": { "multiple": false, "required": true, "types": [ { "type": "sortByField", "named": true } ] }, "op": { "multiple": false, "required": true, "types": [ { "type": "sortByOp", "named": true } ] }, "value": { "multiple": false, "required": true, "types": [ { "type": "sortByVal", "named": true } ] } } }, { "type": "sortByOp", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "equalOp", "named": true } ] } }, { "type": "sortByOptions", "named": true, "fields": {} }, { "type": "sortByVal", "named": true, "fields": { "array_member": { "multiple": true, "required": false, "types": [ { "type": "\"", "named": false }, { "type": "'", "named": false }, { "type": "sortByOptions", "named": true } ] } } }, { "type": "string", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "singleWord", "named": true }, { "type": "stringContent", "named": true } ] } }, { "type": "stringContent", "named": true, "fields": {}, "children": { "multiple": true, "required": false, "types": [ { "type": "escape_sequence", "named": true } ] } }, { "type": "unboundedListFilter", "named": true, "fields": { "field": { "multiple": false, "required": true, "types": [ { "type": "listField", "named": true } ] }, "op": { "multiple": false, "required": true, "types": [ { "type": "listOp", "named": true } ] }, "value": { "multiple": false, "required": true, "types": [ { "type": "listVal", "named": true } ] } } }, { "type": "visitTypeFilter", "named": true, "fields": { "field": { "multiple": false, "required": true, "types": [ { "type": "visitTypeField", "named": true } ] }, "op": { "multiple": false, "required": true, "types": [ { "type": "visitTypeOp", "named": true } ] }, "value": { "multiple": false, "required": true, "types": [ { "type": "visitTypeVal", "named": true } ] } } }, { "type": "visitTypeOp", "named": true, "fields": {}, "children": { "multiple": false, "required": true, "types": [ { "type": "equalOp", "named": true } ] } }, { "type": "visitTypeOptions", "named": true, "fields": {} }, { "type": "visitTypeVal", "named": true, "fields": { "array_member": { "multiple": true, "required": false, "types": [ { "type": "\"", "named": false }, { "type": "'", "named": false }, { "type": "visitTypeOptions", "named": true } ] } } }, { "type": "\"", "named": false }, { "type": "'", "named": false }, { "type": "(", "named": false }, { "type": ")", "named": false }, { "type": ",", "named": false }, { "type": "-", "named": false }, { "type": "[", "named": false }, { "type": "]", "named": false }, { "type": "and", "named": true }, { "type": "any", "named": false }, { "type": "booleanFalse", "named": true }, { "type": "booleanField", "named": true }, { "type": "booleanTrue", "named": true }, { "type": "bzr", "named": false }, { "type": "choiceOp", "named": true }, + { + "type": "containOp", + "named": true + }, { "type": "cran", "named": false }, { "type": "created", "named": false }, { "type": "cvs", "named": false }, { "type": "dateField", "named": true }, { "type": "deb", "named": false }, { "type": "deposit", "named": false }, { "type": "equalOp", "named": true }, { "type": "escape_sequence", "named": true }, { "type": "ftp", "named": false }, { "type": "git", "named": false }, { "type": "hg", "named": false }, { "type": "isoDateTime", "named": true }, { "type": "last_eventful_visit", "named": false }, { "type": "last_release", "named": false }, { "type": "last_revision", "named": false }, { "type": "last_visit", "named": false }, { "type": "limitField", "named": true }, { "type": "listField", "named": true }, { "type": "modified", "named": false }, { "type": "nixguix", "named": false }, { "type": "npm", "named": false }, { "type": "number", "named": true }, { "type": "numericField", "named": true }, { "type": "opam", "named": false }, { "type": "or", "named": true }, { "type": "patternField", "named": true }, { "type": "published", "named": false }, { "type": "pypi", "named": false }, { "type": "rangeOp", "named": true }, { "type": "singleWord", "named": true }, { "type": "sortByField", "named": true }, { "type": "svn", "named": false }, { "type": "tar", "named": false }, { "type": "visitTypeField", "named": true }, { "type": "visits", "named": false } ] \ No newline at end of file diff --git a/swh/search/query_language/src/parser.c b/swh/search/query_language/src/parser.c index 32d2988..b2095d2 100644 --- a/swh/search/query_language/src/parser.c +++ b/swh/search/query_language/src/parser.c @@ -1,3385 +1,3404 @@ #include #if defined(__GNUC__) || defined(__clang__) #pragma GCC diagnostic push #pragma GCC diagnostic ignored "-Wmissing-field-initializers" #endif #ifdef _MSC_VER #pragma optimize("", off) #elif defined(__clang__) #pragma clang optimize off #elif defined(__GNUC__) #pragma GCC optimize ("O0") #endif #define LANGUAGE_VERSION 13 #define STATE_COUNT 126 #define LARGE_STATE_COUNT 2 -#define SYMBOL_COUNT 86 +#define SYMBOL_COUNT 87 #define ALIAS_COUNT 0 -#define TOKEN_COUNT 52 +#define TOKEN_COUNT 53 #define EXTERNAL_TOKEN_COUNT 0 #define FIELD_COUNT 8 #define MAX_ALIAS_SEQUENCE_LENGTH 6 #define PRODUCTION_ID_COUNT 9 enum { anon_sym_LPAREN = 1, anon_sym_RPAREN = 2, sym_sortByField = 3, anon_sym_LBRACK = 4, anon_sym_SQUOTE = 5, anon_sym_DQUOTE = 6, anon_sym_COMMA = 7, anon_sym_RBRACK = 8, anon_sym_DASH = 9, anon_sym_visits = 10, anon_sym_last_visit = 11, anon_sym_last_eventful_visit = 12, anon_sym_last_revision = 13, anon_sym_last_release = 14, anon_sym_created = 15, anon_sym_modified = 16, anon_sym_published = 17, sym_limitField = 18, sym_patternField = 19, sym_booleanField = 20, sym_numericField = 21, sym_visitTypeField = 22, anon_sym_any = 23, anon_sym_bzr = 24, anon_sym_cran = 25, anon_sym_cvs = 26, anon_sym_deb = 27, anon_sym_deposit = 28, anon_sym_ftp = 29, anon_sym_hg = 30, anon_sym_git = 31, anon_sym_nixguix = 32, anon_sym_npm = 33, anon_sym_opam = 34, anon_sym_pypi = 35, anon_sym_svn = 36, anon_sym_tar = 37, sym_listField = 38, sym_dateField = 39, sym_rangeOp = 40, sym_equalOp = 41, - sym_choiceOp = 42, - sym_isoDateTime = 43, - sym_number = 44, - sym_booleanTrue = 45, - sym_booleanFalse = 46, - sym_or = 47, - sym_and = 48, - sym_singleWord = 49, - aux_sym_stringContent_token1 = 50, - sym_escape_sequence = 51, - sym_query = 52, - sym_filters = 53, - sym_sortBy = 54, - sym_sortByOp = 55, - sym_sortByVal = 56, - sym_sortByOptions = 57, - sym_limit = 58, - sym_filter = 59, - sym_patternFilter = 60, - sym_patternOp = 61, - sym_patternVal = 62, - sym_booleanFilter = 63, - sym_booleanOp = 64, - sym_booleanVal = 65, - sym_numericFilter = 66, - sym_numericOp = 67, - sym_numberVal = 68, - sym_boundedListFilter = 69, - sym_visitTypeFilter = 70, - sym_visitTypeOp = 71, - sym_visitTypeVal = 72, - sym_visitTypeOptions = 73, - sym_unboundedListFilter = 74, - sym_listOp = 75, - sym_listVal = 76, - sym_dateFilter = 77, - sym_dateOp = 78, - sym_dateVal = 79, - sym_string = 80, - sym_stringContent = 81, - aux_sym_sortByVal_repeat1 = 82, - aux_sym_visitTypeVal_repeat1 = 83, - aux_sym_listVal_repeat1 = 84, - aux_sym_stringContent_repeat1 = 85, + sym_containOp = 42, + sym_choiceOp = 43, + sym_isoDateTime = 44, + sym_number = 45, + sym_booleanTrue = 46, + sym_booleanFalse = 47, + sym_or = 48, + sym_and = 49, + sym_singleWord = 50, + aux_sym_stringContent_token1 = 51, + sym_escape_sequence = 52, + sym_query = 53, + sym_filters = 54, + sym_sortBy = 55, + sym_sortByOp = 56, + sym_sortByVal = 57, + sym_sortByOptions = 58, + sym_limit = 59, + sym_filter = 60, + sym_patternFilter = 61, + sym_patternOp = 62, + sym_patternVal = 63, + sym_booleanFilter = 64, + sym_booleanOp = 65, + sym_booleanVal = 66, + sym_numericFilter = 67, + sym_numericOp = 68, + sym_numberVal = 69, + sym_boundedListFilter = 70, + sym_visitTypeFilter = 71, + sym_visitTypeOp = 72, + sym_visitTypeVal = 73, + sym_visitTypeOptions = 74, + sym_unboundedListFilter = 75, + sym_listOp = 76, + sym_listVal = 77, + sym_dateFilter = 78, + sym_dateOp = 79, + sym_dateVal = 80, + sym_string = 81, + sym_stringContent = 82, + aux_sym_sortByVal_repeat1 = 83, + aux_sym_visitTypeVal_repeat1 = 84, + aux_sym_listVal_repeat1 = 85, + aux_sym_stringContent_repeat1 = 86, }; static const char * const ts_symbol_names[] = { [ts_builtin_sym_end] = "end", [anon_sym_LPAREN] = "(", [anon_sym_RPAREN] = ")", [sym_sortByField] = "sortByField", [anon_sym_LBRACK] = "[", [anon_sym_SQUOTE] = "'", [anon_sym_DQUOTE] = "\"", [anon_sym_COMMA] = ",", [anon_sym_RBRACK] = "]", [anon_sym_DASH] = "-", [anon_sym_visits] = "visits", [anon_sym_last_visit] = "last_visit", [anon_sym_last_eventful_visit] = "last_eventful_visit", [anon_sym_last_revision] = "last_revision", [anon_sym_last_release] = "last_release", [anon_sym_created] = "created", [anon_sym_modified] = "modified", [anon_sym_published] = "published", [sym_limitField] = "limitField", [sym_patternField] = "patternField", [sym_booleanField] = "booleanField", [sym_numericField] = "numericField", [sym_visitTypeField] = "visitTypeField", [anon_sym_any] = "any", [anon_sym_bzr] = "bzr", [anon_sym_cran] = "cran", [anon_sym_cvs] = "cvs", [anon_sym_deb] = "deb", [anon_sym_deposit] = "deposit", [anon_sym_ftp] = "ftp", [anon_sym_hg] = "hg", [anon_sym_git] = "git", [anon_sym_nixguix] = "nixguix", [anon_sym_npm] = "npm", [anon_sym_opam] = "opam", [anon_sym_pypi] = "pypi", [anon_sym_svn] = "svn", [anon_sym_tar] = "tar", [sym_listField] = "listField", [sym_dateField] = "dateField", [sym_rangeOp] = "rangeOp", [sym_equalOp] = "equalOp", + [sym_containOp] = "containOp", [sym_choiceOp] = "choiceOp", [sym_isoDateTime] = "isoDateTime", [sym_number] = "number", [sym_booleanTrue] = "booleanTrue", [sym_booleanFalse] = "booleanFalse", [sym_or] = "or", [sym_and] = "and", [sym_singleWord] = "singleWord", [aux_sym_stringContent_token1] = "stringContent_token1", [sym_escape_sequence] = "escape_sequence", [sym_query] = "query", [sym_filters] = "filters", [sym_sortBy] = "sortBy", [sym_sortByOp] = "sortByOp", [sym_sortByVal] = "sortByVal", [sym_sortByOptions] = "sortByOptions", [sym_limit] = "limit", [sym_filter] = "filter", [sym_patternFilter] = "patternFilter", [sym_patternOp] = "patternOp", [sym_patternVal] = "patternVal", [sym_booleanFilter] = "booleanFilter", [sym_booleanOp] = "booleanOp", [sym_booleanVal] = "booleanVal", [sym_numericFilter] = "numericFilter", [sym_numericOp] = "numericOp", [sym_numberVal] = "numberVal", [sym_boundedListFilter] = "boundedListFilter", [sym_visitTypeFilter] = "visitTypeFilter", [sym_visitTypeOp] = "visitTypeOp", [sym_visitTypeVal] = "visitTypeVal", [sym_visitTypeOptions] = "visitTypeOptions", [sym_unboundedListFilter] = "unboundedListFilter", [sym_listOp] = "listOp", [sym_listVal] = "listVal", [sym_dateFilter] = "dateFilter", [sym_dateOp] = "dateOp", [sym_dateVal] = "dateVal", [sym_string] = "string", [sym_stringContent] = "stringContent", [aux_sym_sortByVal_repeat1] = "sortByVal_repeat1", [aux_sym_visitTypeVal_repeat1] = "visitTypeVal_repeat1", [aux_sym_listVal_repeat1] = "listVal_repeat1", [aux_sym_stringContent_repeat1] = "stringContent_repeat1", }; static const TSSymbol ts_symbol_map[] = { [ts_builtin_sym_end] = ts_builtin_sym_end, [anon_sym_LPAREN] = anon_sym_LPAREN, [anon_sym_RPAREN] = anon_sym_RPAREN, [sym_sortByField] = sym_sortByField, [anon_sym_LBRACK] = anon_sym_LBRACK, [anon_sym_SQUOTE] = anon_sym_SQUOTE, [anon_sym_DQUOTE] = anon_sym_DQUOTE, [anon_sym_COMMA] = anon_sym_COMMA, [anon_sym_RBRACK] = anon_sym_RBRACK, [anon_sym_DASH] = anon_sym_DASH, [anon_sym_visits] = anon_sym_visits, [anon_sym_last_visit] = anon_sym_last_visit, [anon_sym_last_eventful_visit] = anon_sym_last_eventful_visit, [anon_sym_last_revision] = anon_sym_last_revision, [anon_sym_last_release] = anon_sym_last_release, [anon_sym_created] = anon_sym_created, [anon_sym_modified] = anon_sym_modified, [anon_sym_published] = anon_sym_published, [sym_limitField] = sym_limitField, [sym_patternField] = sym_patternField, [sym_booleanField] = sym_booleanField, [sym_numericField] = sym_numericField, [sym_visitTypeField] = sym_visitTypeField, [anon_sym_any] = anon_sym_any, [anon_sym_bzr] = anon_sym_bzr, [anon_sym_cran] = anon_sym_cran, [anon_sym_cvs] = anon_sym_cvs, [anon_sym_deb] = anon_sym_deb, [anon_sym_deposit] = anon_sym_deposit, [anon_sym_ftp] = anon_sym_ftp, [anon_sym_hg] = anon_sym_hg, [anon_sym_git] = anon_sym_git, [anon_sym_nixguix] = anon_sym_nixguix, [anon_sym_npm] = anon_sym_npm, [anon_sym_opam] = anon_sym_opam, [anon_sym_pypi] = anon_sym_pypi, [anon_sym_svn] = anon_sym_svn, [anon_sym_tar] = anon_sym_tar, [sym_listField] = sym_listField, [sym_dateField] = sym_dateField, [sym_rangeOp] = sym_rangeOp, [sym_equalOp] = sym_equalOp, + [sym_containOp] = sym_containOp, [sym_choiceOp] = sym_choiceOp, [sym_isoDateTime] = sym_isoDateTime, [sym_number] = sym_number, [sym_booleanTrue] = sym_booleanTrue, [sym_booleanFalse] = sym_booleanFalse, [sym_or] = sym_or, [sym_and] = sym_and, [sym_singleWord] = sym_singleWord, [aux_sym_stringContent_token1] = aux_sym_stringContent_token1, [sym_escape_sequence] = sym_escape_sequence, [sym_query] = sym_query, [sym_filters] = sym_filters, [sym_sortBy] = sym_sortBy, [sym_sortByOp] = sym_sortByOp, [sym_sortByVal] = sym_sortByVal, [sym_sortByOptions] = sym_sortByOptions, [sym_limit] = sym_limit, [sym_filter] = sym_filter, [sym_patternFilter] = sym_patternFilter, [sym_patternOp] = sym_patternOp, [sym_patternVal] = sym_patternVal, [sym_booleanFilter] = sym_booleanFilter, [sym_booleanOp] = sym_booleanOp, [sym_booleanVal] = sym_booleanVal, [sym_numericFilter] = sym_numericFilter, [sym_numericOp] = sym_numericOp, [sym_numberVal] = sym_numberVal, [sym_boundedListFilter] = sym_boundedListFilter, [sym_visitTypeFilter] = sym_visitTypeFilter, [sym_visitTypeOp] = sym_visitTypeOp, [sym_visitTypeVal] = sym_visitTypeVal, [sym_visitTypeOptions] = sym_visitTypeOptions, [sym_unboundedListFilter] = sym_unboundedListFilter, [sym_listOp] = sym_listOp, [sym_listVal] = sym_listVal, [sym_dateFilter] = sym_dateFilter, [sym_dateOp] = sym_dateOp, [sym_dateVal] = sym_dateVal, [sym_string] = sym_string, [sym_stringContent] = sym_stringContent, [aux_sym_sortByVal_repeat1] = aux_sym_sortByVal_repeat1, [aux_sym_visitTypeVal_repeat1] = aux_sym_visitTypeVal_repeat1, [aux_sym_listVal_repeat1] = aux_sym_listVal_repeat1, [aux_sym_stringContent_repeat1] = aux_sym_stringContent_repeat1, }; static const TSSymbolMetadata ts_symbol_metadata[] = { [ts_builtin_sym_end] = { .visible = false, .named = true, }, [anon_sym_LPAREN] = { .visible = true, .named = false, }, [anon_sym_RPAREN] = { .visible = true, .named = false, }, [sym_sortByField] = { .visible = true, .named = true, }, [anon_sym_LBRACK] = { .visible = true, .named = false, }, [anon_sym_SQUOTE] = { .visible = true, .named = false, }, [anon_sym_DQUOTE] = { .visible = true, .named = false, }, [anon_sym_COMMA] = { .visible = true, .named = false, }, [anon_sym_RBRACK] = { .visible = true, .named = false, }, [anon_sym_DASH] = { .visible = true, .named = false, }, [anon_sym_visits] = { .visible = true, .named = false, }, [anon_sym_last_visit] = { .visible = true, .named = false, }, [anon_sym_last_eventful_visit] = { .visible = true, .named = false, }, [anon_sym_last_revision] = { .visible = true, .named = false, }, [anon_sym_last_release] = { .visible = true, .named = false, }, [anon_sym_created] = { .visible = true, .named = false, }, [anon_sym_modified] = { .visible = true, .named = false, }, [anon_sym_published] = { .visible = true, .named = false, }, [sym_limitField] = { .visible = true, .named = true, }, [sym_patternField] = { .visible = true, .named = true, }, [sym_booleanField] = { .visible = true, .named = true, }, [sym_numericField] = { .visible = true, .named = true, }, [sym_visitTypeField] = { .visible = true, .named = true, }, [anon_sym_any] = { .visible = true, .named = false, }, [anon_sym_bzr] = { .visible = true, .named = false, }, [anon_sym_cran] = { .visible = true, .named = false, }, [anon_sym_cvs] = { .visible = true, .named = false, }, [anon_sym_deb] = { .visible = true, .named = false, }, [anon_sym_deposit] = { .visible = true, .named = false, }, [anon_sym_ftp] = { .visible = true, .named = false, }, [anon_sym_hg] = { .visible = true, .named = false, }, [anon_sym_git] = { .visible = true, .named = false, }, [anon_sym_nixguix] = { .visible = true, .named = false, }, [anon_sym_npm] = { .visible = true, .named = false, }, [anon_sym_opam] = { .visible = true, .named = false, }, [anon_sym_pypi] = { .visible = true, .named = false, }, [anon_sym_svn] = { .visible = true, .named = false, }, [anon_sym_tar] = { .visible = true, .named = false, }, [sym_listField] = { .visible = true, .named = true, }, [sym_dateField] = { .visible = true, .named = true, }, [sym_rangeOp] = { .visible = true, .named = true, }, [sym_equalOp] = { .visible = true, .named = true, }, + [sym_containOp] = { + .visible = true, + .named = true, + }, [sym_choiceOp] = { .visible = true, .named = true, }, [sym_isoDateTime] = { .visible = true, .named = true, }, [sym_number] = { .visible = true, .named = true, }, [sym_booleanTrue] = { .visible = true, .named = true, }, [sym_booleanFalse] = { .visible = true, .named = true, }, [sym_or] = { .visible = true, .named = true, }, [sym_and] = { .visible = true, .named = true, }, [sym_singleWord] = { .visible = true, .named = true, }, [aux_sym_stringContent_token1] = { .visible = false, .named = false, }, [sym_escape_sequence] = { .visible = true, .named = true, }, [sym_query] = { .visible = true, .named = true, }, [sym_filters] = { .visible = true, .named = true, }, [sym_sortBy] = { .visible = true, .named = true, }, [sym_sortByOp] = { .visible = true, .named = true, }, [sym_sortByVal] = { .visible = true, .named = true, }, [sym_sortByOptions] = { .visible = true, .named = true, }, [sym_limit] = { .visible = true, .named = true, }, [sym_filter] = { .visible = true, .named = true, }, [sym_patternFilter] = { .visible = true, .named = true, }, [sym_patternOp] = { .visible = true, .named = true, }, [sym_patternVal] = { .visible = true, .named = true, }, [sym_booleanFilter] = { .visible = true, .named = true, }, [sym_booleanOp] = { .visible = true, .named = true, }, [sym_booleanVal] = { .visible = true, .named = true, }, [sym_numericFilter] = { .visible = true, .named = true, }, [sym_numericOp] = { .visible = true, .named = true, }, [sym_numberVal] = { .visible = true, .named = true, }, [sym_boundedListFilter] = { .visible = true, .named = true, }, [sym_visitTypeFilter] = { .visible = true, .named = true, }, [sym_visitTypeOp] = { .visible = true, .named = true, }, [sym_visitTypeVal] = { .visible = true, .named = true, }, [sym_visitTypeOptions] = { .visible = true, .named = true, }, [sym_unboundedListFilter] = { .visible = true, .named = true, }, [sym_listOp] = { .visible = true, .named = true, }, [sym_listVal] = { .visible = true, .named = true, }, [sym_dateFilter] = { .visible = true, .named = true, }, [sym_dateOp] = { .visible = true, .named = true, }, [sym_dateVal] = { .visible = true, .named = true, }, [sym_string] = { .visible = true, .named = true, }, [sym_stringContent] = { .visible = true, .named = true, }, [aux_sym_sortByVal_repeat1] = { .visible = false, .named = false, }, [aux_sym_visitTypeVal_repeat1] = { .visible = false, .named = false, }, [aux_sym_listVal_repeat1] = { .visible = false, .named = false, }, [aux_sym_stringContent_repeat1] = { .visible = false, .named = false, }, }; enum { field_array_member = 1, field_category = 2, field_field = 3, field_left = 4, field_op = 5, field_operator = 6, field_right = 7, field_value = 8, }; static const char * const ts_field_names[] = { [0] = NULL, [field_array_member] = "array_member", [field_category] = "category", [field_field] = "field", [field_left] = "left", [field_op] = "op", [field_operator] = "operator", [field_right] = "right", [field_value] = "value", }; static const TSFieldMapSlice ts_field_map_slices[PRODUCTION_ID_COUNT] = { [1] = {.index = 0, .length = 1}, [2] = {.index = 1, .length = 3}, [3] = {.index = 4, .length = 3}, [4] = {.index = 7, .length = 1}, [5] = {.index = 8, .length = 2}, [6] = {.index = 10, .length = 2}, [7] = {.index = 12, .length = 3}, [8] = {.index = 15, .length = 4}, }; static const TSFieldMapEntry ts_field_map_entries[] = { [0] = {field_category, 0}, [1] = {field_field, 0}, {field_op, 1}, {field_value, 2}, [4] = {field_left, 0}, {field_operator, 1}, {field_right, 2}, [7] = {field_array_member, 1}, [8] = {field_array_member, 1}, {field_array_member, 2, .inherited = true}, [10] = {field_array_member, 0, .inherited = true}, {field_array_member, 1, .inherited = true}, [12] = {field_array_member, 1}, {field_array_member, 2}, {field_array_member, 3}, [15] = {field_array_member, 1}, {field_array_member, 2}, {field_array_member, 3}, {field_array_member, 4, .inherited = true}, }; static const TSSymbol ts_alias_sequences[PRODUCTION_ID_COUNT][MAX_ALIAS_SEQUENCE_LENGTH] = { [0] = {0}, }; static const uint16_t ts_non_terminal_alias_map[] = { 0, }; static inline bool sym_singleWord_character_set_1(int32_t c) { return (c < '"' ? (c < '\r' ? (c < '\t' ? c == 0 : c <= '\n') : (c <= '\r' || c == ' ')) : (c <= '"' || (c < '[' ? (c < ',' ? (c >= '\'' && c <= ')') : c <= ',') : (c <= '[' || c == ']')))); } static bool ts_lex(TSLexer *lexer, TSStateId state) { START_LEXER(); eof = lexer->eof(lexer); switch (state) { case 0: - if (eof) ADVANCE(241); + if (eof) ADVANCE(242); if (lookahead == '!') ADVANCE(10); - if (lookahead == '"') ADVANCE(247); - if (lookahead == '\'') ADVANCE(246); - if (lookahead == '(') ADVANCE(242); - if (lookahead == ')') ADVANCE(243); - if (lookahead == ',') ADVANCE(248); - if (lookahead == '-') ADVANCE(250); - if (lookahead == '<') ADVANCE(282); - if (lookahead == '=') ADVANCE(281); - if (lookahead == '>') ADVANCE(282); - if (lookahead == '[') ADVANCE(245); - if (lookahead == '\\') ADVANCE(216); - if (lookahead == ']') ADVANCE(249); + if (lookahead == '"') ADVANCE(248); + if (lookahead == '\'') ADVANCE(247); + if (lookahead == '(') ADVANCE(243); + if (lookahead == ')') ADVANCE(244); + if (lookahead == ',') ADVANCE(249); + if (lookahead == '-') ADVANCE(251); + if (lookahead == ':') ADVANCE(285); + if (lookahead == '<') ADVANCE(283); + if (lookahead == '=') ADVANCE(282); + if (lookahead == '>') ADVANCE(283); + if (lookahead == '[') ADVANCE(246); + if (lookahead == '\\') ADVANCE(217); + if (lookahead == ']') ADVANCE(250); if (lookahead == 'a') ADVANCE(126); if (lookahead == 'b') ADVANCE(215); if (lookahead == 'c') ADVANCE(150); if (lookahead == 'd') ADVANCE(49); if (lookahead == 'f') ADVANCE(18); if (lookahead == 'g') ADVANCE(85); if (lookahead == 'h') ADVANCE(78); if (lookahead == 'i') ADVANCE(127); if (lookahead == 'k') ADVANCE(50); if (lookahead == 'l') ADVANCE(19); if (lookahead == 'm') ADVANCE(59); if (lookahead == 'n') ADVANCE(86); if (lookahead == 'o') ADVANCE(146); if (lookahead == 'p') ADVANCE(198); if (lookahead == 's') ADVANCE(141); if (lookahead == 't') ADVANCE(23); if (lookahead == 'v') ADVANCE(90); if (lookahead == '\t' || lookahead == '\n' || lookahead == '\r' || - lookahead == ' ') SKIP(239) - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(293); + lookahead == ' ') SKIP(240) + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(295); END_STATE(); case 1: if (lookahead == '\n') SKIP(4) - if (lookahead == '"') ADVANCE(247); - if (lookahead == '\'') ADVANCE(246); - if (lookahead == '\\') ADVANCE(216); + if (lookahead == '"') ADVANCE(248); + if (lookahead == '\'') ADVANCE(247); + if (lookahead == '\\') ADVANCE(217); if (lookahead == '\t' || lookahead == '\r' || - lookahead == ' ') ADVANCE(301); - if (lookahead != 0) ADVANCE(302); + lookahead == ' ') ADVANCE(303); + if (lookahead != 0) ADVANCE(304); END_STATE(); case 2: if (lookahead == ' ') ADVANCE(89); END_STATE(); case 3: - if (lookahead == '"') ADVANCE(247); - if (lookahead == '\'') ADVANCE(246); - if (lookahead == ',') ADVANCE(248); - if (lookahead == ']') ADVANCE(249); + if (lookahead == '"') ADVANCE(248); + if (lookahead == '\'') ADVANCE(247); + if (lookahead == ',') ADVANCE(249); + if (lookahead == ']') ADVANCE(250); if (lookahead == '\t' || lookahead == '\n' || lookahead == '\r' || lookahead == ' ') SKIP(3) if (lookahead != 0 && lookahead != '(' && lookahead != ')' && - lookahead != '[') ADVANCE(300); + lookahead != '[') ADVANCE(302); END_STATE(); case 4: - if (lookahead == '"') ADVANCE(247); - if (lookahead == '\'') ADVANCE(246); + if (lookahead == '"') ADVANCE(248); + if (lookahead == '\'') ADVANCE(247); if (lookahead == '\t' || lookahead == '\n' || lookahead == '\r' || lookahead == ' ') SKIP(4) END_STATE(); case 5: - if (lookahead == '(') ADVANCE(242); - if (lookahead == '=') ADVANCE(283); + if (lookahead == '(') ADVANCE(243); + if (lookahead == '=') ADVANCE(284); if (lookahead == 'c') ADVANCE(157); if (lookahead == 'k') ADVANCE(50); if (lookahead == 'l') ADVANCE(32); if (lookahead == 'm') ADVANCE(60); if (lookahead == 'o') ADVANCE(156); if (lookahead == 'p') ADVANCE(203); if (lookahead == 's') ADVANCE(140); if (lookahead == 'v') ADVANCE(113); if (lookahead == '\t' || lookahead == '\n' || lookahead == '\r' || lookahead == ' ') SKIP(5) - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(294); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(296); END_STATE(); case 6: - if (lookahead == '-') ADVANCE(224); + if (lookahead == '-') ADVANCE(225); END_STATE(); case 7: - if (lookahead == '-') ADVANCE(225); + if (lookahead == '-') ADVANCE(226); END_STATE(); case 8: - if (lookahead == ':') ADVANCE(226); + if (lookahead == ':') ADVANCE(227); END_STATE(); case 9: - if (lookahead == ':') ADVANCE(227); + if (lookahead == ':') ADVANCE(229); END_STATE(); case 10: - if (lookahead == '=') ADVANCE(281); + if (lookahead == '=') ADVANCE(282); END_STATE(); case 11: if (lookahead == '_') ADVANCE(53); END_STATE(); case 12: if (lookahead == '_') ADVANCE(35); END_STATE(); case 13: if (lookahead == '_') ADVANCE(206); END_STATE(); case 14: if (lookahead == '_') ADVANCE(189); if (lookahead == 'e') ADVANCE(42); - if (lookahead == 's') ADVANCE(251); + if (lookahead == 's') ADVANCE(252); END_STATE(); case 15: if (lookahead == '_') ADVANCE(189); if (lookahead == 'e') ADVANCE(42); - if (lookahead == 's') ADVANCE(262); + if (lookahead == 's') ADVANCE(263); END_STATE(); case 16: if (lookahead == '_') ADVANCE(73); END_STATE(); case 17: if (lookahead == '_') ADVANCE(207); END_STATE(); case 18: if (lookahead == 'a') ADVANCE(116); if (lookahead == 't') ADVANCE(147); END_STATE(); case 19: if (lookahead == 'a') ADVANCE(133); if (lookahead == 'i') ADVANCE(37); END_STATE(); case 20: - if (lookahead == 'a') ADVANCE(260); + if (lookahead == 'a') ADVANCE(261); END_STATE(); case 21: if (lookahead == 'a') ADVANCE(47); END_STATE(); case 22: if (lookahead == 'a') ADVANCE(124); END_STATE(); case 23: if (lookahead == 'a') ADVANCE(152); if (lookahead == 'r') ADVANCE(200); END_STATE(); case 24: if (lookahead == 'a') ADVANCE(129); if (lookahead == 'e') ADVANCE(27); END_STATE(); case 25: if (lookahead == 'a') ADVANCE(82); END_STATE(); case 26: if (lookahead == 'a') ADVANCE(162); if (lookahead == 'i') ADVANCE(125); END_STATE(); case 27: if (lookahead == 'a') ADVANCE(191); END_STATE(); case 28: if (lookahead == 'a') ADVANCE(194); END_STATE(); case 29: if (lookahead == 'a') ADVANCE(192); END_STATE(); case 30: if (lookahead == 'a') ADVANCE(166); END_STATE(); case 31: if (lookahead == 'a') ADVANCE(167); END_STATE(); case 32: if (lookahead == 'a') ADVANCE(134); if (lookahead == 'i') ADVANCE(37); END_STATE(); case 33: - if (lookahead == 'b') ADVANCE(268); + if (lookahead == 'b') ADVANCE(269); if (lookahead == 'p') ADVANCE(145); END_STATE(); case 34: if (lookahead == 'b') ADVANCE(119); END_STATE(); case 35: if (lookahead == 'b') ADVANCE(213); END_STATE(); case 36: if (lookahead == 'b') ADVANCE(120); END_STATE(); case 37: if (lookahead == 'c') ADVANCE(63); if (lookahead == 'm') ADVANCE(97); END_STATE(); case 38: - if (lookahead == 'd') ADVANCE(299); + if (lookahead == 'd') ADVANCE(301); END_STATE(); case 39: - if (lookahead == 'd') ADVANCE(299); - if (lookahead == 'y') ADVANCE(264); + if (lookahead == 'd') ADVANCE(301); + if (lookahead == 'y') ADVANCE(265); END_STATE(); case 40: - if (lookahead == 'd') ADVANCE(256); + if (lookahead == 'd') ADVANCE(257); END_STATE(); case 41: - if (lookahead == 'd') ADVANCE(279); + if (lookahead == 'd') ADVANCE(280); END_STATE(); case 42: - if (lookahead == 'd') ADVANCE(261); + if (lookahead == 'd') ADVANCE(262); END_STATE(); case 43: - if (lookahead == 'd') ADVANCE(257); + if (lookahead == 'd') ADVANCE(258); END_STATE(); case 44: - if (lookahead == 'd') ADVANCE(258); + if (lookahead == 'd') ADVANCE(259); END_STATE(); case 45: - if (lookahead == 'd') ADVANCE(280); + if (lookahead == 'd') ADVANCE(281); END_STATE(); case 46: if (lookahead == 'd') ADVANCE(87); END_STATE(); case 47: if (lookahead == 'd') ADVANCE(29); END_STATE(); case 48: if (lookahead == 'd') ADVANCE(109); END_STATE(); case 49: if (lookahead == 'e') ADVANCE(33); END_STATE(); case 50: if (lookahead == 'e') ADVANCE(212); END_STATE(); case 51: - if (lookahead == 'e') ADVANCE(295); + if (lookahead == 'e') ADVANCE(297); END_STATE(); case 52: - if (lookahead == 'e') ADVANCE(296); + if (lookahead == 'e') ADVANCE(298); END_STATE(); case 53: if (lookahead == 'e') ADVANCE(205); if (lookahead == 'r') ADVANCE(55); if (lookahead == 'v') ADVANCE(111); END_STATE(); case 54: - if (lookahead == 'e') ADVANCE(279); + if (lookahead == 'e') ADVANCE(280); END_STATE(); case 55: if (lookahead == 'e') ADVANCE(118); END_STATE(); case 56: - if (lookahead == 'e') ADVANCE(263); + if (lookahead == 'e') ADVANCE(264); END_STATE(); case 57: - if (lookahead == 'e') ADVANCE(255); + if (lookahead == 'e') ADVANCE(256); END_STATE(); case 58: - if (lookahead == 'e') ADVANCE(280); + if (lookahead == 'e') ADVANCE(281); END_STATE(); case 59: if (lookahead == 'e') ADVANCE(190); if (lookahead == 'o') ADVANCE(46); END_STATE(); case 60: if (lookahead == 'e') ADVANCE(190); if (lookahead == 'o') ADVANCE(48); END_STATE(); case 61: if (lookahead == 'e') ADVANCE(27); END_STATE(); case 62: if (lookahead == 'e') ADVANCE(40); END_STATE(); case 63: if (lookahead == 'e') ADVANCE(137); END_STATE(); case 64: if (lookahead == 'e') ADVANCE(43); END_STATE(); case 65: if (lookahead == 'e') ADVANCE(136); END_STATE(); case 66: if (lookahead == 'e') ADVANCE(44); END_STATE(); case 67: if (lookahead == 'e') ADVANCE(30); END_STATE(); case 68: if (lookahead == 'e') ADVANCE(45); END_STATE(); case 69: if (lookahead == 'e') ADVANCE(28); END_STATE(); case 70: if (lookahead == 'e') ADVANCE(31); END_STATE(); case 71: if (lookahead == 'e') ADVANCE(121); END_STATE(); case 72: if (lookahead == 'e') ADVANCE(138); END_STATE(); case 73: if (lookahead == 'e') ADVANCE(208); if (lookahead == 'r') ADVANCE(71); if (lookahead == 'v') ADVANCE(114); END_STATE(); case 74: if (lookahead == 'f') ADVANCE(199); END_STATE(); case 75: if (lookahead == 'f') ADVANCE(99); END_STATE(); case 76: if (lookahead == 'f') ADVANCE(103); END_STATE(); case 77: if (lookahead == 'f') ADVANCE(204); END_STATE(); case 78: - if (lookahead == 'g') ADVANCE(271); + if (lookahead == 'g') ADVANCE(272); END_STATE(); case 79: if (lookahead == 'g') ADVANCE(201); END_STATE(); case 80: if (lookahead == 'g') ADVANCE(202); END_STATE(); case 81: if (lookahead == 'g') ADVANCE(96); END_STATE(); case 82: if (lookahead == 'g') ADVANCE(54); END_STATE(); case 83: if (lookahead == 'h') ADVANCE(66); END_STATE(); case 84: if (lookahead == 'h') ADVANCE(68); END_STATE(); case 85: if (lookahead == 'i') ADVANCE(177); END_STATE(); case 86: if (lookahead == 'i') ADVANCE(211); if (lookahead == 'o') ADVANCE(178); if (lookahead == 'p') ADVANCE(123); END_STATE(); case 87: if (lookahead == 'i') ADVANCE(75); END_STATE(); case 88: - if (lookahead == 'i') ADVANCE(276); + if (lookahead == 'i') ADVANCE(277); END_STATE(); case 89: if (lookahead == 'i') ADVANCE(127); END_STATE(); case 90: if (lookahead == 'i') ADVANCE(170); END_STATE(); case 91: if (lookahead == 'i') ADVANCE(210); END_STATE(); case 92: if (lookahead == 'i') ADVANCE(81); END_STATE(); case 93: if (lookahead == 'i') ADVANCE(143); END_STATE(); case 94: if (lookahead == 'i') ADVANCE(160); END_STATE(); case 95: if (lookahead == 'i') ADVANCE(165); END_STATE(); case 96: if (lookahead == 'i') ADVANCE(130); END_STATE(); case 97: if (lookahead == 'i') ADVANCE(180); END_STATE(); case 98: if (lookahead == 'i') ADVANCE(181); END_STATE(); case 99: if (lookahead == 'i') ADVANCE(64); END_STATE(); case 100: if (lookahead == 'i') ADVANCE(182); END_STATE(); case 101: if (lookahead == 'i') ADVANCE(183); END_STATE(); case 102: if (lookahead == 'i') ADVANCE(184); END_STATE(); case 103: if (lookahead == 'i') ADVANCE(68); END_STATE(); case 104: if (lookahead == 'i') ADVANCE(188); END_STATE(); case 105: if (lookahead == 'i') ADVANCE(185); END_STATE(); case 106: if (lookahead == 'i') ADVANCE(193); END_STATE(); case 107: if (lookahead == 'i') ADVANCE(144); END_STATE(); case 108: if (lookahead == 'i') ADVANCE(168); END_STATE(); case 109: if (lookahead == 'i') ADVANCE(76); END_STATE(); case 110: if (lookahead == 'i') ADVANCE(169); END_STATE(); case 111: if (lookahead == 'i') ADVANCE(172); END_STATE(); case 112: if (lookahead == 'i') ADVANCE(173); END_STATE(); case 113: if (lookahead == 'i') ADVANCE(174); END_STATE(); case 114: if (lookahead == 'i') ADVANCE(175); END_STATE(); case 115: if (lookahead == 'i') ADVANCE(176); END_STATE(); case 116: if (lookahead == 'l') ADVANCE(163); END_STATE(); case 117: if (lookahead == 'l') ADVANCE(13); END_STATE(); case 118: if (lookahead == 'l') ADVANCE(67); if (lookahead == 'v') ADVANCE(95); END_STATE(); case 119: if (lookahead == 'l') ADVANCE(94); END_STATE(); case 120: if (lookahead == 'l') ADVANCE(108); END_STATE(); case 121: if (lookahead == 'l') ADVANCE(70); if (lookahead == 'v') ADVANCE(110); END_STATE(); case 122: if (lookahead == 'l') ADVANCE(17); END_STATE(); case 123: - if (lookahead == 'm') ADVANCE(274); + if (lookahead == 'm') ADVANCE(275); END_STATE(); case 124: - if (lookahead == 'm') ADVANCE(275); + if (lookahead == 'm') ADVANCE(276); END_STATE(); case 125: if (lookahead == 'm') ADVANCE(97); END_STATE(); case 126: if (lookahead == 'n') ADVANCE(39); END_STATE(); case 127: - if (lookahead == 'n') ADVANCE(284); + if (lookahead == 'n') ADVANCE(286); END_STATE(); case 128: - if (lookahead == 'n') ADVANCE(277); + if (lookahead == 'n') ADVANCE(278); END_STATE(); case 129: - if (lookahead == 'n') ADVANCE(266); + if (lookahead == 'n') ADVANCE(267); END_STATE(); case 130: - if (lookahead == 'n') ADVANCE(260); + if (lookahead == 'n') ADVANCE(261); END_STATE(); case 131: - if (lookahead == 'n') ADVANCE(254); + if (lookahead == 'n') ADVANCE(255); END_STATE(); case 132: - if (lookahead == 'n') ADVANCE(280); + if (lookahead == 'n') ADVANCE(281); END_STATE(); case 133: if (lookahead == 'n') ADVANCE(79); if (lookahead == 's') ADVANCE(179); END_STATE(); case 134: if (lookahead == 'n') ADVANCE(79); if (lookahead == 's') ADVANCE(195); END_STATE(); case 135: if (lookahead == 'n') ADVANCE(38); END_STATE(); case 136: if (lookahead == 'n') ADVANCE(187); END_STATE(); case 137: if (lookahead == 'n') ADVANCE(164); END_STATE(); case 138: if (lookahead == 'n') ADVANCE(196); END_STATE(); case 139: if (lookahead == 'o') ADVANCE(46); END_STATE(); case 140: if (lookahead == 'o') ADVANCE(155); END_STATE(); case 141: if (lookahead == 'o') ADVANCE(155); if (lookahead == 'v') ADVANCE(128); END_STATE(); case 142: if (lookahead == 'o') ADVANCE(154); END_STATE(); case 143: if (lookahead == 'o') ADVANCE(131); END_STATE(); case 144: if (lookahead == 'o') ADVANCE(132); END_STATE(); case 145: if (lookahead == 'o') ADVANCE(171); END_STATE(); case 146: if (lookahead == 'p') ADVANCE(22); - if (lookahead == 'r') ADVANCE(298); + if (lookahead == 'r') ADVANCE(300); END_STATE(); case 147: - if (lookahead == 'p') ADVANCE(270); + if (lookahead == 'p') ADVANCE(271); END_STATE(); case 148: if (lookahead == 'p') ADVANCE(88); END_STATE(); case 149: if (lookahead == 'p') ADVANCE(56); END_STATE(); case 150: if (lookahead == 'r') ADVANCE(24); if (lookahead == 'v') ADVANCE(159); END_STATE(); case 151: - if (lookahead == 'r') ADVANCE(265); + if (lookahead == 'r') ADVANCE(266); END_STATE(); case 152: - if (lookahead == 'r') ADVANCE(278); + if (lookahead == 'r') ADVANCE(279); END_STATE(); case 153: - if (lookahead == 'r') ADVANCE(297); + if (lookahead == 'r') ADVANCE(299); END_STATE(); case 154: if (lookahead == 'r') ADVANCE(41); END_STATE(); case 155: if (lookahead == 'r') ADVANCE(186); END_STATE(); case 156: if (lookahead == 'r') ADVANCE(92); END_STATE(); case 157: if (lookahead == 'r') ADVANCE(69); END_STATE(); case 158: if (lookahead == 'r') ADVANCE(61); END_STATE(); case 159: - if (lookahead == 's') ADVANCE(267); + if (lookahead == 's') ADVANCE(268); END_STATE(); case 160: if (lookahead == 's') ADVANCE(83); END_STATE(); case 161: - if (lookahead == 's') ADVANCE(251); + if (lookahead == 's') ADVANCE(252); END_STATE(); case 162: if (lookahead == 's') ADVANCE(179); END_STATE(); case 163: if (lookahead == 's') ADVANCE(52); END_STATE(); case 164: if (lookahead == 's') ADVANCE(54); END_STATE(); case 165: if (lookahead == 's') ADVANCE(93); END_STATE(); case 166: if (lookahead == 's') ADVANCE(57); END_STATE(); case 167: if (lookahead == 's') ADVANCE(58); END_STATE(); case 168: if (lookahead == 's') ADVANCE(84); END_STATE(); case 169: if (lookahead == 's') ADVANCE(107); END_STATE(); case 170: if (lookahead == 's') ADVANCE(98); END_STATE(); case 171: if (lookahead == 's') ADVANCE(100); END_STATE(); case 172: if (lookahead == 's') ADVANCE(101); END_STATE(); case 173: if (lookahead == 's') ADVANCE(102); END_STATE(); case 174: if (lookahead == 's') ADVANCE(104); END_STATE(); case 175: if (lookahead == 's') ADVANCE(105); END_STATE(); case 176: if (lookahead == 's') ADVANCE(106); END_STATE(); case 177: - if (lookahead == 't') ADVANCE(272); + if (lookahead == 't') ADVANCE(273); END_STATE(); case 178: if (lookahead == 't') ADVANCE(2); END_STATE(); case 179: if (lookahead == 't') ADVANCE(11); END_STATE(); case 180: - if (lookahead == 't') ADVANCE(259); + if (lookahead == 't') ADVANCE(260); END_STATE(); case 181: if (lookahead == 't') ADVANCE(14); END_STATE(); case 182: - if (lookahead == 't') ADVANCE(269); + if (lookahead == 't') ADVANCE(270); END_STATE(); case 183: - if (lookahead == 't') ADVANCE(252); + if (lookahead == 't') ADVANCE(253); END_STATE(); case 184: - if (lookahead == 't') ADVANCE(253); + if (lookahead == 't') ADVANCE(254); END_STATE(); case 185: - if (lookahead == 't') ADVANCE(280); + if (lookahead == 't') ADVANCE(281); END_STATE(); case 186: if (lookahead == 't') ADVANCE(12); END_STATE(); case 187: if (lookahead == 't') ADVANCE(74); END_STATE(); case 188: if (lookahead == 't') ADVANCE(15); END_STATE(); case 189: if (lookahead == 't') ADVANCE(214); END_STATE(); case 190: if (lookahead == 't') ADVANCE(21); END_STATE(); case 191: if (lookahead == 't') ADVANCE(62); END_STATE(); case 192: if (lookahead == 't') ADVANCE(20); END_STATE(); case 193: if (lookahead == 't') ADVANCE(161); END_STATE(); case 194: if (lookahead == 't') ADVANCE(68); END_STATE(); case 195: if (lookahead == 't') ADVANCE(16); END_STATE(); case 196: if (lookahead == 't') ADVANCE(77); END_STATE(); case 197: if (lookahead == 'u') ADVANCE(34); END_STATE(); case 198: if (lookahead == 'u') ADVANCE(34); if (lookahead == 'y') ADVANCE(148); END_STATE(); case 199: if (lookahead == 'u') ADVANCE(117); END_STATE(); case 200: if (lookahead == 'u') ADVANCE(51); END_STATE(); case 201: if (lookahead == 'u') ADVANCE(25); END_STATE(); case 202: if (lookahead == 'u') ADVANCE(91); END_STATE(); case 203: if (lookahead == 'u') ADVANCE(36); END_STATE(); case 204: if (lookahead == 'u') ADVANCE(122); END_STATE(); case 205: if (lookahead == 'v') ADVANCE(65); END_STATE(); case 206: if (lookahead == 'v') ADVANCE(112); END_STATE(); case 207: if (lookahead == 'v') ADVANCE(114); END_STATE(); case 208: if (lookahead == 'v') ADVANCE(72); END_STATE(); case 209: if (lookahead == 'w') ADVANCE(142); END_STATE(); case 210: - if (lookahead == 'x') ADVANCE(273); + if (lookahead == 'x') ADVANCE(274); END_STATE(); case 211: if (lookahead == 'x') ADVANCE(80); END_STATE(); case 212: if (lookahead == 'y') ADVANCE(209); END_STATE(); case 213: - if (lookahead == 'y') ADVANCE(244); + if (lookahead == 'y') ADVANCE(245); END_STATE(); case 214: if (lookahead == 'y') ADVANCE(149); END_STATE(); case 215: if (lookahead == 'z') ADVANCE(151); END_STATE(); case 216: + if (lookahead == '\t' || + lookahead == '\n' || + lookahead == '\r' || + lookahead == ' ' || + lookahead == 'T') ADVANCE(216); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(220); + END_STATE(); + case 217: if (lookahead == '"' || lookahead == '\'' || lookahead == '/' || lookahead == '\\' || lookahead == 'b' || lookahead == 'n' || lookahead == 'r' || lookahead == 't' || - lookahead == 'u') ADVANCE(303); - END_STATE(); - case 217: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(7); + lookahead == 'u') ADVANCE(305); END_STATE(); case 218: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(289); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(7); END_STATE(); case 219: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(285); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(291); END_STATE(); case 220: if (('0' <= lookahead && lookahead <= '9')) ADVANCE(8); END_STATE(); case 221: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(287); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(289); END_STATE(); case 222: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(286); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(287); END_STATE(); case 223: if (('0' <= lookahead && lookahead <= '9')) ADVANCE(288); END_STATE(); case 224: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(217); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(290); END_STATE(); case 225: if (('0' <= lookahead && lookahead <= '9')) ADVANCE(218); END_STATE(); case 226: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(221); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(219); END_STATE(); case 227: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(219); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(221); END_STATE(); case 228: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(222); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(223); END_STATE(); case 229: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(223); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(222); END_STATE(); case 230: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(6); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(224); END_STATE(); case 231: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(230); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(6); END_STATE(); case 232: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(229); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(231); END_STATE(); case 233: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(231); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(230); END_STATE(); case 234: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(9); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(232); END_STATE(); case 235: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(234); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(9); END_STATE(); case 236: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(232); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(235); END_STATE(); case 237: - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(236); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(233); END_STATE(); case 238: if (('0' <= lookahead && lookahead <= '9')) ADVANCE(237); END_STATE(); case 239: - if (eof) ADVANCE(241); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(238); + END_STATE(); + case 240: + if (eof) ADVANCE(242); if (lookahead == '!') ADVANCE(10); - if (lookahead == '"') ADVANCE(247); - if (lookahead == '\'') ADVANCE(246); - if (lookahead == '(') ADVANCE(242); - if (lookahead == ')') ADVANCE(243); - if (lookahead == ',') ADVANCE(248); - if (lookahead == '-') ADVANCE(250); - if (lookahead == '<') ADVANCE(282); - if (lookahead == '=') ADVANCE(281); - if (lookahead == '>') ADVANCE(282); - if (lookahead == '[') ADVANCE(245); - if (lookahead == ']') ADVANCE(249); + if (lookahead == '"') ADVANCE(248); + if (lookahead == '\'') ADVANCE(247); + if (lookahead == '(') ADVANCE(243); + if (lookahead == ')') ADVANCE(244); + if (lookahead == ',') ADVANCE(249); + if (lookahead == '-') ADVANCE(251); + if (lookahead == ':') ADVANCE(285); + if (lookahead == '<') ADVANCE(283); + if (lookahead == '=') ADVANCE(282); + if (lookahead == '>') ADVANCE(283); + if (lookahead == '[') ADVANCE(246); + if (lookahead == ']') ADVANCE(250); if (lookahead == 'a') ADVANCE(126); if (lookahead == 'b') ADVANCE(215); if (lookahead == 'c') ADVANCE(150); if (lookahead == 'd') ADVANCE(49); if (lookahead == 'f') ADVANCE(18); if (lookahead == 'g') ADVANCE(85); if (lookahead == 'h') ADVANCE(78); if (lookahead == 'i') ADVANCE(127); if (lookahead == 'k') ADVANCE(50); if (lookahead == 'l') ADVANCE(19); if (lookahead == 'm') ADVANCE(59); if (lookahead == 'n') ADVANCE(86); if (lookahead == 'o') ADVANCE(146); if (lookahead == 'p') ADVANCE(198); if (lookahead == 's') ADVANCE(141); if (lookahead == 't') ADVANCE(23); if (lookahead == 'v') ADVANCE(90); if (lookahead == '\t' || lookahead == '\n' || lookahead == '\r' || - lookahead == ' ') SKIP(239) - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(293); + lookahead == ' ') SKIP(240) + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(295); END_STATE(); - case 240: - if (eof) ADVANCE(241); + case 241: + if (eof) ADVANCE(242); if (lookahead == '!') ADVANCE(10); - if (lookahead == '"') ADVANCE(247); - if (lookahead == '\'') ADVANCE(246); - if (lookahead == ')') ADVANCE(243); - if (lookahead == ',') ADVANCE(248); - if (lookahead == '-') ADVANCE(250); - if (lookahead == '<') ADVANCE(282); - if (lookahead == '=') ADVANCE(281); - if (lookahead == '>') ADVANCE(282); - if (lookahead == ']') ADVANCE(249); + if (lookahead == '"') ADVANCE(248); + if (lookahead == '\'') ADVANCE(247); + if (lookahead == ')') ADVANCE(244); + if (lookahead == ',') ADVANCE(249); + if (lookahead == '-') ADVANCE(251); + if (lookahead == '<') ADVANCE(283); + if (lookahead == '=') ADVANCE(282); + if (lookahead == '>') ADVANCE(283); + if (lookahead == ']') ADVANCE(250); if (lookahead == 'a') ADVANCE(135); if (lookahead == 'c') ADVANCE(158); if (lookahead == 'l') ADVANCE(26); if (lookahead == 'm') ADVANCE(139); if (lookahead == 'o') ADVANCE(153); if (lookahead == 'p') ADVANCE(197); if (lookahead == 's') ADVANCE(140); if (lookahead == 'v') ADVANCE(115); if (lookahead == '\t' || lookahead == '\n' || lookahead == '\r' || - lookahead == ' ') SKIP(240) - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(233); - END_STATE(); - case 241: - ACCEPT_TOKEN(ts_builtin_sym_end); + lookahead == ' ') SKIP(241) + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(234); END_STATE(); case 242: - ACCEPT_TOKEN(anon_sym_LPAREN); + ACCEPT_TOKEN(ts_builtin_sym_end); END_STATE(); case 243: - ACCEPT_TOKEN(anon_sym_RPAREN); + ACCEPT_TOKEN(anon_sym_LPAREN); END_STATE(); case 244: - ACCEPT_TOKEN(sym_sortByField); + ACCEPT_TOKEN(anon_sym_RPAREN); END_STATE(); case 245: - ACCEPT_TOKEN(anon_sym_LBRACK); + ACCEPT_TOKEN(sym_sortByField); END_STATE(); case 246: - ACCEPT_TOKEN(anon_sym_SQUOTE); + ACCEPT_TOKEN(anon_sym_LBRACK); END_STATE(); case 247: - ACCEPT_TOKEN(anon_sym_DQUOTE); + ACCEPT_TOKEN(anon_sym_SQUOTE); END_STATE(); case 248: - ACCEPT_TOKEN(anon_sym_COMMA); + ACCEPT_TOKEN(anon_sym_DQUOTE); END_STATE(); case 249: - ACCEPT_TOKEN(anon_sym_RBRACK); + ACCEPT_TOKEN(anon_sym_COMMA); END_STATE(); case 250: - ACCEPT_TOKEN(anon_sym_DASH); + ACCEPT_TOKEN(anon_sym_RBRACK); END_STATE(); case 251: - ACCEPT_TOKEN(anon_sym_visits); + ACCEPT_TOKEN(anon_sym_DASH); END_STATE(); case 252: - ACCEPT_TOKEN(anon_sym_last_visit); + ACCEPT_TOKEN(anon_sym_visits); END_STATE(); case 253: - ACCEPT_TOKEN(anon_sym_last_eventful_visit); + ACCEPT_TOKEN(anon_sym_last_visit); END_STATE(); case 254: - ACCEPT_TOKEN(anon_sym_last_revision); + ACCEPT_TOKEN(anon_sym_last_eventful_visit); END_STATE(); case 255: - ACCEPT_TOKEN(anon_sym_last_release); + ACCEPT_TOKEN(anon_sym_last_revision); END_STATE(); case 256: - ACCEPT_TOKEN(anon_sym_created); + ACCEPT_TOKEN(anon_sym_last_release); END_STATE(); case 257: - ACCEPT_TOKEN(anon_sym_modified); + ACCEPT_TOKEN(anon_sym_created); END_STATE(); case 258: - ACCEPT_TOKEN(anon_sym_published); + ACCEPT_TOKEN(anon_sym_modified); END_STATE(); case 259: - ACCEPT_TOKEN(sym_limitField); + ACCEPT_TOKEN(anon_sym_published); END_STATE(); case 260: - ACCEPT_TOKEN(sym_patternField); + ACCEPT_TOKEN(sym_limitField); END_STATE(); case 261: - ACCEPT_TOKEN(sym_booleanField); + ACCEPT_TOKEN(sym_patternField); END_STATE(); case 262: - ACCEPT_TOKEN(sym_numericField); + ACCEPT_TOKEN(sym_booleanField); END_STATE(); case 263: - ACCEPT_TOKEN(sym_visitTypeField); + ACCEPT_TOKEN(sym_numericField); END_STATE(); case 264: - ACCEPT_TOKEN(anon_sym_any); + ACCEPT_TOKEN(sym_visitTypeField); END_STATE(); case 265: - ACCEPT_TOKEN(anon_sym_bzr); + ACCEPT_TOKEN(anon_sym_any); END_STATE(); case 266: - ACCEPT_TOKEN(anon_sym_cran); + ACCEPT_TOKEN(anon_sym_bzr); END_STATE(); case 267: - ACCEPT_TOKEN(anon_sym_cvs); + ACCEPT_TOKEN(anon_sym_cran); END_STATE(); case 268: - ACCEPT_TOKEN(anon_sym_deb); + ACCEPT_TOKEN(anon_sym_cvs); END_STATE(); case 269: - ACCEPT_TOKEN(anon_sym_deposit); + ACCEPT_TOKEN(anon_sym_deb); END_STATE(); case 270: - ACCEPT_TOKEN(anon_sym_ftp); + ACCEPT_TOKEN(anon_sym_deposit); END_STATE(); case 271: - ACCEPT_TOKEN(anon_sym_hg); + ACCEPT_TOKEN(anon_sym_ftp); END_STATE(); case 272: - ACCEPT_TOKEN(anon_sym_git); + ACCEPT_TOKEN(anon_sym_hg); END_STATE(); case 273: - ACCEPT_TOKEN(anon_sym_nixguix); + ACCEPT_TOKEN(anon_sym_git); END_STATE(); case 274: - ACCEPT_TOKEN(anon_sym_npm); + ACCEPT_TOKEN(anon_sym_nixguix); END_STATE(); case 275: - ACCEPT_TOKEN(anon_sym_opam); + ACCEPT_TOKEN(anon_sym_npm); END_STATE(); case 276: - ACCEPT_TOKEN(anon_sym_pypi); + ACCEPT_TOKEN(anon_sym_opam); END_STATE(); case 277: - ACCEPT_TOKEN(anon_sym_svn); + ACCEPT_TOKEN(anon_sym_pypi); END_STATE(); case 278: - ACCEPT_TOKEN(anon_sym_tar); + ACCEPT_TOKEN(anon_sym_svn); END_STATE(); case 279: - ACCEPT_TOKEN(sym_listField); + ACCEPT_TOKEN(anon_sym_tar); END_STATE(); case 280: - ACCEPT_TOKEN(sym_dateField); + ACCEPT_TOKEN(sym_listField); END_STATE(); case 281: - ACCEPT_TOKEN(sym_rangeOp); + ACCEPT_TOKEN(sym_dateField); END_STATE(); case 282: ACCEPT_TOKEN(sym_rangeOp); - if (lookahead == '=') ADVANCE(281); END_STATE(); case 283: - ACCEPT_TOKEN(sym_equalOp); + ACCEPT_TOKEN(sym_rangeOp); + if (lookahead == '=') ADVANCE(282); END_STATE(); case 284: - ACCEPT_TOKEN(sym_choiceOp); + ACCEPT_TOKEN(sym_equalOp); END_STATE(); case 285: - ACCEPT_TOKEN(sym_isoDateTime); + ACCEPT_TOKEN(sym_containOp); END_STATE(); case 286: - ACCEPT_TOKEN(sym_isoDateTime); - if (lookahead == '+') ADVANCE(235); - if (lookahead == '.') ADVANCE(238); - if (lookahead == 'Z') ADVANCE(285); + ACCEPT_TOKEN(sym_choiceOp); END_STATE(); case 287: ACCEPT_TOKEN(sym_isoDateTime); - if (lookahead == '+') ADVANCE(235); - if (lookahead == ':') ADVANCE(228); - if (lookahead == 'Z') ADVANCE(285); END_STATE(); case 288: ACCEPT_TOKEN(sym_isoDateTime); - if (lookahead == '+') ADVANCE(235); - if (lookahead == 'Z') ADVANCE(285); + if (lookahead == '+') ADVANCE(236); + if (lookahead == '.') ADVANCE(239); + if (lookahead == 'Z') ADVANCE(287); END_STATE(); case 289: ACCEPT_TOKEN(sym_isoDateTime); - if (lookahead == '+') ADVANCE(235); + if (lookahead == '+') ADVANCE(236); + if (lookahead == ':') ADVANCE(228); + if (lookahead == 'Z') ADVANCE(287); + END_STATE(); + case 290: + ACCEPT_TOKEN(sym_isoDateTime); + if (lookahead == '+') ADVANCE(236); + if (lookahead == 'Z') ADVANCE(287); + END_STATE(); + case 291: + ACCEPT_TOKEN(sym_isoDateTime); if (lookahead == '\t' || lookahead == '\n' || lookahead == '\r' || lookahead == ' ' || - lookahead == 'T') ADVANCE(289); - if (lookahead == 'Z') ADVANCE(285); + lookahead == 'T') ADVANCE(216); if (('0' <= lookahead && lookahead <= '9')) ADVANCE(220); END_STATE(); - case 290: - ACCEPT_TOKEN(sym_number); - if (lookahead == '-') ADVANCE(224); - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(294); - END_STATE(); - case 291: - ACCEPT_TOKEN(sym_number); - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(290); - END_STATE(); case 292: ACCEPT_TOKEN(sym_number); - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(291); + if (lookahead == '-') ADVANCE(225); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(296); END_STATE(); case 293: ACCEPT_TOKEN(sym_number); if (('0' <= lookahead && lookahead <= '9')) ADVANCE(292); END_STATE(); case 294: ACCEPT_TOKEN(sym_number); - if (('0' <= lookahead && lookahead <= '9')) ADVANCE(294); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(293); END_STATE(); case 295: - ACCEPT_TOKEN(sym_booleanTrue); + ACCEPT_TOKEN(sym_number); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(294); END_STATE(); case 296: - ACCEPT_TOKEN(sym_booleanFalse); + ACCEPT_TOKEN(sym_number); + if (('0' <= lookahead && lookahead <= '9')) ADVANCE(296); END_STATE(); case 297: - ACCEPT_TOKEN(sym_or); + ACCEPT_TOKEN(sym_booleanTrue); END_STATE(); case 298: + ACCEPT_TOKEN(sym_booleanFalse); + END_STATE(); + case 299: + ACCEPT_TOKEN(sym_or); + END_STATE(); + case 300: ACCEPT_TOKEN(sym_or); if (lookahead == 'i') ADVANCE(81); END_STATE(); - case 299: + case 301: ACCEPT_TOKEN(sym_and); END_STATE(); - case 300: + case 302: ACCEPT_TOKEN(sym_singleWord); - if (!sym_singleWord_character_set_1(lookahead)) ADVANCE(300); + if (!sym_singleWord_character_set_1(lookahead)) ADVANCE(302); END_STATE(); - case 301: + case 303: ACCEPT_TOKEN(aux_sym_stringContent_token1); if (lookahead == '\t' || lookahead == '\r' || - lookahead == ' ') ADVANCE(301); + lookahead == ' ') ADVANCE(303); if (lookahead != 0 && lookahead != '\n' && lookahead != '"' && lookahead != '\'' && - lookahead != '\\') ADVANCE(302); + lookahead != '\\') ADVANCE(304); END_STATE(); - case 302: + case 304: ACCEPT_TOKEN(aux_sym_stringContent_token1); if (lookahead != 0 && lookahead != '\n' && lookahead != '"' && lookahead != '\'' && - lookahead != '\\') ADVANCE(302); + lookahead != '\\') ADVANCE(304); END_STATE(); - case 303: + case 305: ACCEPT_TOKEN(sym_escape_sequence); END_STATE(); default: return false; } } static const TSLexMode ts_lex_modes[STATE_COUNT] = { [0] = {.lex_state = 0}, [1] = {.lex_state = 5}, [2] = {.lex_state = 5}, [3] = {.lex_state = 0}, [4] = {.lex_state = 0}, [5] = {.lex_state = 5}, [6] = {.lex_state = 0}, [7] = {.lex_state = 5}, [8] = {.lex_state = 0}, [9] = {.lex_state = 5}, [10] = {.lex_state = 0}, [11] = {.lex_state = 0}, - [12] = {.lex_state = 240}, - [13] = {.lex_state = 240}, - [14] = {.lex_state = 240}, - [15] = {.lex_state = 240}, - [16] = {.lex_state = 240}, - [17] = {.lex_state = 240}, - [18] = {.lex_state = 240}, - [19] = {.lex_state = 240}, - [20] = {.lex_state = 240}, - [21] = {.lex_state = 240}, - [22] = {.lex_state = 240}, - [23] = {.lex_state = 240}, - [24] = {.lex_state = 240}, - [25] = {.lex_state = 240}, - [26] = {.lex_state = 240}, - [27] = {.lex_state = 240}, - [28] = {.lex_state = 240}, - [29] = {.lex_state = 240}, - [30] = {.lex_state = 240}, - [31] = {.lex_state = 240}, - [32] = {.lex_state = 240}, - [33] = {.lex_state = 240}, - [34] = {.lex_state = 240}, - [35] = {.lex_state = 240}, - [36] = {.lex_state = 240}, - [37] = {.lex_state = 240}, - [38] = {.lex_state = 240}, - [39] = {.lex_state = 240}, - [40] = {.lex_state = 240}, - [41] = {.lex_state = 240}, - [42] = {.lex_state = 240}, - [43] = {.lex_state = 240}, + [12] = {.lex_state = 241}, + [13] = {.lex_state = 241}, + [14] = {.lex_state = 241}, + [15] = {.lex_state = 241}, + [16] = {.lex_state = 241}, + [17] = {.lex_state = 241}, + [18] = {.lex_state = 241}, + [19] = {.lex_state = 241}, + [20] = {.lex_state = 241}, + [21] = {.lex_state = 241}, + [22] = {.lex_state = 241}, + [23] = {.lex_state = 241}, + [24] = {.lex_state = 241}, + [25] = {.lex_state = 241}, + [26] = {.lex_state = 241}, + [27] = {.lex_state = 241}, + [28] = {.lex_state = 241}, + [29] = {.lex_state = 241}, + [30] = {.lex_state = 241}, + [31] = {.lex_state = 241}, + [32] = {.lex_state = 241}, + [33] = {.lex_state = 241}, + [34] = {.lex_state = 241}, + [35] = {.lex_state = 241}, + [36] = {.lex_state = 241}, + [37] = {.lex_state = 241}, + [38] = {.lex_state = 241}, + [39] = {.lex_state = 241}, + [40] = {.lex_state = 241}, + [41] = {.lex_state = 241}, + [42] = {.lex_state = 241}, + [43] = {.lex_state = 241}, [44] = {.lex_state = 3}, - [45] = {.lex_state = 240}, - [46] = {.lex_state = 240}, + [45] = {.lex_state = 241}, + [46] = {.lex_state = 241}, [47] = {.lex_state = 3}, [48] = {.lex_state = 3}, [49] = {.lex_state = 1}, [50] = {.lex_state = 1}, [51] = {.lex_state = 0}, [52] = {.lex_state = 1}, [53] = {.lex_state = 0}, [54] = {.lex_state = 0}, [55] = {.lex_state = 0}, [56] = {.lex_state = 0}, [57] = {.lex_state = 0}, [58] = {.lex_state = 0}, [59] = {.lex_state = 1}, [60] = {.lex_state = 0}, [61] = {.lex_state = 0}, [62] = {.lex_state = 0}, [63] = {.lex_state = 0}, [64] = {.lex_state = 0}, [65] = {.lex_state = 0}, [66] = {.lex_state = 0}, [67] = {.lex_state = 0}, [68] = {.lex_state = 0}, [69] = {.lex_state = 0}, [70] = {.lex_state = 0}, [71] = {.lex_state = 0}, [72] = {.lex_state = 0}, [73] = {.lex_state = 0}, [74] = {.lex_state = 0}, [75] = {.lex_state = 0}, [76] = {.lex_state = 0}, [77] = {.lex_state = 0}, [78] = {.lex_state = 0}, [79] = {.lex_state = 0}, - [80] = {.lex_state = 240}, + [80] = {.lex_state = 241}, [81] = {.lex_state = 3}, [82] = {.lex_state = 0}, [83] = {.lex_state = 0}, [84] = {.lex_state = 0}, [85] = {.lex_state = 0}, [86] = {.lex_state = 0}, [87] = {.lex_state = 0}, [88] = {.lex_state = 0}, [89] = {.lex_state = 5}, [90] = {.lex_state = 0}, - [91] = {.lex_state = 5}, + [91] = {.lex_state = 0}, [92] = {.lex_state = 0}, [93] = {.lex_state = 0}, [94] = {.lex_state = 5}, [95] = {.lex_state = 0}, [96] = {.lex_state = 0}, [97] = {.lex_state = 0}, [98] = {.lex_state = 0}, [99] = {.lex_state = 0}, [100] = {.lex_state = 5}, - [101] = {.lex_state = 240}, + [101] = {.lex_state = 241}, [102] = {.lex_state = 5}, - [103] = {.lex_state = 240}, - [104] = {.lex_state = 240}, + [103] = {.lex_state = 241}, + [104] = {.lex_state = 241}, [105] = {.lex_state = 0}, [106] = {.lex_state = 0}, - [107] = {.lex_state = 240}, + [107] = {.lex_state = 241}, [108] = {.lex_state = 0}, [109] = {.lex_state = 0}, [110] = {.lex_state = 0}, [111] = {.lex_state = 0}, [112] = {.lex_state = 0}, [113] = {.lex_state = 0}, [114] = {.lex_state = 0}, [115] = {.lex_state = 5}, [116] = {.lex_state = 0}, [117] = {.lex_state = 5}, [118] = {.lex_state = 0}, [119] = {.lex_state = 0}, [120] = {.lex_state = 0}, [121] = {.lex_state = 0}, [122] = {.lex_state = 0}, [123] = {.lex_state = 0}, [124] = {.lex_state = 0}, [125] = {.lex_state = 5}, }; static const uint16_t ts_parse_table[LARGE_STATE_COUNT][SYMBOL_COUNT] = { [0] = { [ts_builtin_sym_end] = ACTIONS(1), [anon_sym_LPAREN] = ACTIONS(1), [anon_sym_RPAREN] = ACTIONS(1), [sym_sortByField] = ACTIONS(1), [anon_sym_LBRACK] = ACTIONS(1), [anon_sym_SQUOTE] = ACTIONS(1), [anon_sym_DQUOTE] = ACTIONS(1), [anon_sym_COMMA] = ACTIONS(1), [anon_sym_RBRACK] = ACTIONS(1), [anon_sym_DASH] = ACTIONS(1), [anon_sym_visits] = ACTIONS(1), [anon_sym_last_visit] = ACTIONS(1), [anon_sym_last_eventful_visit] = ACTIONS(1), [anon_sym_last_revision] = ACTIONS(1), [anon_sym_last_release] = ACTIONS(1), [anon_sym_created] = ACTIONS(1), [anon_sym_modified] = ACTIONS(1), [anon_sym_published] = ACTIONS(1), [sym_limitField] = ACTIONS(1), [sym_patternField] = ACTIONS(1), [sym_booleanField] = ACTIONS(1), [sym_numericField] = ACTIONS(1), [sym_visitTypeField] = ACTIONS(1), [anon_sym_any] = ACTIONS(1), [anon_sym_bzr] = ACTIONS(1), [anon_sym_cran] = ACTIONS(1), [anon_sym_cvs] = ACTIONS(1), [anon_sym_deb] = ACTIONS(1), [anon_sym_deposit] = ACTIONS(1), [anon_sym_ftp] = ACTIONS(1), [anon_sym_hg] = ACTIONS(1), [anon_sym_git] = ACTIONS(1), [anon_sym_nixguix] = ACTIONS(1), [anon_sym_npm] = ACTIONS(1), [anon_sym_opam] = ACTIONS(1), [anon_sym_pypi] = ACTIONS(1), [anon_sym_svn] = ACTIONS(1), [anon_sym_tar] = ACTIONS(1), [sym_listField] = ACTIONS(1), [sym_dateField] = ACTIONS(1), [sym_rangeOp] = ACTIONS(1), [sym_equalOp] = ACTIONS(1), + [sym_containOp] = ACTIONS(1), [sym_choiceOp] = ACTIONS(1), [sym_isoDateTime] = ACTIONS(1), [sym_number] = ACTIONS(1), [sym_booleanTrue] = ACTIONS(1), [sym_booleanFalse] = ACTIONS(1), [sym_or] = ACTIONS(1), [sym_and] = ACTIONS(1), [sym_escape_sequence] = ACTIONS(1), }, [1] = { [sym_query] = STATE(116), [sym_filters] = STATE(21), [sym_filter] = STATE(31), [sym_patternFilter] = STATE(30), [sym_booleanFilter] = STATE(30), [sym_numericFilter] = STATE(30), [sym_boundedListFilter] = STATE(30), [sym_visitTypeFilter] = STATE(29), [sym_unboundedListFilter] = STATE(30), [sym_dateFilter] = STATE(30), [anon_sym_LPAREN] = ACTIONS(3), [sym_patternField] = ACTIONS(5), [sym_booleanField] = ACTIONS(7), [sym_numericField] = ACTIONS(9), [sym_visitTypeField] = ACTIONS(11), [sym_listField] = ACTIONS(13), [sym_dateField] = ACTIONS(15), }, }; static const uint16_t ts_small_parse_table[] = { [0] = 15, ACTIONS(3), 1, anon_sym_LPAREN, ACTIONS(5), 1, sym_patternField, ACTIONS(7), 1, sym_booleanField, ACTIONS(9), 1, sym_numericField, ACTIONS(11), 1, sym_visitTypeField, ACTIONS(13), 1, sym_listField, ACTIONS(15), 1, sym_dateField, ACTIONS(17), 1, sym_sortByField, ACTIONS(19), 1, sym_limitField, STATE(25), 1, sym_filters, STATE(29), 1, sym_visitTypeFilter, STATE(31), 1, sym_filter, STATE(55), 1, sym_sortBy, STATE(56), 1, sym_limit, STATE(30), 6, sym_patternFilter, sym_booleanFilter, sym_numericFilter, sym_boundedListFilter, sym_unboundedListFilter, sym_dateFilter, [51] = 5, ACTIONS(21), 1, anon_sym_SQUOTE, ACTIONS(23), 1, anon_sym_DQUOTE, STATE(96), 1, sym_visitTypeOptions, ACTIONS(25), 2, anon_sym_COMMA, anon_sym_RBRACK, ACTIONS(27), 15, anon_sym_any, anon_sym_bzr, anon_sym_cran, anon_sym_cvs, anon_sym_deb, anon_sym_deposit, anon_sym_ftp, anon_sym_hg, anon_sym_git, anon_sym_nixguix, anon_sym_npm, anon_sym_opam, anon_sym_pypi, anon_sym_svn, anon_sym_tar, [82] = 5, ACTIONS(29), 1, anon_sym_SQUOTE, ACTIONS(31), 1, anon_sym_DQUOTE, ACTIONS(33), 1, anon_sym_RBRACK, STATE(83), 1, sym_visitTypeOptions, ACTIONS(27), 15, anon_sym_any, anon_sym_bzr, anon_sym_cran, anon_sym_cvs, anon_sym_deb, anon_sym_deposit, anon_sym_ftp, anon_sym_hg, anon_sym_git, anon_sym_nixguix, anon_sym_npm, anon_sym_opam, anon_sym_pypi, anon_sym_svn, anon_sym_tar, [112] = 11, ACTIONS(3), 1, anon_sym_LPAREN, ACTIONS(5), 1, sym_patternField, ACTIONS(7), 1, sym_booleanField, ACTIONS(9), 1, sym_numericField, ACTIONS(11), 1, sym_visitTypeField, ACTIONS(13), 1, sym_listField, ACTIONS(15), 1, sym_dateField, STATE(29), 1, sym_visitTypeFilter, STATE(31), 1, sym_filter, STATE(80), 1, sym_filters, STATE(30), 6, sym_patternFilter, sym_booleanFilter, sym_numericFilter, sym_boundedListFilter, sym_unboundedListFilter, sym_dateFilter, [151] = 2, STATE(106), 1, sym_visitTypeOptions, ACTIONS(27), 15, anon_sym_any, anon_sym_bzr, anon_sym_cran, anon_sym_cvs, anon_sym_deb, anon_sym_deposit, anon_sym_ftp, anon_sym_hg, anon_sym_git, anon_sym_nixguix, anon_sym_npm, anon_sym_opam, anon_sym_pypi, anon_sym_svn, anon_sym_tar, [172] = 11, ACTIONS(3), 1, anon_sym_LPAREN, ACTIONS(5), 1, sym_patternField, ACTIONS(7), 1, sym_booleanField, ACTIONS(9), 1, sym_numericField, ACTIONS(11), 1, sym_visitTypeField, ACTIONS(13), 1, sym_listField, ACTIONS(15), 1, sym_dateField, STATE(25), 1, sym_filters, STATE(29), 1, sym_visitTypeFilter, STATE(31), 1, sym_filter, STATE(30), 6, sym_patternFilter, sym_booleanFilter, sym_numericFilter, sym_boundedListFilter, sym_unboundedListFilter, sym_dateFilter, [211] = 2, STATE(114), 1, sym_visitTypeOptions, ACTIONS(27), 15, anon_sym_any, anon_sym_bzr, anon_sym_cran, anon_sym_cvs, anon_sym_deb, anon_sym_deposit, anon_sym_ftp, anon_sym_hg, anon_sym_git, anon_sym_nixguix, anon_sym_npm, anon_sym_opam, anon_sym_pypi, anon_sym_svn, anon_sym_tar, [232] = 11, ACTIONS(3), 1, anon_sym_LPAREN, ACTIONS(5), 1, sym_patternField, ACTIONS(7), 1, sym_booleanField, ACTIONS(9), 1, sym_numericField, ACTIONS(11), 1, sym_visitTypeField, ACTIONS(13), 1, sym_listField, ACTIONS(15), 1, sym_dateField, STATE(29), 1, sym_visitTypeFilter, STATE(31), 1, sym_filter, STATE(35), 1, sym_filters, STATE(30), 6, sym_patternFilter, sym_booleanFilter, sym_numericFilter, sym_boundedListFilter, sym_unboundedListFilter, sym_dateFilter, [271] = 2, STATE(111), 1, sym_visitTypeOptions, ACTIONS(27), 15, anon_sym_any, anon_sym_bzr, anon_sym_cran, anon_sym_cvs, anon_sym_deb, anon_sym_deposit, anon_sym_ftp, anon_sym_hg, anon_sym_git, anon_sym_nixguix, anon_sym_npm, anon_sym_opam, anon_sym_pypi, anon_sym_svn, anon_sym_tar, [292] = 2, STATE(110), 1, sym_visitTypeOptions, ACTIONS(27), 15, anon_sym_any, anon_sym_bzr, anon_sym_cran, anon_sym_cvs, anon_sym_deb, anon_sym_deposit, anon_sym_ftp, anon_sym_hg, anon_sym_git, anon_sym_nixguix, anon_sym_npm, anon_sym_opam, anon_sym_pypi, anon_sym_svn, anon_sym_tar, [313] = 6, ACTIONS(35), 1, anon_sym_SQUOTE, ACTIONS(37), 1, anon_sym_DQUOTE, ACTIONS(41), 1, anon_sym_DASH, STATE(88), 1, sym_sortByOptions, ACTIONS(39), 2, anon_sym_COMMA, anon_sym_RBRACK, ACTIONS(43), 8, anon_sym_visits, anon_sym_last_visit, anon_sym_last_eventful_visit, anon_sym_last_revision, anon_sym_last_release, anon_sym_created, anon_sym_modified, anon_sym_published, [340] = 6, ACTIONS(41), 1, anon_sym_DASH, ACTIONS(45), 1, anon_sym_SQUOTE, ACTIONS(47), 1, anon_sym_DQUOTE, ACTIONS(49), 1, anon_sym_RBRACK, STATE(86), 1, sym_sortByOptions, ACTIONS(43), 8, anon_sym_visits, anon_sym_last_visit, anon_sym_last_eventful_visit, anon_sym_last_revision, anon_sym_last_release, anon_sym_created, anon_sym_modified, anon_sym_published, [366] = 3, ACTIONS(41), 1, anon_sym_DASH, STATE(108), 1, sym_sortByOptions, ACTIONS(43), 8, anon_sym_visits, anon_sym_last_visit, anon_sym_last_eventful_visit, anon_sym_last_revision, anon_sym_last_release, anon_sym_created, anon_sym_modified, anon_sym_published, [383] = 3, ACTIONS(41), 1, anon_sym_DASH, STATE(123), 1, sym_sortByOptions, ACTIONS(43), 8, anon_sym_visits, anon_sym_last_visit, anon_sym_last_eventful_visit, anon_sym_last_revision, anon_sym_last_release, anon_sym_created, anon_sym_modified, anon_sym_published, [400] = 3, ACTIONS(41), 1, anon_sym_DASH, STATE(122), 1, sym_sortByOptions, ACTIONS(43), 8, anon_sym_visits, anon_sym_last_visit, anon_sym_last_eventful_visit, anon_sym_last_revision, anon_sym_last_release, anon_sym_created, anon_sym_modified, anon_sym_published, [417] = 3, ACTIONS(41), 1, anon_sym_DASH, STATE(113), 1, sym_sortByOptions, ACTIONS(43), 8, anon_sym_visits, anon_sym_last_visit, anon_sym_last_eventful_visit, anon_sym_last_revision, anon_sym_last_release, anon_sym_created, anon_sym_modified, anon_sym_published, [434] = 1, ACTIONS(51), 8, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, anon_sym_COMMA, anon_sym_RBRACK, sym_limitField, sym_or, sym_and, [445] = 1, ACTIONS(53), 8, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, anon_sym_COMMA, anon_sym_RBRACK, sym_limitField, sym_or, sym_and, [456] = 1, ACTIONS(55), 8, anon_sym_visits, anon_sym_last_visit, anon_sym_last_eventful_visit, anon_sym_last_revision, anon_sym_last_release, anon_sym_created, anon_sym_modified, anon_sym_published, [467] = 7, ACTIONS(17), 1, sym_sortByField, ACTIONS(19), 1, sym_limitField, ACTIONS(57), 1, ts_builtin_sym_end, ACTIONS(59), 1, sym_or, ACTIONS(61), 1, sym_and, STATE(51), 1, sym_sortBy, STATE(57), 1, sym_limit, [489] = 1, ACTIONS(63), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [498] = 1, ACTIONS(65), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [507] = 1, ACTIONS(67), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [516] = 1, ACTIONS(69), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [525] = 1, ACTIONS(71), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [534] = 1, ACTIONS(73), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [543] = 1, ACTIONS(75), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [552] = 1, ACTIONS(77), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [561] = 1, ACTIONS(79), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [570] = 1, ACTIONS(81), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [579] = 1, ACTIONS(83), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [588] = 1, ACTIONS(85), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [597] = 1, ACTIONS(87), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [606] = 2, ACTIONS(89), 1, sym_and, ACTIONS(69), 5, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, [617] = 1, ACTIONS(91), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [626] = 1, ACTIONS(93), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [635] = 1, ACTIONS(95), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [644] = 1, ACTIONS(97), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [653] = 1, ACTIONS(99), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [662] = 1, ACTIONS(101), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [671] = 1, ACTIONS(103), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [680] = 1, ACTIONS(105), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [689] = 5, ACTIONS(107), 1, anon_sym_SQUOTE, ACTIONS(109), 1, anon_sym_DQUOTE, ACTIONS(113), 1, sym_singleWord, STATE(99), 1, sym_string, ACTIONS(111), 2, anon_sym_COMMA, anon_sym_RBRACK, [706] = 1, ACTIONS(115), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [715] = 1, ACTIONS(117), 6, ts_builtin_sym_end, anon_sym_RPAREN, sym_sortByField, sym_limitField, sym_or, sym_and, [724] = 5, ACTIONS(107), 1, anon_sym_SQUOTE, ACTIONS(109), 1, anon_sym_DQUOTE, ACTIONS(113), 1, sym_singleWord, ACTIONS(119), 1, anon_sym_RBRACK, STATE(69), 1, sym_string, [740] = 5, ACTIONS(107), 1, anon_sym_SQUOTE, ACTIONS(109), 1, anon_sym_DQUOTE, ACTIONS(113), 1, sym_singleWord, STATE(38), 1, sym_patternVal, STATE(39), 1, sym_string, [756] = 3, STATE(50), 1, aux_sym_stringContent_repeat1, ACTIONS(121), 2, anon_sym_SQUOTE, anon_sym_DQUOTE, ACTIONS(123), 2, aux_sym_stringContent_token1, sym_escape_sequence, [768] = 3, STATE(50), 1, aux_sym_stringContent_repeat1, ACTIONS(125), 2, anon_sym_SQUOTE, anon_sym_DQUOTE, ACTIONS(127), 2, aux_sym_stringContent_token1, sym_escape_sequence, [780] = 4, ACTIONS(19), 1, sym_limitField, ACTIONS(130), 1, ts_builtin_sym_end, ACTIONS(132), 1, sym_and, STATE(119), 1, sym_limit, [793] = 3, STATE(49), 1, aux_sym_stringContent_repeat1, STATE(124), 1, sym_stringContent, ACTIONS(134), 2, aux_sym_stringContent_token1, sym_escape_sequence, [804] = 1, ACTIONS(136), 4, anon_sym_SQUOTE, anon_sym_DQUOTE, anon_sym_COMMA, anon_sym_RBRACK, [811] = 1, ACTIONS(138), 4, anon_sym_SQUOTE, anon_sym_DQUOTE, anon_sym_COMMA, anon_sym_RBRACK, [818] = 4, ACTIONS(19), 1, sym_limitField, ACTIONS(140), 1, ts_builtin_sym_end, ACTIONS(142), 1, sym_and, STATE(109), 1, sym_limit, [831] = 4, ACTIONS(17), 1, sym_sortByField, ACTIONS(140), 1, ts_builtin_sym_end, ACTIONS(144), 1, sym_and, STATE(109), 1, sym_sortBy, [844] = 4, ACTIONS(17), 1, sym_sortByField, ACTIONS(130), 1, ts_builtin_sym_end, ACTIONS(146), 1, sym_and, STATE(119), 1, sym_sortBy, [857] = 1, ACTIONS(148), 4, anon_sym_SQUOTE, anon_sym_DQUOTE, anon_sym_COMMA, anon_sym_RBRACK, [864] = 3, STATE(49), 1, aux_sym_stringContent_repeat1, STATE(120), 1, sym_stringContent, ACTIONS(134), 2, aux_sym_stringContent_token1, sym_escape_sequence, [875] = 2, STATE(41), 1, sym_booleanVal, ACTIONS(150), 2, sym_booleanTrue, sym_booleanFalse, [883] = 3, ACTIONS(19), 1, sym_limitField, ACTIONS(152), 1, ts_builtin_sym_end, STATE(112), 1, sym_limit, [893] = 1, ACTIONS(154), 3, ts_builtin_sym_end, sym_limitField, sym_and, [899] = 3, ACTIONS(17), 1, sym_sortByField, ACTIONS(140), 1, ts_builtin_sym_end, STATE(109), 1, sym_sortBy, [909] = 3, ACTIONS(156), 1, anon_sym_COMMA, ACTIONS(158), 1, anon_sym_RBRACK, STATE(66), 1, aux_sym_sortByVal_repeat1, [919] = 1, ACTIONS(160), 3, ts_builtin_sym_end, sym_limitField, sym_and, [925] = 3, ACTIONS(162), 1, anon_sym_COMMA, ACTIONS(165), 1, anon_sym_RBRACK, STATE(66), 1, aux_sym_sortByVal_repeat1, [935] = 3, ACTIONS(167), 1, anon_sym_COMMA, ACTIONS(169), 1, anon_sym_RBRACK, STATE(73), 1, aux_sym_visitTypeVal_repeat1, [945] = 1, ACTIONS(171), 3, ts_builtin_sym_end, sym_limitField, sym_and, [951] = 3, ACTIONS(173), 1, anon_sym_COMMA, ACTIONS(175), 1, anon_sym_RBRACK, STATE(85), 1, aux_sym_listVal_repeat1, [961] = 3, ACTIONS(156), 1, anon_sym_COMMA, ACTIONS(177), 1, anon_sym_RBRACK, STATE(64), 1, aux_sym_sortByVal_repeat1, [971] = 1, ACTIONS(179), 3, ts_builtin_sym_end, sym_limitField, sym_and, [977] = 1, ACTIONS(181), 3, ts_builtin_sym_end, sym_sortByField, sym_and, [983] = 3, ACTIONS(167), 1, anon_sym_COMMA, ACTIONS(183), 1, anon_sym_RBRACK, STATE(79), 1, aux_sym_visitTypeVal_repeat1, [993] = 3, ACTIONS(19), 1, sym_limitField, ACTIONS(140), 1, ts_builtin_sym_end, STATE(109), 1, sym_limit, [1003] = 3, ACTIONS(17), 1, sym_sortByField, ACTIONS(152), 1, ts_builtin_sym_end, STATE(112), 1, sym_sortBy, [1013] = 3, ACTIONS(156), 1, anon_sym_COMMA, ACTIONS(185), 1, anon_sym_RBRACK, STATE(66), 1, aux_sym_sortByVal_repeat1, [1023] = 1, ACTIONS(187), 3, ts_builtin_sym_end, sym_limitField, sym_and, [1029] = 3, ACTIONS(189), 1, anon_sym_COMMA, ACTIONS(192), 1, anon_sym_RBRACK, STATE(78), 1, aux_sym_listVal_repeat1, [1039] = 3, ACTIONS(194), 1, anon_sym_COMMA, ACTIONS(197), 1, anon_sym_RBRACK, STATE(79), 1, aux_sym_visitTypeVal_repeat1, [1049] = 3, ACTIONS(59), 1, sym_or, ACTIONS(89), 1, sym_and, ACTIONS(199), 1, anon_sym_RPAREN, [1059] = 1, ACTIONS(201), 3, anon_sym_SQUOTE, anon_sym_DQUOTE, sym_singleWord, [1065] = 3, ACTIONS(167), 1, anon_sym_COMMA, ACTIONS(203), 1, anon_sym_RBRACK, STATE(79), 1, aux_sym_visitTypeVal_repeat1, [1075] = 3, ACTIONS(167), 1, anon_sym_COMMA, ACTIONS(205), 1, anon_sym_RBRACK, STATE(82), 1, aux_sym_visitTypeVal_repeat1, [1085] = 1, ACTIONS(207), 3, ts_builtin_sym_end, sym_limitField, sym_and, [1091] = 3, ACTIONS(173), 1, anon_sym_COMMA, ACTIONS(209), 1, anon_sym_RBRACK, STATE(78), 1, aux_sym_listVal_repeat1, [1101] = 3, ACTIONS(156), 1, anon_sym_COMMA, ACTIONS(211), 1, anon_sym_RBRACK, STATE(76), 1, aux_sym_sortByVal_repeat1, [1111] = 2, ACTIONS(213), 1, sym_choiceOp, STATE(98), 1, sym_listOp, [1118] = 1, ACTIONS(215), 2, anon_sym_COMMA, anon_sym_RBRACK, [1123] = 2, ACTIONS(217), 1, sym_number, STATE(43), 1, sym_numberVal, [1130] = 1, ACTIONS(219), 2, anon_sym_COMMA, anon_sym_RBRACK, [1135] = 2, ACTIONS(221), 1, - sym_equalOp, + sym_containOp, STATE(48), 1, sym_patternOp, [1142] = 2, ACTIONS(223), 1, anon_sym_LBRACK, STATE(71), 1, sym_sortByVal, [1149] = 1, ACTIONS(225), 2, anon_sym_COMMA, anon_sym_RBRACK, [1154] = 2, ACTIONS(227), 1, sym_equalOp, STATE(60), 1, sym_booleanOp, [1161] = 2, ACTIONS(229), 1, anon_sym_LBRACK, STATE(45), 1, sym_visitTypeVal, [1168] = 1, ACTIONS(231), 2, anon_sym_COMMA, anon_sym_RBRACK, [1173] = 1, ACTIONS(233), 2, sym_booleanTrue, sym_booleanFalse, [1178] = 2, ACTIONS(235), 1, anon_sym_LBRACK, STATE(46), 1, sym_listVal, [1185] = 1, ACTIONS(237), 2, anon_sym_COMMA, anon_sym_RBRACK, [1190] = 2, ACTIONS(239), 1, sym_equalOp, STATE(92), 1, sym_sortByOp, [1197] = 2, ACTIONS(241), 1, sym_rangeOp, STATE(89), 1, sym_numericOp, [1204] = 2, ACTIONS(243), 1, sym_equalOp, STATE(95), 1, sym_visitTypeOp, [1211] = 2, ACTIONS(245), 1, sym_isoDateTime, STATE(26), 1, sym_dateVal, [1218] = 2, ACTIONS(247), 1, sym_rangeOp, STATE(103), 1, sym_dateOp, [1225] = 1, ACTIONS(249), 1, anon_sym_LBRACK, [1229] = 1, ACTIONS(251), 1, anon_sym_SQUOTE, [1233] = 1, ACTIONS(253), 1, sym_isoDateTime, [1237] = 1, ACTIONS(255), 1, anon_sym_DQUOTE, [1241] = 1, ACTIONS(152), 1, ts_builtin_sym_end, [1245] = 1, ACTIONS(257), 1, anon_sym_SQUOTE, [1249] = 1, ACTIONS(257), 1, anon_sym_DQUOTE, [1253] = 1, ACTIONS(259), 1, ts_builtin_sym_end, [1257] = 1, ACTIONS(255), 1, anon_sym_SQUOTE, [1261] = 1, ACTIONS(251), 1, anon_sym_DQUOTE, [1265] = 1, ACTIONS(261), 1, sym_number, [1269] = 1, ACTIONS(263), 1, ts_builtin_sym_end, [1273] = 1, ACTIONS(265), 1, sym_equalOp, [1277] = 1, ACTIONS(267), 1, anon_sym_LBRACK, [1281] = 1, ACTIONS(140), 1, ts_builtin_sym_end, [1285] = 1, ACTIONS(269), 1, anon_sym_DQUOTE, [1289] = 1, ACTIONS(271), 1, anon_sym_LBRACK, [1293] = 1, ACTIONS(273), 1, anon_sym_SQUOTE, [1297] = 1, ACTIONS(273), 1, anon_sym_DQUOTE, [1301] = 1, ACTIONS(269), 1, anon_sym_SQUOTE, [1305] = 1, ACTIONS(275), 1, sym_number, }; static const uint32_t ts_small_parse_table_map[] = { [SMALL_STATE(2)] = 0, [SMALL_STATE(3)] = 51, [SMALL_STATE(4)] = 82, [SMALL_STATE(5)] = 112, [SMALL_STATE(6)] = 151, [SMALL_STATE(7)] = 172, [SMALL_STATE(8)] = 211, [SMALL_STATE(9)] = 232, [SMALL_STATE(10)] = 271, [SMALL_STATE(11)] = 292, [SMALL_STATE(12)] = 313, [SMALL_STATE(13)] = 340, [SMALL_STATE(14)] = 366, [SMALL_STATE(15)] = 383, [SMALL_STATE(16)] = 400, [SMALL_STATE(17)] = 417, [SMALL_STATE(18)] = 434, [SMALL_STATE(19)] = 445, [SMALL_STATE(20)] = 456, [SMALL_STATE(21)] = 467, [SMALL_STATE(22)] = 489, [SMALL_STATE(23)] = 498, [SMALL_STATE(24)] = 507, [SMALL_STATE(25)] = 516, [SMALL_STATE(26)] = 525, [SMALL_STATE(27)] = 534, [SMALL_STATE(28)] = 543, [SMALL_STATE(29)] = 552, [SMALL_STATE(30)] = 561, [SMALL_STATE(31)] = 570, [SMALL_STATE(32)] = 579, [SMALL_STATE(33)] = 588, [SMALL_STATE(34)] = 597, [SMALL_STATE(35)] = 606, [SMALL_STATE(36)] = 617, [SMALL_STATE(37)] = 626, [SMALL_STATE(38)] = 635, [SMALL_STATE(39)] = 644, [SMALL_STATE(40)] = 653, [SMALL_STATE(41)] = 662, [SMALL_STATE(42)] = 671, [SMALL_STATE(43)] = 680, [SMALL_STATE(44)] = 689, [SMALL_STATE(45)] = 706, [SMALL_STATE(46)] = 715, [SMALL_STATE(47)] = 724, [SMALL_STATE(48)] = 740, [SMALL_STATE(49)] = 756, [SMALL_STATE(50)] = 768, [SMALL_STATE(51)] = 780, [SMALL_STATE(52)] = 793, [SMALL_STATE(53)] = 804, [SMALL_STATE(54)] = 811, [SMALL_STATE(55)] = 818, [SMALL_STATE(56)] = 831, [SMALL_STATE(57)] = 844, [SMALL_STATE(58)] = 857, [SMALL_STATE(59)] = 864, [SMALL_STATE(60)] = 875, [SMALL_STATE(61)] = 883, [SMALL_STATE(62)] = 893, [SMALL_STATE(63)] = 899, [SMALL_STATE(64)] = 909, [SMALL_STATE(65)] = 919, [SMALL_STATE(66)] = 925, [SMALL_STATE(67)] = 935, [SMALL_STATE(68)] = 945, [SMALL_STATE(69)] = 951, [SMALL_STATE(70)] = 961, [SMALL_STATE(71)] = 971, [SMALL_STATE(72)] = 977, [SMALL_STATE(73)] = 983, [SMALL_STATE(74)] = 993, [SMALL_STATE(75)] = 1003, [SMALL_STATE(76)] = 1013, [SMALL_STATE(77)] = 1023, [SMALL_STATE(78)] = 1029, [SMALL_STATE(79)] = 1039, [SMALL_STATE(80)] = 1049, [SMALL_STATE(81)] = 1059, [SMALL_STATE(82)] = 1065, [SMALL_STATE(83)] = 1075, [SMALL_STATE(84)] = 1085, [SMALL_STATE(85)] = 1091, [SMALL_STATE(86)] = 1101, [SMALL_STATE(87)] = 1111, [SMALL_STATE(88)] = 1118, [SMALL_STATE(89)] = 1123, [SMALL_STATE(90)] = 1130, [SMALL_STATE(91)] = 1135, [SMALL_STATE(92)] = 1142, [SMALL_STATE(93)] = 1149, [SMALL_STATE(94)] = 1154, [SMALL_STATE(95)] = 1161, [SMALL_STATE(96)] = 1168, [SMALL_STATE(97)] = 1173, [SMALL_STATE(98)] = 1178, [SMALL_STATE(99)] = 1185, [SMALL_STATE(100)] = 1190, [SMALL_STATE(101)] = 1197, [SMALL_STATE(102)] = 1204, [SMALL_STATE(103)] = 1211, [SMALL_STATE(104)] = 1218, [SMALL_STATE(105)] = 1225, [SMALL_STATE(106)] = 1229, [SMALL_STATE(107)] = 1233, [SMALL_STATE(108)] = 1237, [SMALL_STATE(109)] = 1241, [SMALL_STATE(110)] = 1245, [SMALL_STATE(111)] = 1249, [SMALL_STATE(112)] = 1253, [SMALL_STATE(113)] = 1257, [SMALL_STATE(114)] = 1261, [SMALL_STATE(115)] = 1265, [SMALL_STATE(116)] = 1269, [SMALL_STATE(117)] = 1273, [SMALL_STATE(118)] = 1277, [SMALL_STATE(119)] = 1281, [SMALL_STATE(120)] = 1285, [SMALL_STATE(121)] = 1289, [SMALL_STATE(122)] = 1293, [SMALL_STATE(123)] = 1297, [SMALL_STATE(124)] = 1301, [SMALL_STATE(125)] = 1305, }; static const TSParseActionEntry ts_parse_actions[] = { [0] = {.entry = {.count = 0, .reusable = false}}, [1] = {.entry = {.count = 1, .reusable = false}}, RECOVER(), [3] = {.entry = {.count = 1, .reusable = true}}, SHIFT(5), [5] = {.entry = {.count = 1, .reusable = true}}, SHIFT(91), [7] = {.entry = {.count = 1, .reusable = true}}, SHIFT(94), [9] = {.entry = {.count = 1, .reusable = true}}, SHIFT(101), [11] = {.entry = {.count = 1, .reusable = true}}, SHIFT(102), [13] = {.entry = {.count = 1, .reusable = true}}, SHIFT(87), [15] = {.entry = {.count = 1, .reusable = true}}, SHIFT(104), [17] = {.entry = {.count = 1, .reusable = true}}, SHIFT(100), [19] = {.entry = {.count = 1, .reusable = true}}, SHIFT(117), [21] = {.entry = {.count = 1, .reusable = true}}, SHIFT(11), [23] = {.entry = {.count = 1, .reusable = true}}, SHIFT(10), [25] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_visitTypeVal_repeat1, 1), [27] = {.entry = {.count = 1, .reusable = true}}, SHIFT(58), [29] = {.entry = {.count = 1, .reusable = true}}, SHIFT(6), [31] = {.entry = {.count = 1, .reusable = true}}, SHIFT(8), [33] = {.entry = {.count = 1, .reusable = true}}, SHIFT(28), [35] = {.entry = {.count = 1, .reusable = true}}, SHIFT(16), [37] = {.entry = {.count = 1, .reusable = true}}, SHIFT(15), [39] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_sortByVal_repeat1, 1), [41] = {.entry = {.count = 1, .reusable = true}}, SHIFT(20), [43] = {.entry = {.count = 1, .reusable = true}}, SHIFT(54), [45] = {.entry = {.count = 1, .reusable = true}}, SHIFT(17), [47] = {.entry = {.count = 1, .reusable = true}}, SHIFT(14), [49] = {.entry = {.count = 1, .reusable = true}}, SHIFT(84), [51] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_string, 1), [53] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_string, 3), [55] = {.entry = {.count = 1, .reusable = true}}, SHIFT(53), [57] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_query, 1), [59] = {.entry = {.count = 1, .reusable = true}}, SHIFT(9), [61] = {.entry = {.count = 1, .reusable = true}}, SHIFT(2), [63] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_listVal, 4, .production_id = 5), [65] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_visitTypeVal, 5, .production_id = 7), [67] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_visitTypeVal, 3, .production_id = 4), [69] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_filters, 3, .production_id = 3), [71] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_dateFilter, 3, .production_id = 2), [73] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_listVal, 2), [75] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_visitTypeVal, 2), [77] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_boundedListFilter, 1), [79] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_filter, 1, .production_id = 1), [81] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_filters, 1), [83] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_listVal, 3, .production_id = 4), [85] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_filters, 3), [87] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_visitTypeVal, 4, .production_id = 5), [89] = {.entry = {.count = 1, .reusable = true}}, SHIFT(7), [91] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_visitTypeVal, 6, .production_id = 8), [93] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_dateVal, 1), [95] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_patternFilter, 3, .production_id = 2), [97] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_patternVal, 1), [99] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_booleanVal, 1), [101] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_booleanFilter, 3, .production_id = 2), [103] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_numberVal, 1), [105] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_numericFilter, 3, .production_id = 2), [107] = {.entry = {.count = 1, .reusable = true}}, SHIFT(52), [109] = {.entry = {.count = 1, .reusable = true}}, SHIFT(59), [111] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_listVal_repeat1, 1), [113] = {.entry = {.count = 1, .reusable = true}}, SHIFT(18), [115] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_visitTypeFilter, 3, .production_id = 2), [117] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_unboundedListFilter, 3, .production_id = 2), [119] = {.entry = {.count = 1, .reusable = true}}, SHIFT(27), [121] = {.entry = {.count = 1, .reusable = false}}, REDUCE(sym_stringContent, 1), [123] = {.entry = {.count = 1, .reusable = true}}, SHIFT(50), [125] = {.entry = {.count = 1, .reusable = false}}, REDUCE(aux_sym_stringContent_repeat1, 2), [127] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_stringContent_repeat1, 2), SHIFT_REPEAT(50), [130] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_query, 2), [132] = {.entry = {.count = 1, .reusable = true}}, SHIFT(74), [134] = {.entry = {.count = 1, .reusable = true}}, SHIFT(49), [136] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sortByOptions, 2), [138] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sortByOptions, 1), [140] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_query, 3), [142] = {.entry = {.count = 1, .reusable = true}}, SHIFT(61), [144] = {.entry = {.count = 1, .reusable = true}}, SHIFT(75), [146] = {.entry = {.count = 1, .reusable = true}}, SHIFT(63), [148] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_visitTypeOptions, 1), [150] = {.entry = {.count = 1, .reusable = true}}, SHIFT(40), [152] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_query, 4), [154] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sortByVal, 6, .production_id = 8), [156] = {.entry = {.count = 1, .reusable = true}}, SHIFT(12), [158] = {.entry = {.count = 1, .reusable = true}}, SHIFT(62), [160] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sortByVal, 5, .production_id = 7), [162] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_sortByVal_repeat1, 2, .production_id = 6), SHIFT_REPEAT(12), [165] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_sortByVal_repeat1, 2, .production_id = 6), [167] = {.entry = {.count = 1, .reusable = true}}, SHIFT(3), [169] = {.entry = {.count = 1, .reusable = true}}, SHIFT(23), [171] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sortByVal, 4, .production_id = 5), [173] = {.entry = {.count = 1, .reusable = true}}, SHIFT(44), [175] = {.entry = {.count = 1, .reusable = true}}, SHIFT(32), [177] = {.entry = {.count = 1, .reusable = true}}, SHIFT(65), [179] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sortBy, 3, .production_id = 2), [181] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_limit, 3, .production_id = 2), [183] = {.entry = {.count = 1, .reusable = true}}, SHIFT(36), [185] = {.entry = {.count = 1, .reusable = true}}, SHIFT(68), [187] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sortByVal, 3, .production_id = 4), [189] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_listVal_repeat1, 2, .production_id = 6), SHIFT_REPEAT(44), [192] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_listVal_repeat1, 2, .production_id = 6), [194] = {.entry = {.count = 2, .reusable = true}}, REDUCE(aux_sym_visitTypeVal_repeat1, 2, .production_id = 6), SHIFT_REPEAT(3), [197] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_visitTypeVal_repeat1, 2, .production_id = 6), [199] = {.entry = {.count = 1, .reusable = true}}, SHIFT(33), [201] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_patternOp, 1), [203] = {.entry = {.count = 1, .reusable = true}}, SHIFT(34), [205] = {.entry = {.count = 1, .reusable = true}}, SHIFT(24), [207] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sortByVal, 2), [209] = {.entry = {.count = 1, .reusable = true}}, SHIFT(22), [211] = {.entry = {.count = 1, .reusable = true}}, SHIFT(77), [213] = {.entry = {.count = 1, .reusable = true}}, SHIFT(118), [215] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_sortByVal_repeat1, 2, .production_id = 4), [217] = {.entry = {.count = 1, .reusable = true}}, SHIFT(42), [219] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_sortByVal_repeat1, 4, .production_id = 7), [221] = {.entry = {.count = 1, .reusable = true}}, SHIFT(81), [223] = {.entry = {.count = 1, .reusable = true}}, SHIFT(13), [225] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_visitTypeVal_repeat1, 4, .production_id = 7), [227] = {.entry = {.count = 1, .reusable = true}}, SHIFT(97), [229] = {.entry = {.count = 1, .reusable = true}}, SHIFT(4), [231] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_visitTypeVal_repeat1, 2, .production_id = 4), [233] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_booleanOp, 1), [235] = {.entry = {.count = 1, .reusable = true}}, SHIFT(47), [237] = {.entry = {.count = 1, .reusable = true}}, REDUCE(aux_sym_listVal_repeat1, 2, .production_id = 4), [239] = {.entry = {.count = 1, .reusable = true}}, SHIFT(121), [241] = {.entry = {.count = 1, .reusable = true}}, SHIFT(125), [243] = {.entry = {.count = 1, .reusable = true}}, SHIFT(105), [245] = {.entry = {.count = 1, .reusable = true}}, SHIFT(37), [247] = {.entry = {.count = 1, .reusable = true}}, SHIFT(107), [249] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_visitTypeOp, 1), [251] = {.entry = {.count = 1, .reusable = true}}, SHIFT(67), [253] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_dateOp, 1), [255] = {.entry = {.count = 1, .reusable = true}}, SHIFT(70), [257] = {.entry = {.count = 1, .reusable = true}}, SHIFT(93), [259] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_query, 5), [261] = {.entry = {.count = 1, .reusable = true}}, SHIFT(72), [263] = {.entry = {.count = 1, .reusable = true}}, ACCEPT_INPUT(), [265] = {.entry = {.count = 1, .reusable = true}}, SHIFT(115), [267] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_listOp, 1), [269] = {.entry = {.count = 1, .reusable = true}}, SHIFT(19), [271] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_sortByOp, 1), [273] = {.entry = {.count = 1, .reusable = true}}, SHIFT(90), [275] = {.entry = {.count = 1, .reusable = true}}, REDUCE(sym_numericOp, 1), }; #ifdef __cplusplus extern "C" { #endif #ifdef _WIN32 #define extern __declspec(dllexport) #endif extern const TSLanguage *tree_sitter_swh_search_ql(void) { static const TSLanguage language = { .version = LANGUAGE_VERSION, .symbol_count = SYMBOL_COUNT, .alias_count = ALIAS_COUNT, .token_count = TOKEN_COUNT, .external_token_count = EXTERNAL_TOKEN_COUNT, .state_count = STATE_COUNT, .large_state_count = LARGE_STATE_COUNT, .production_id_count = PRODUCTION_ID_COUNT, .field_count = FIELD_COUNT, .max_alias_sequence_length = MAX_ALIAS_SEQUENCE_LENGTH, .parse_table = &ts_parse_table[0][0], .small_parse_table = ts_small_parse_table, .small_parse_table_map = ts_small_parse_table_map, .parse_actions = ts_parse_actions, .symbol_names = ts_symbol_names, .field_names = ts_field_names, .field_map_slices = ts_field_map_slices, .field_map_entries = ts_field_map_entries, .symbol_metadata = ts_symbol_metadata, .public_symbol_map = ts_symbol_map, .alias_map = ts_non_terminal_alias_map, .alias_sequences = &ts_alias_sequences[0][0], .lex_modes = ts_lex_modes, .lex_fn = ts_lex, }; return &language; } #ifdef __cplusplus } #endif diff --git a/swh/search/query_language/test/corpus/combinations.txt b/swh/search/query_language/test/corpus/combinations.txt index 999ed76..7953e96 100644 --- a/swh/search/query_language/test/corpus/combinations.txt +++ b/swh/search/query_language/test/corpus/combinations.txt @@ -1,82 +1,82 @@ ============================== Empty query (should throw error) ============================== --- (ERROR) ================== Origins with django as keyword, python language, and more than 5 visits ================== -origin = django and language in ["python"] and visits >= 5 +origin : django and language in ["python"] and visits >= 5 --- -(query (filters (filters (filters (filter (patternFilter (patternField) (patternOp (equalOp)) (patternVal (string (singleWord)))))) (and) (filters (filter (unboundedListFilter (listField) (listOp (choiceOp)) (listVal (string (stringContent))))))) (and) (filters (filter (numericFilter (numericField) (numericOp (rangeOp)) (numberVal (number))))))) +(query (filters (filters (filters (filter (patternFilter (patternField) (patternOp (containOp)) (patternVal (string (singleWord)))))) (and) (filters (filter (unboundedListFilter (listField) (listOp (choiceOp)) (listVal (string (stringContent))))))) (and) (filters (filter (numericFilter (numericField) (numericOp (rangeOp)) (numberVal (number))))))) ================== 10 origins with latest revision after 2020-01-01 ================== last_revision > 2020-01-01 limit = 10 --- (query (filters (filter (dateFilter (dateField) (dateOp (rangeOp)) (dateVal (isoDateTime))))) (limit (limitField) (equalOp) (number))) ================== Origins with last visit date not in 2020-2021 (sorted by number of visits) ================== last_visit > 2021-01-01 or last_visit < 2020-01-01 sort_by = ["visits"] --- (query (filters (filters (filter (dateFilter (dateField) (dateOp (rangeOp)) (dateVal (isoDateTime))))) (or) (filters (filter (dateFilter (dateField) (dateOp (rangeOp)) (dateVal (isoDateTime)))))) (sortBy (sortByField) (sortByOp (equalOp)) (sortByVal (sortByOptions)))) ================== Unvisited origins with kubernetes in metadata or minikube in url ================== -visited = false and metadata = "kubernetes" or origin = "minikube" +visited = false and metadata : "kubernetes" or origin : "minikube" --- -(query (filters (filters (filters (filter (booleanFilter (booleanField) (booleanOp (equalOp)) (booleanVal (booleanFalse))))) (and) (filters (filter (patternFilter (patternField) (patternOp (equalOp)) (patternVal (string (stringContent))))))) (or) (filters (filter (patternFilter (patternField) (patternOp (equalOp)) (patternVal (string (stringContent)))))))) +(query (filters (filters (filters (filter (booleanFilter (booleanField) (booleanOp (equalOp)) (booleanVal (booleanFalse))))) (and) (filters (filter (patternFilter (patternField) (patternOp (containOp)) (patternVal (string (stringContent))))))) (or) (filters (filter (patternFilter (patternField) (patternOp (containOp)) (patternVal (string (stringContent)))))))) ================== Origins with "orchestration" or "kubectl" as keywords and language as "go" or "rust" ================== keyword in ["orchestration", "kubectl"] and language in ["go", "rust"] --- (query (filters (filters (filter (unboundedListFilter (listField) (listOp (choiceOp)) (listVal (string (stringContent)) (string (stringContent)))))) (and) (filters (filter (unboundedListFilter (listField) (listOp (choiceOp)) (listVal (string (stringContent)) (string (stringContent)))))))) ================== Origins with a GPL-3 license that have "debian" in their url or have visit type as "deb" ================== -(origin = debian or visit_type = ["deb"]) and license in ["GPL-3"] +(origin : debian or visit_type = ["deb"]) and license in ["GPL-3"] --- -(query (filters (filters (filters (filters (filter (patternFilter (patternField) (patternOp (equalOp)) (patternVal (string (singleWord)))))) (or) (filters (filter (boundedListFilter (visitTypeFilter (visitTypeField) (visitTypeOp (equalOp)) (visitTypeVal (visitTypeOptions)))))))) (and) (filters (filter (unboundedListFilter (listField) (listOp (choiceOp)) (listVal (string (stringContent)))))))) +(query (filters (filters (filters (filters (filter (patternFilter (patternField) (patternOp (containOp)) (patternVal (string (singleWord)))))) (or) (filters (filter (boundedListFilter (visitTypeFilter (visitTypeField) (visitTypeOp (equalOp)) (visitTypeVal (visitTypeOptions)))))))) (and) (filters (filter (unboundedListFilter (listField) (listOp (choiceOp)) (listVal (string (stringContent)))))))) ================== Origins with `and` and `or` inside filter values ================== -(origin = "foo and bar or baz") +(origin : "foo and bar or baz") --- -(query (filters (filters (filter (patternFilter (patternField) (patternOp (equalOp)) (patternVal (string (stringContent)))))))) +(query (filters (filters (filter (patternFilter (patternField) (patternOp (containOp)) (patternVal (string (stringContent)))))))) ================== Origins with `'` and `"` inside filter values ================== -(origin = "foo \\ \'bar\' \"baz\" ") +(origin : "foo \\ \'bar\' \"baz\" ") --- -(query (filters (filters (filter (patternFilter (patternField) (patternOp (equalOp)) (patternVal (string (stringContent (escape_sequence) (escape_sequence) (escape_sequence) (escape_sequence) (escape_sequence))))))))) +(query (filters (filters (filter (patternFilter (patternField) (patternOp (containOp)) (patternVal (string (stringContent (escape_sequence) (escape_sequence) (escape_sequence) (escape_sequence) (escape_sequence))))))))) ================== Incomplete conjunction operators should throw error ================== visits > 5 and --- (query (filters (filter (numericFilter (numericField) (numericOp (rangeOp)) (numberVal (number))))) (ERROR (and))) diff --git a/swh/search/query_language/tokens.js b/swh/search/query_language/tokens.js index 9b29164..c8859cb 100644 --- a/swh/search/query_language/tokens.js +++ b/swh/search/query_language/tokens.js @@ -1,108 +1,110 @@ // Copyright (C) 2021 The Software Heritage developers // See the AUTHORS file at the top-level directory of this distribution // License: GNU General Public License version 3, or any later version // See top-level LICENSE file for more information // Field tokens const visitTypeField = 'visit_type'; const sortByField = 'sort_by'; const limitField = 'limit'; // Field categories const patternFields = ['origin', 'metadata']; const booleanFields = ['visited']; const numericFields = ['visits']; const boundedListFields = [visitTypeField]; const listFields = ['language', 'license', 'keyword']; const dateFields = [ 'last_visit', 'last_eventful_visit', 'last_revision', 'last_release', 'created', 'modified', 'published' ]; const fields = [].concat( patternFields, booleanFields, numericFields, boundedListFields, listFields, dateFields ); // Operators const equalOp = ['=']; +const containOp = [':']; const rangeOp = ['<', '<=', '=', '!=', '>=', '>']; const choiceOp = ['in', 'not in']; // Values const sortByOptions = [ 'visits', 'last_visit', 'last_eventful_visit', 'last_revision', 'last_release', 'created', 'modified', 'published' ]; const visitTypeOptions = [ "any", "bzr", "cran", "cvs", "deb", "deposit", "ftp", "hg", "git", "nixguix", "npm", "opam", "pypi", "svn", "tar" ]; // Extra tokens const OR = "or"; const AND = "and"; const TRUE = "true"; const FALSE = "false"; module.exports = { // Field tokens visitTypeField, sortByField, limitField, // Field categories patternFields, booleanFields, numericFields, boundedListFields, listFields, dateFields, fields, // Operators equalOp, + containOp, rangeOp, choiceOp, // Values sortByOptions, visitTypeOptions, // Extra tokens OR, AND, TRUE, FALSE } diff --git a/swh/search/tests/test_elasticsearch.py b/swh/search/tests/test_elasticsearch.py index 9dc8dd5..7e460d5 100644 --- a/swh/search/tests/test_elasticsearch.py +++ b/swh/search/tests/test_elasticsearch.py @@ -1,192 +1,278 @@ # Copyright (C) 2019-2022 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU General Public License version 3, or any later version # See top-level LICENSE file for more information from datetime import datetime, timedelta, timezone from textwrap import dedent import types import unittest from elasticsearch.helpers.errors import BulkIndexError import pytest from swh.search.exc import SearchQuerySyntaxError from swh.search.metrics import OPERATIONS_METRIC from .test_search import CommonSearchTest now = datetime.now(tz=timezone.utc).isoformat() -now_minus_5_hours = (datetime.now(tz=timezone.utc) - timedelta(hours=5)).isoformat() -now_plus_5_hours = (datetime.now(tz=timezone.utc) + timedelta(hours=5)).isoformat() +now_minus_5_days = (datetime.now(tz=timezone.utc) - timedelta(days=5)).isoformat() +now_plus_5_days = (datetime.now(tz=timezone.utc) + timedelta(days=5)).isoformat() ORIGINS = [ { "url": "http://foobar.1.com", "nb_visits": 1, - "last_visit_date": now_minus_5_hours, - "last_eventful_visit_date": now_minus_5_hours, + "last_visit_date": now_minus_5_days, + "last_eventful_visit_date": now_minus_5_days, }, { "url": "http://foobar.2.com", "nb_visits": 2, "last_visit_date": now, "last_eventful_visit_date": now, }, { "url": "http://foobar.3.com", "nb_visits": 3, - "last_visit_date": now_plus_5_hours, - "last_eventful_visit_date": now_minus_5_hours, + "last_visit_date": now_plus_5_days, + "last_eventful_visit_date": now_minus_5_days, }, { "url": "http://barbaz.4.com", "nb_visits": 3, - "last_visit_date": now_plus_5_hours, - "last_eventful_visit_date": now_minus_5_hours, + "last_visit_date": now_plus_5_days, + "last_eventful_visit_date": now_minus_5_days, }, ] class BaseElasticsearchTest(unittest.TestCase): @pytest.fixture(autouse=True) def _instantiate_search(self, swh_search, elasticsearch_host, mocker): self._elasticsearch_host = elasticsearch_host self.search = swh_search self.mocker = mocker # override self.search.origin_update to catch painless script errors # and pretty print them origin_update = self.search.origin_update def _origin_update(self, *args, **kwargs): script_error = False error_detail = "" try: origin_update(*args, **kwargs) except BulkIndexError as e: error = e.errors[0].get("update", {}).get("error", {}).get("caused_by") if error and "script_stack" in error: script_error = True error_detail = dedent( f""" Painless update script failed ({error.get('reason')}). error type: {error.get('caused_by', {}).get('type')} error reason: {error.get('caused_by', {}).get('reason')} script stack: """ ) error_detail += "\n".join(error["script_stack"]) else: raise e assert script_error is False, error_detail[1:] self.search.origin_update = types.MethodType(_origin_update, self.search) def reset(self): self.search.deinitialize() self.search.initialize() class TestElasticsearchSearch(CommonSearchTest, BaseElasticsearchTest): def test_metrics_update_duration(self): mock = self.mocker.patch("swh.search.metrics.statsd.timing") for url in ["http://foobar.bar", "http://foobar.baz"]: self.search.origin_update([{"url": url}]) assert mock.call_count == 2 def test_metrics_search_duration(self): mock = self.mocker.patch("swh.search.metrics.statsd.timing") for url_pattern in ["foobar", "foobaz"]: self.search.origin_search(url_pattern=url_pattern, with_visit=True) assert mock.call_count == 2 def test_metrics_indexation_counters(self): mock_es = self.mocker.patch("elasticsearch.helpers.bulk") mock_es.return_value = 2, ["error"] mock_metrics = self.mocker.patch("swh.search.metrics.statsd.increment") self.search.origin_update([{"url": "http://foobar.baz"}]) assert mock_metrics.call_count == 2 mock_metrics.assert_any_call( OPERATIONS_METRIC, 2, tags={ "endpoint": "origin_update", "object_type": "document", "operation": "index", }, ) mock_metrics.assert_any_call( OPERATIONS_METRIC, 1, tags={ "endpoint": "origin_update", "object_type": "document", "operation": "index_error", }, ) def test_write_alias_usage(self): mock = self.mocker.patch("elasticsearch.helpers.bulk") mock.return_value = 2, ["result"] self.search.origin_update([{"url": "http://foobar.baz"}]) assert mock.call_args[1]["index"] == "test-write" def test_read_alias_usage(self): mock = self.mocker.patch("elasticsearch.Elasticsearch.search") mock.return_value = {"hits": {"hits": []}} self.search.origin_search(url_pattern="foobar.baz") assert mock.call_args[1]["index"] == "test-read" def test_sort_by_and_limit_query(self): self.search.origin_update(ORIGINS) self.search.flush() def _check_results(query, origin_indices): page = self.search.origin_search(url_pattern="foobar", query=query) results = [r["url"] for r in page.results] assert results == [ORIGINS[index]["url"] for index in origin_indices] _check_results("sort_by = [-visits]", [2, 1, 0]) _check_results("sort_by = [last_visit]", [0, 1, 2]) _check_results("sort_by = [-last_eventful_visit, visits]", [1, 0, 2]) _check_results("sort_by = [last_eventful_visit,-last_visit]", [2, 0, 1]) _check_results("sort_by = [-visits] limit = 1", [2]) _check_results("sort_by = [last_visit] and limit = 2", [0, 1]) _check_results("sort_by = [-last_eventful_visit, visits] limit = 3", [1, 0, 2]) def test_search_ql_simple(self): self.search.origin_update(ORIGINS) self.search.flush() results = { r["url"] - for r in self.search.origin_search(query='origin = "foobar"').results + for r in self.search.origin_search(query='origin : "foobar"').results } assert results == { "http://foobar.1.com", "http://foobar.2.com", "http://foobar.3.com", } + def test_search_ql_datetimes(self): + self.search.origin_update(ORIGINS) + self.search.flush() + + now_minus_5_minutes = ( + datetime.now(tz=timezone.utc) - timedelta(minutes=5) + ).isoformat() + now_plus_5_minutes = ( + datetime.now(tz=timezone.utc) + timedelta(minutes=5) + ).isoformat() + + results = { + r["url"] + for r in self.search.origin_search( + query=( + f"last_visit < {now_minus_5_minutes} " + f"or last_visit > {now_plus_5_minutes}" + ) + ).results + } + assert results == { + "http://foobar.1.com", + "http://foobar.3.com", + "http://barbaz.4.com", + } + + def test_search_ql_dates(self): + self.search.origin_update(ORIGINS) + self.search.flush() + + now_minus_2_days = ( + (datetime.now(tz=timezone.utc) - timedelta(days=2)).date().isoformat() + ) + now_plus_2_days = ( + (datetime.now(tz=timezone.utc) + timedelta(days=2)).date().isoformat() + ) + + results = { + r["url"] + for r in self.search.origin_search( + query=( + f"last_visit < {now_minus_2_days} " + f"or last_visit > {now_plus_2_days}" + ) + ).results + } + assert results == { + "http://foobar.1.com", + "http://foobar.3.com", + "http://barbaz.4.com", + } + + def test_search_ql_visited(self): + self.search.origin_update( + [ + { + "url": "http://foobar.1.com", + "has_visits": True, + "nb_visits": 1, + "last_visit_date": now_minus_5_days, + "last_eventful_visit_date": now_minus_5_days, + }, + {"url": "http://foobar.2.com",}, + {"url": "http://foobar.3.com", "has_visits": False,}, + ] + ) + self.search.flush() + + assert { + r["url"] for r in self.search.origin_search(query="visited = true").results + } == {"http://foobar.1.com"} + assert { + r["url"] for r in self.search.origin_search(query="visited = false").results + } == {"http://foobar.2.com", "http://foobar.3.com"} + + assert ( + self.search.origin_search( + query="visited = true and visited = false" + ).results + == [] + ) + assert ( + self.search.origin_search(query="visited = false", with_visit=True).results + == [] + ) + def test_query_syntax_error(self): self.search.origin_update(ORIGINS) self.search.flush() with pytest.raises(SearchQuerySyntaxError): self.search.origin_search(query="foobar") diff --git a/swh/search/tests/test_translator.py b/swh/search/tests/test_translator.py index 6de5bd5..9789c62 100644 --- a/swh/search/tests/test_translator.py +++ b/swh/search/tests/test_translator.py @@ -1,405 +1,442 @@ # Copyright (C) 2021 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU General Public License version 3, or any later version # See top-level LICENSE file for more information import pytest from swh.search.translator import Translator from swh.search.utils import get_expansion def _test_results(query, expected): output = Translator().parse_query(query) assert output == expected def test_empty_query(): query = "" with pytest.raises(Exception): _test_results(query, {}) def test_conjunction_operators(): query = "visited = true or visits > 2 and visits < 5" expected = { "filters": { "bool": { "should": [ {"term": {"has_visits": True}}, { "bool": { "must": [ {"range": {"nb_visits": {"gt": 2}}}, {"range": {"nb_visits": {"lt": 5}}}, ] } }, ] } } } _test_results(query, expected) +def test_visited(): + query = "visited = true" + expected = { + "filters": {"term": {"has_visits": True}}, + } + _test_results(query, expected) + + query = "visited = false" + expected = { + "filters": { + "bool": { + "should": [ + {"term": {"has_visits": False}}, + {"bool": {"must_not": {"exists": {"field": "has_visits"}}}}, + ] + } + } + } + _test_results(query, expected) + + def test_conjunction_op_precedence_override(): - query = "(visited = false or visits > 2) and visits < 5" + query = "(visited = true or visits > 2) and visits < 5" expected = { "filters": { "bool": { "must": [ { "bool": { "should": [ - {"term": {"has_visits": False}}, + {"term": {"has_visits": True}}, {"range": {"nb_visits": {"gt": 2}}}, ] } }, {"range": {"nb_visits": {"lt": 5}}}, ] } } } _test_results(query, expected) def test_limit_and_sortby(): query = "visited = true sort_by = [-visits,last_visit] limit = 15" expected = { "filters": {"term": {"has_visits": True}}, "sortBy": ["-visits", "last_visit"], "limit": 15, } _test_results(query, expected) def test_deeply_nested_filters(): query = "(((visited = true and visits > 0)))" expected = { "filters": { "bool": { "must": [ {"term": {"has_visits": True},}, {"range": {"nb_visits": {"gt": 0}}}, ] } }, } _test_results(query, expected) def test_origin_and_metadata_filters(): - query = 'origin = django or metadata = "framework and web"' + query = 'origin : django or metadata : "framework and web"' expected = { "filters": { "bool": { "should": [ { "multi_match": { "query": "django", "type": "bool_prefix", "operator": "and", "fields": [ "url.as_you_type", "url.as_you_type._2gram", "url.as_you_type._3gram", ], } }, { "nested": { "path": "intrinsic_metadata", "query": { "multi_match": { "query": "framework and web", "type": "cross_fields", "operator": "and", "fields": ["intrinsic_metadata.*"], "lenient": True, } }, } }, ] } } } _test_results(query, expected) def test_visits_not_equal_to_filter(): query = "visits != 5" expected = { "filters": { "bool": {"must_not": [{"range": {"nb_visits": {"gte": 5, "lte": 5}}},]} }, } _test_results(query, expected) def test_visit_type_filter(): query = 'visit_type = [git,"pypi"]' expected = {"filters": {"terms": {"visit_types": ["git", "pypi"]}}} _test_results(query, expected) def test_keyword_filter(): query = r"""keyword in [word1, "word2 \" \' word3"]""" expected = { "filters": { "nested": { "path": "intrinsic_metadata", "query": { "multi_match": { "query": r"""word1 word2 " ' word3""", "fields": [ get_expansion("keywords", ".") + "^2", get_expansion("descriptions", "."), ], } }, } } } _test_results(query, expected) def test_language_filter(): query = 'language in [python, "go lang", cpp]' expected = { "filters": { "nested": { "path": "intrinsic_metadata", "query": { "bool": { "should": [ { "match": { get_expansion( "programming_languages", "." ): "python" } }, { "match": { get_expansion( "programming_languages", "." ): "go lang" } }, { "match": { get_expansion("programming_languages", "."): "cpp" } }, ] } }, } } } _test_results(query, expected) def test_license_filter(): query = 'license in ["GPL 3", Apache, MIT]' expected = { "filters": { "nested": { "path": "intrinsic_metadata", "query": { "bool": { "should": [ {"match": {get_expansion("licenses", "."): "GPL 3"}}, {"match": {get_expansion("licenses", "."): "Apache"}}, {"match": {get_expansion("licenses", "."): "MIT"}}, ] } }, } } } _test_results(query, expected) def test_date_created_not_equal_to_filter(): query = "created != 2020-01-01" expected = { "filters": { "nested": { "path": "intrinsic_metadata", "query": { "bool": { "must_not": [ { "range": { get_expansion("date_created", "."): { "gte": "2020-01-01", "lte": "2020-01-01", } } } ] } }, } } } _test_results(query, expected) def test_date_created_greater_than_filter(): query = "created >= 2020-01-01" expected = { "filters": { "nested": { "path": "intrinsic_metadata", "query": { "bool": { "must": [ { "range": { get_expansion("date_created", "."): { "gte": "2020-01-01", } } } ] } }, } } } _test_results(query, expected) +def test_visit_date_range(): + query = "last_visit >= 2020-01-01 and last_visit < 2021-01-01" + expected = { + "filters": { + "bool": { + "must": [ + {"range": {"last_visit_date": {"gte": "2020-01-01"}}}, + {"range": {"last_visit_date": {"lt": "2021-01-01"}}}, + ] + } + }, + } + + _test_results(query, expected) + + def test_last_eventful_visit_not_equal_to_filter(): query = "last_visit != 2020-01-01" expected = { "filters": { "bool": { "must_not": [ { "range": { "last_visit_date": { "gte": "2020-01-01", "lte": "2020-01-01", } } } ] } } } _test_results(query, expected) def test_last_eventful_visit_less_than_to_filter(): query = "last_visit < 2020-01-01" expected = {"filters": {"range": {"last_visit_date": {"lt": "2020-01-01"}}}} _test_results(query, expected) def test_keyword_no_escape_inside_filter(): # any keyword (filter name/operator/value) inside a filter # must be considered a string. - query = r'''origin = "language in [\'go lang\', python]"''' + query = r'''origin : "language in [\'go lang\', python]"''' expected = { "filters": { "multi_match": { "query": r"""language in ['go lang', python]""", "type": "bool_prefix", "operator": "and", "fields": [ "url.as_you_type", "url.as_you_type._2gram", "url.as_you_type._3gram", ], } } } _test_results(query, expected) def test_escaped_punctuation_parsing(): query = r"""keyword in ["foo \'\" bar"]""" expected = { "filters": { "nested": { "path": "intrinsic_metadata", "query": { "multi_match": { "query": r"""foo '" bar""", "fields": [ get_expansion("keywords", ".") + "^2", get_expansion("descriptions", "."), ], } }, } } } _test_results(query, expected) def test_nonascii(): query = r"""keyword in ["café"]""" expected = { "filters": { "nested": { "path": "intrinsic_metadata", "query": { "multi_match": { "query": r"""café""", "fields": [ get_expansion("keywords", ".") + "^2", get_expansion("descriptions", "."), ], } }, } } } _test_results(query, expected) def test_nonascii_before_operator(): query = r"""keyword in ["🐍"] and visited = true""" expected = { "filters": { "bool": { "must": [ { "nested": { "path": "intrinsic_metadata", "query": { "multi_match": { "query": r"""🐍""", "fields": [ get_expansion("keywords", ".") + "^2", get_expansion("descriptions", "."), ], } }, }, }, {"term": {"has_visits": True,},}, ], } } } _test_results(query, expected) diff --git a/swh/search/translator.py b/swh/search/translator.py index b8bff14..2e29c71 100644 --- a/swh/search/translator.py +++ b/swh/search/translator.py @@ -1,308 +1,324 @@ # Copyright (C) 2021-2022 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU General Public License version 3, or any later version # See top-level LICENSE file for more information import logging import os import tempfile from pkg_resources import resource_filename from tree_sitter import Language, Parser from swh.search.exc import SearchQuerySyntaxError from swh.search.utils import get_expansion, unescape logger = logging.getLogger(__name__) class Translator: RANGE_OPERATOR_MAP = { ">": "gt", "<": "lt", ">=": "gte", "<=": "lte", } def __init__(self): ql_path = resource_filename("swh.search", "static/swh_ql.so") if not os.path.exists(ql_path): logging.info("%s does not exist, building in temporary directory", ql_path) self._build_dir = tempfile.TemporaryDirectory(prefix="swh.search-build") source_path = resource_filename("swh.search", "query_language") ql_path = os.path.join(self._build_dir.name, "swh_ql.so") Language.build_library(ql_path, [source_path]) search_ql = Language(ql_path, "swh_search_ql") self.parser = Parser() self.parser.set_language(search_ql) self.query = "" def parse_query(self, query): self.query = query.encode() tree = self.parser.parse(self.query) self.query_node = tree.root_node if self.query_node.has_error: raise SearchQuerySyntaxError("Invalid query") return self._traverse(self.query_node) def _traverse(self, node): if len(node.children) == 3 and node.children[1].type == "filters": # filters => ( filters ) return self._traverse(node.children[1]) # Go past the () brackets if node.type == "query": result = {} for child in node.children: # query => filters sort_by limit result[child.type] = self._traverse(child) return result if node.type == "filters": if len(node.children) == 1: # query => filters # filters => filters # filters => filter # Current node is just a wrapper, so go one level deep return self._traverse(node.children[0]) if len(node.children) == 3: # filters => filters conj_op filters filters1 = self._traverse(node.children[0]) conj_op = self._get_value(node.children[1]) filters2 = self._traverse(node.children[2]) if conj_op == "and": # "must" is equivalent to "AND" return {"bool": {"must": [filters1, filters2]}} if conj_op == "or": # "should" is equivalent to "OR" return {"bool": {"should": [filters1, filters2]}} if node.type == "filter": filter_category = node.children[0] return self._parse_filter(filter_category) if node.type == "sortBy": return self._parse_filter(node) if node.type == "limit": return self._parse_filter(node) return Exception( f"Unknown node type ({node.type}) " f"or unexpected number of children ({node.children})" ) def _get_value(self, node): if ( len(node.children) > 0 and node.children[0].type == "[" and node.children[-1].type == "]" ): # array return [self._get_value(child) for child in node.children if child.is_named] start = node.start_point[1] end = node.end_point[1] value = self.query[start:end].decode() if len(value) > 1 and ( (value[0] == "'" and value[-1] == "'") or (value[0] and value[-1] == '"') ): return unescape(value[1:-1]) if node.type in ["number", "numberVal"]: return int(value) return unescape(value) def _parse_filter(self, filter): if filter.type == "boundedListFilter": filter = filter.children[0] children = filter.children assert len(children) == 3 category = filter.type name, op, value = [self._get_value(child) for child in children] if category == "patternFilter": if name == "origin": return { "multi_match": { "query": value, "type": "bool_prefix", "operator": "and", "fields": [ "url.as_you_type", "url.as_you_type._2gram", "url.as_you_type._3gram", ], } } elif name == "metadata": return { "nested": { "path": "intrinsic_metadata", "query": { "multi_match": { "query": value, # Makes it so that the "foo bar" query returns # documents which contain "foo" in a field and "bar" # in a different field "type": "cross_fields", # All keywords must be found in a document for it to # be considered a match. # TODO: allow missing keywords? "operator": "and", # Searches on all fields of the intrinsic_metadata dict, # recursively. "fields": ["intrinsic_metadata.*"], # date{Created,Modified,Published} are of type date "lenient": True, } }, } } if category == "booleanFilter": if name == "visited": - return {"term": {"has_visits": value == "true"}} + if value == "true": + return {"term": {"has_visits": True}} + else: + # non-visited origins will typically not have "has_visits" set + # at all + return { + "bool": { + "should": [ + {"term": {"has_visits": False}}, + { + "bool": { + "must_not": {"exists": {"field": "has_visits"}} + } + }, + ] + } + } if category == "numericFilter": if name == "visits": if op in ["=", "!="]: return { "bool": { ("must" if op == "=" else "must_not"): [ {"range": {"nb_visits": {"gte": value, "lte": value}}} ] } } else: return { "range": {"nb_visits": {self.RANGE_OPERATOR_MAP[op]: value}} } if category == "visitTypeFilter": if name == "visit_type": return {"terms": {"visit_types": value}} if category == "unboundedListFilter": value_array = value if name == "keyword": return { "nested": { "path": "intrinsic_metadata", "query": { "multi_match": { "query": " ".join(value_array), "fields": [ get_expansion("keywords", ".") + "^2", get_expansion("descriptions", "."), # "^2" boosts an origin's score by 2x # if it the queried keywords are # found in its intrinsic_metadata.keywords ], } }, } } elif name in ["language", "license"]: name_mapping = { "language": "programming_languages", "license": "licenses", } name = name_mapping[name] return { "nested": { "path": "intrinsic_metadata", "query": { "bool": { "should": [ {"match": {get_expansion(name, "."): val}} for val in value_array ], } }, } } if category == "dateFilter": if name in ["created", "modified", "published"]: if op in ["=", "!="]: return { "nested": { "path": "intrinsic_metadata", "query": { "bool": { ("must" if op == "=" else "must_not"): [ { "range": { get_expansion(f"date_{name}", "."): { "gte": value, "lte": value, } } } ], } }, } } return { "nested": { "path": "intrinsic_metadata", "query": { "bool": { "must": [ { "range": { get_expansion(f"date_{name}", "."): { self.RANGE_OPERATOR_MAP[op]: value, } } } ], } }, } } else: if op in ["=", "!="]: return { "bool": { ("must" if op == "=" else "must_not"): [ { "range": { f"{name}_date": {"gte": value, "lte": value,} } } ], } } return { "range": { f"{name}_date": { self.RANGE_OPERATOR_MAP[op]: value.replace("Z", "+00:00"), } } } if category == "sortBy": return value if category == "limit": return value raise SearchQuerySyntaxError(f"Unknown filter {category}.{name}")