Page MenuHomeSoftware Heritage
Paste P1364

swh search kibana profiling
ActivePublic
Actions

Authored by KShivendu on May 16 2022, 8:52 PM.
[
// Default query template (Don't run)
{
"query": {
"bool": {
"must": ["filters"]
}
}
},
// visited = true
{
"query": {
"bool": {
"must": [{"term": {"has_visits": true}}]
}
}
},
// (((visited = true and visits > 0)))
{
"query": {
"bool": {
"must": [{
"bool": {
"must": [
{"term": {"has_visits": true}},
{"range": {"nb_visits": {"gt": 0}}}
]
}
}]
}
}
},
// origin : django and metadata : "web"
{
"query": {
"bool": {
"must": [{
"bool": {
"must": [
{
"multi_match": {
"query": "django",
"type": "bool_prefix",
"operator": "and",
"fields": [
"url.as_you_type",
"url.as_you_type._2gram",
"url.as_you_type._3gram"
]
}
},
{
"nested": {
"path": "intrinsic_metadata",
"query": {
"multi_match": {
"query": "web",
"type": "cross_fields",
"operator": "and",
"fields": ["intrinsic_metadata.*"],
"lenient": true
}
}
}
}
]
}
}]
}
}
},
// origin : django
{
"query": {
"bool": {
"must": [{
"bool": {
"should": [
{
"multi_match": {
"query": "django",
"type": "bool_prefix",
"operator": "and",
"fields": [
"url.as_you_type",
"url.as_you_type._2gram",
"url.as_you_type._3gram"
]
}
}
]
}
}]
}
}
},
// metadata : "web"
{
"query": {
"bool": {
"must": [{
"bool": {
"should": [
{
"nested": {
"path": "intrinsic_metadata",
"query": {
"multi_match": {
"query": "web",
"type": "cross_fields",
"operator": "and",
"fields": ["intrinsic_metadata.*"],
"lenient": true
}
}
}
}
]
}
}]
}
}
},
// origin : django or metadata : "web"
{
"query": {
"bool": {
"must": [{
"bool": {
"should": [
{
"multi_match": {
"query": "django",
"type": "bool_prefix",
"operator": "and",
"fields": [
"url.as_you_type",
"url.as_you_type._2gram",
"url.as_you_type._3gram"
]
}
},
{
"nested": {
"path": "intrinsic_metadata",
"query": {
"multi_match": {
"query": "framework and web",
"type": "cross_fields",
"operator": "and",
"fields": ["intrinsic_metadata.*"],
"lenient": true
}
}
}
}
]
}
}]
}
}
},
// visits != 5
{
"query": {
"bool": {
"must": [{
"bool": {"must_not": [{"range": {"nb_visits": {"gte": 5, "lte": 5}}}]}
}]
}
}
},
// visit_type = [git,"pypi"]
{
"query": {
"bool": {
"must": [
{"terms": {"visit_types": ["git", "pypi"]}}
]
}
}
}
]

Event Timeline

KShivendu created this paste.May 16 2022, 8:52 PM
KShivendu created this object in space S1 Public.
KShivendu added a comment.May 16 2022, 8:54 PM

Partial docker-compose.search.yml for introducing Kibana:

services:
  kibana:
    image: docker.elastic.co/kibana/kibana:7.15.2
    environment:
      SERVER_NAME: kibana.example.org
      ELASTICSEARCH_HOSTS: '["http://elasticsearch:9200"]'
    ports:
      - 5601:5601
KShivendu edited the content of this paste. (Show Details)May 16 2022, 8:58 PM
KShivendu added a comment.May 16 2022, 9:00 PM

Please run the above-mentioned queries using the Kibana search profiler as shown in the attached screenshot.

KShivendu edited the content of this paste. (Show Details)May 16 2022, 9:08 PM
KShivendu mentioned this in T4251: [swh-search] Investigate long search queries response time.May 17 2022, 8:49 AM
About · Developer information · Legal