There is no occurrences of this error in the logs and the consumers don't have any lag, so yes, I guess it is.
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Feed Advanced Search
Advanced Search
Advanced Search
Apr 25 2022
Apr 25 2022
anlambert closed T4192: Unstuck swh.search build as Resolved by committing rDSEA377ba2c8c194: conftest: Fix tests hang with elasticsearch 7.17.3.
Apr 20 2022
Apr 20 2022
vsellier closed T3671: staging - swh-search (metadata indexer) is unable to update a document due to an unparseable date as Resolved.
vlorentz added a comment to T3671: staging - swh-search (metadata indexer) is unable to update a document due to an unparseable date.
@vsellier This is fixed, right?
Apr 19 2022
Apr 19 2022
vlorentz closed Restricted Maniphest Task, a subtask of T3926: Better syntax errors for the search query language, as Resolved.
Mar 23 2022
Mar 23 2022
bchauvet updated the task description for T3952: Make the search query language a first class citizen .
Mar 3 2022
Mar 3 2022
vlorentz reopened Restricted Maniphest Task, a subtask of T3926: Better syntax errors for the search query language, as Open.
vlorentz closed Restricted Maniphest Task, a subtask of T3926: Better syntax errors for the search query language, as Resolved.
Feb 23 2022
Feb 23 2022
anlambert closed T3967: "Link" header is not properly displayed in apidoc when it contains [], a subtask of T3559: Enable the swh-search QL in staging, as Resolved.
Feb 22 2022
Feb 22 2022
vlorentz added a parent task for T3558: Enable the swh-search QL in production: T3952: Make the search query language a first class citizen .
zack added a subtask for T3952: Make the search query language a first class citizen : T3560: Polish the swh-search QL.
zack added a parent task for T3560: Polish the swh-search QL: T3952: Make the search query language a first class citizen .
vsellier added a comment to T3784: swh-search / staging: transient timeouts on elasticsearch queries.
After the elasticsearch restart, there is no more message relative to any gc overhead in the logs but there were a couple of timeouts during the night.
Further investigations are needed
Feb 21 2022
Feb 21 2022
vsellier added a comment to T3784: swh-search / staging: transient timeouts on elasticsearch queries.
Elastisearch was restarted and the sentry issues closed.
Let's monitor if the gcs are coming coming again
vsellier added a comment to T3784: swh-search / staging: transient timeouts on elasticsearch queries.
first, clean the unused resources, even if it will not free a lot of resources:
- aliases cleanup
vsellier@search-esnode0 ~ % export ES_SERVER=192.168.130.80:9200
vsellier@search-esnode0 ~ % curl -XGET http://$ES_SERVER/_cat/aliases
origin-read origin-v0.11 - - - -
origin-write origin-v0.11 - - - -
origin-v0.9.0-read origin-v0.9.0 - - - -
origin-v0.9.0-write origin-v0.9.0 - - - -
vsellier@search-esnode0 ~ % curl -XDELETE http://$ES_SERVER/origin-v0.9.0/_alias/origin-v0.9.0-read
{"acknowledged":true}% vsellier@search-esnode0 ~ % curl -XDELETE -H "Content-Type: application/json" http://$ES_SERVER/origin-v0.9.0/_alias/origin-v0.9.0-write
{"acknowledged":true}%vlorentz closed T3927: swh-search crashes when "only show origins visited at least once" is not checked, a subtask of T3560: Polish the swh-search QL, as Resolved.
vlorentz closed T3930: Update swh-search QL examples, a subtask of T3560: Polish the swh-search QL, as Resolved.
vlorentz closed Restricted Maniphest Task, a subtask of T3560: Polish the swh-search QL, as Resolved.
vlorentz closed T3931: Pagination UI does not work at all when using the search QL, a subtask of T3560: Polish the swh-search QL, as Resolved.
vsellier changed the status of T3784: swh-search / staging: transient timeouts on elasticsearch queries from Open to Work in Progress.
Feb 18 2022
Feb 18 2022
ardumont closed T3944: Deploy swh-search v0.13.0, a subtask of T3927: swh-search crashes when "only show origins visited at least once" is not checked, as Resolved.
ardumont closed T3944: Deploy swh-search v0.13.0, a subtask of T3930: Update swh-search QL examples, as Resolved.
@ardumont I'm neither able to access my account nor reset my password (Not receiving email from sentry.softwareheritage.org).
Is it possible to do something from your side?
Feb 17 2022
Feb 17 2022
ardumont moved T3944: Deploy swh-search v0.13.0 from in-progress to deployed/landed/monitoring on the System administration board.
ardumont changed the status of T3944: Deploy swh-search v0.13.0, a subtask of T3930: Update swh-search QL examples, from Open to Work in Progress.
ardumont changed the status of T3944: Deploy swh-search v0.13.0, a subtask of T3927: swh-search crashes when "only show origins visited at least once" is not checked, from Open to Work in Progress.
vsellier added a comment to T3784: swh-search / staging: transient timeouts on elasticsearch queries.
looks like the server is short in heap
[2022-02-17T15:26:30,847][INFO ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][5965188] overhead, spent [408ms] collecting in the last [1s]
[2022-02-17T15:27:08,154][INFO ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][5965225] overhead, spent [296ms] collecting in the last [1s]
[2022-02-17T15:29:31,383][WARN ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][young][5965368][3283] duration [1s], collections [1]/[1.1s], total [1s]/[5.8m], memory [8.2gb]->[5.4gb]/[16gb], all_pools {[young] [2.8gb]->[0b]/[0b]}{[old] [4.7gb]->[5.3gb]/[16gb]}{[survivor] [652mb]->[184mb]/[0b]}
[2022-02-17T15:29:31,384][WARN ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][5965368] overhead, spent [1s] collecting in the last [1.1s]
[2022-02-17T15:31:49,449][INFO ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][5965506] overhead, spent [260ms] collecting in the last [1s]
[2022-02-17T15:33:46,505][INFO ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][5965623] overhead, spent [256ms] collecting in the last [1s]
[2022-02-17T15:37:11,728][INFO ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][5965828] overhead, spent [372ms] collecting in the last [1s]
[2022-02-17T15:47:19,087][INFO ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][5966435] overhead, spent [289ms] collecting in the last [1s]
[2022-02-17T15:49:56,439][INFO ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][5966592] overhead, spent [315ms] collecting in the last [1.1s]
[2022-02-17T15:55:40,579][INFO ][o.e.m.j.JvmGcMonitorService] [search-esnode0] [gc][5966936] overhead, spent [274ms] collecting in the last [1s]ardumont moved T3944: Deploy swh-search v0.13.0 from Backlog to Weekly backlog on the System administration board.
Feb 16 2022
Feb 16 2022
ardumont closed T3943: rebuild python3-tree-sitter for python 3.10, a subtask of T3944: Deploy swh-search v0.13.0, as Resolved.
ardumont moved T3943: rebuild python3-tree-sitter for python 3.10 from in-progress to deployed/landed/monitoring on the System administration board.
ardumont changed the status of T3943: rebuild python3-tree-sitter for python 3.10 from Open to Work in Progress.
ardumont changed the status of T3943: rebuild python3-tree-sitter for python 3.10, a subtask of T3944: Deploy swh-search v0.13.0, from Open to Work in Progress.
Package python3-tree-sitter rebuilt [1].
ardumont moved T3943: rebuild python3-tree-sitter for python 3.10 from Backlog to Weekly backlog on the System administration board.
vlorentz added a subtask for T3560: Polish the swh-search QL: T3926: Better syntax errors for the search query language.
vlorentz added a parent task for T3926: Better syntax errors for the search query language: T3560: Polish the swh-search QL.
vlorentz added parent tasks for T3944: Deploy swh-search v0.13.0: Unknown Object (Maniphest Task), T3930: Update swh-search QL examples, T3927: swh-search crashes when "only show origins visited at least once" is not checked, Unknown Object (Maniphest Task).
vlorentz added a subtask for T3930: Update swh-search QL examples: T3944: Deploy swh-search v0.13.0.
vlorentz added a subtask for T3944: Deploy swh-search v0.13.0: T3943: rebuild python3-tree-sitter for python 3.10.
vlorentz added a parent task for T3943: rebuild python3-tree-sitter for python 3.10: T3944: Deploy swh-search v0.13.0.
Feb 14 2022
Feb 14 2022
vlorentz triaged T3931: Pagination UI does not work at all when using the search QL as Normal priority.
I agree that having actually "working" examples in the doc is a must.
vlorentz triaged T3927: swh-search crashes when "only show origins visited at least once" is not checked as Normal priority.
vlorentz closed T3910: Add an option to enable the swh-search QL for some users, a subtask of T3909: Get feedback on the swh-search QL, as Resolved.
Feb 13 2022
Feb 13 2022
@ardumont request sent, thanks :)
Feb 12 2022
Feb 12 2022
@ardumont request sent, thanks :)
@bchauvet you might as well request access to sentry [1] as well ;)
Also, is it possible to give me access to Sentry?
@vlorentz Can you check what's the query (swh ql) for this request using sentry?
Feb 11 2022
Feb 11 2022
Great feature !
Possible invalid queries [WIP]:
vlorentz added a subtask for T3926: Better syntax errors for the search query language: Unknown Object (Maniphest Task).
Feb 10 2022
Feb 10 2022
In T3910#78645, @KShivendu wrote:
- When exactly did it go live?
Yes, I can see the option. It's awesome to see 3 months of my hard work being live :D
Thanks a lot @ardumont
So can you please include me in the beta tester list?
Hi @vlorentz, I wasn't aware that the SWH staff members are using the query language. I thought that it is yet to be used by anyone. I'd like to try it and maybe even restart contributing whenever I'm free :)
Feb 7 2022
Feb 7 2022
vlorentz triaged T3910: Add an option to enable the swh-search QL for some users as Normal priority.
Jan 10 2022
Jan 10 2022
The build of swh-search is now ok in master and swh-envrionment.
The swh-environment build is still failing but it's due to the swh-graph tests flakiness (it will be solved in T3831)
I don't know, it was pinned so I kept it pinned but yes as the versions specified in the diffs are the last ones, I guess we can remove it
Why are we pinning tree-sitter at all?
vsellier added a revision to T3838: swh-search build is ko: D6897: Unpin the tree-sitter dependency.
Dec 13 2021
Dec 13 2021
Explicitly setting the LIBFFI_TMPDIR environment variable indeed fixes the hang: D6828.