Page MenuHomeSoftware Heritage

Some swh-search tests are failing since the release of elasticsearch 7.10
Closed, MigratedEdits Locked

Description

It looks like the new release of elasticsearch (7.10) introduced some breaking changes as some tests of swh-search are now failing.

For the record, I have executed the tests using the previous elasticsearch release (7.9.3) and they all passed.

Event Timeline

anlambert triaged this task as Normal priority.Nov 17 2020, 5:53 PM
anlambert created this task.
vlorentz raised the priority of this task from Normal to High.
vlorentz added a subscriber: vlorentz.

I believe this to be a bug, ES ignores a specificed tokenizer (P871, while earlier versions behaved like this: P872).

I reported it here: https://discuss.elastic.co/t/es-7-10-ignores-analyzer-in-as-you-type-fields/256113

The upstream elasticsearch bug is fixed but not released yet. In the mean time, I think we can pin the version of ES in the jenkins docker image to one that doesn't have the bug (which I think @ardumont wants to do).

yes, i will try to do that and open a diff ;)

yes, i will try to do that and open a diff ;)

D4675

And as usual with this kind of diff, i'm unsure how to properly test it.

I just ensured through vagrant that i could indeed use the apt-get install -y elasticsearch=7.9.3
stanza and it worked, beats me how to do better though.

Build failed [1]

There is a check done at the end of the build which checks for upgrades and fails if there are some.
In that case, there is an upgrade possible from 7.9.3 to 7.10.0 which we don't want (we do want 7.9.3).
So the solution is to actually add a pin configuration version on 7.9.3 so version > 7.10.0 are ignored by apt.

Thanks to @olasd to suggest it to me ;)

I'll attend to that soon.

[1] https://jenkins.softwareheritage.org/view/all/job/jenkins-tools/job/swh-jenkins-dockerfiles/lastBuild/consoleFull

So swh-search build is back to green with ^
And the pile of diffs on search as well
\m/

Given that the pinning is only excluding the 7.10.0, as soon as the 7.10.1
(assuming it's the release version with the fix), this should continue working with that version onwards.

So we should be able to close this.

I realized that dev wil have the problem locally as well (well most likely
temporarily for such value of temporary...)

So I adapted the readme to explicit what needs to happen locally in D4679.

Let's close this, everything is green (well blue).

I'm currently stuck on the debian packaging which i'll track in another task.