search finds word in field where there is no such word

General Discussions - English
1

Hi,

i have a strange problem. when searching with manticore search i get a hit in a field where is no such word which makes problem with my custom ranking.

Details:
Table:

create table test1 (id bigint, titel text, autor text) min_prefix_len='1' min_infix_len='2' index_exact_words='1' morphology='lemmatize_de_all' morphology_skip_fields='autor' min_stemming_len='4' expand_keywords='exact';

Data:

replace into test1 (titel,autor) values ('Selbstbestimmt Sterben fragezeichen  Ein theologisches Gespräch mit dem Roman \"Ein ganzes halbes Jahr\" von Jojo Moyes','Maren Klingelhöfer');

Query:

SELECT id,* ,packedfactors() as dbg1 FROM test1 where match ('@@relaxed (@(titel,autor) sterben moyes jojo  ) ') option ranker=expr('1') ; show meta;
+--------------------+------------------------------------------------------------------------------------------------------------------------+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                 | titel                                                                                                                  | autor               | dbg1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+--------------------+------------------------------------------------------------------------------------------------------------------------+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 290380135511621636 | Selbstbestimmt Sterben fragezeichen  Ein theologisches Gespräch mit dem Roman "Ein ganzes halbes Jahr" von Jojo Moyes  | Maren Klingelhöfer  | bm25=500, bm25a=0.500000, field_mask=3, doc_word_count=3, field0=(lcs=1, hit_count=3, word_count=3, tf_idf=0.000000, min_idf=0.000000, max_idf=0.000000, sum_idf=0.000000, min_hit_pos=2, min_best_span_pos=2, exact_hit=0, max_window_hits=1, min_gaps=12, exact_order=0, lccs=1, wlccs=0.000000, atc=0.000000), field1=(lcs=1, hit_count=1, word_count=1, tf_idf=0.000000, min_idf=0.000000, max_idf=0.000000, sum_idf=0.000000, min_hit_pos=1, min_best_span_pos=1, exact_hit=0, max_window_hits=1, min_gaps=0, exact_order=0, lccs=1, wlccs=0.000000, atc=0.000000), word0=(tf=1, idf=0.000000), word1=(tf=2, idf=0.000000), word2=(tf=1, idf=0.000000) |
+--------------------+------------------------------------------------------------------------------------------------------------------------+---------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+----------------+----------+
| Variable_name  | Value    |
+----------------+----------+
| total          | 1        |
| total_found    | 1        |
| total_relation | eq       |
| time           | 0.000    |
| keyword[0]     | =jojo    |
| docs[0]        | 1        |
| hits[0]        | 1        |
| keyword[1]     | =moyes   |
| docs[1]        | 1        |
| hits[1]        | 2        |
| keyword[2]     | =sterben |
| docs[2]        | 1        |
| hits[2]        | 1        |
| keyword[3]     | jojo     |
| docs[3]        | 1        |
| hits[3]        | 1        |
| keyword[4]     | moyes    |
| docs[4]        | 1        |
| hits[4]        | 1        |
| keyword[5]     | sterben  |
| docs[5]        | 1        |
| hits[5]        | 1        |
+----------------+----------+

so it finds in field 1 (autor) also the word “=moyes” - but the field contains not this word.
why ?

maybe its because of the morphing - but the create table statement says it should ignore the autor column for morphing. and “moyes” is far away from “maren”

BTW:
Server version: 6.0.4 1a3a4ea82@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306) git branch HEAD (no branch)

am i missing something?

regards
Stefan Gutschireiter

2

stripped it down to the morphplogy option:

mysql> create table test1 (id bigint, titel text, autor text ) min_prefix_len='1' min_infix_len='2' index_exact_words='1' min_stemming_len='4' expand_keywords='exact';
Query OK, 0 rows affected (0.00 sec)

mysql> replace into test1 (titel,autor) values ('Selbstbestimmt Sterben fragezeichen  Ein theologisches Gespräch mit dem Roman \"Ein ganzes halbes Jahr\" von Jojo Moyes','Maren Klingelhöfer');
Query OK, 1 row affected (0.01 sec)

mysql> SELECT id,* ,packedfactors() as dbg1 FROM test1 where match ('@@relaxed (@(autor) =moyes  ) ') option ranker=expr('1') ; show meta;
Empty set (0.00 sec)

+----------------+--------+
| Variable_name  | Value  |
+----------------+--------+
| total          | 0      |
| total_found    | 0      |
| total_relation | eq     |
| time           | 0.000  |
| keyword[0]     | =moyes |
| docs[0]        | 1      |
| hits[0]        | 1      |
+----------------+--------+
7 rows in set (0.00 sec)

mysql> SELECT id,* ,packedfactors() as dbg1 FROM test1 where match ('@@relaxed (@(titel,autor) moyes  ) ') option ranker=expr('1') ; show meta;
+--------------------+------------------------------------------------------------------------------------------------------------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                 | titel                                                                                                                  | autor               | dbg1                                                                                                                                                                                                                                                                                                                                          |
+--------------------+------------------------------------------------------------------------------------------------------------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 290380135511621637 | Selbstbestimmt Sterben fragezeichen  Ein theologisches Gespräch mit dem Roman "Ein ganzes halbes Jahr" von Jojo Moyes  | Maren Klingelhöfer  | bm25=500, bm25a=0.500000, field_mask=1, doc_word_count=1, field0=(lcs=1, hit_count=2, word_count=1, tf_idf=0.000000, min_idf=0.000000, max_idf=0.000000, sum_idf=0.000000, min_hit_pos=16, min_best_span_pos=16, exact_hit=0, max_window_hits=1, min_gaps=0, exact_order=1, lccs=1, wlccs=0.000000, atc=0.000000), word0=(tf=2, idf=0.000000) |
+--------------------+------------------------------------------------------------------------------------------------------------------------+---------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

+----------------+--------+
| Variable_name  | Value  |
+----------------+--------+
| total          | 1      |
| total_found    | 1      |
| total_relation | eq     |
| time           | 0.000  |
| keyword[0]     | =moyes |
| docs[0]        | 1      |
| hits[0]        | 1      |
| keyword[1]     | moyes  |
| docs[1]        | 1      |
| hits[1]        | 1      |
+----------------+--------+
10 rows in set (0.00 sec)

now it doesn’t find the word in the autor column.
but why does he find the word in this column (especially the exact word “=moyes”) when morpholgy turned on?
first of all shouldn’t it find the word “moyes” and not “=moyes” if morpholgy is involved.
and when i specified to skip this files for morpholgy should the query not behave like in my second test?

3

Looks like a bug to me. I’ve created an issue on Github - Match in a wrong field · Issue #1140 · manticoresoftware/manticoresearch · GitHub

4

thanks !

5

@steven2612 the bug has been fixed. Thanks for reporting.

6

Thank you !