A sample valid Config file need for Manticore 6

General Discussions - English
1

Hi everyone,

Thanks to manticore team for providinga such great technologie.
I cannot make the last version run on my debian 11. usually i was using version 4 with a config file for plain index. Now my config file is not correctly parsed by the indexer and i have a lot of errors.

Can someone provide me a valid example of plain index sample with a config file ?

2

you could find the config at the installation package.

Could you show what error do you have? Could you post searchd.log from the daemon start to check the errors?

3

Thanks, i found a way to make my config file working, but the 6th version have limitations like maximum of 256 fields, when i try then engine with a limited set of fields (like 4-5 max), everything work fine.
But for my apps i need to index more then 256 fields, so i tried 256 fiedls (text, float, int and date).

No way to make the indexer work, i hav a bug.
Here are the repport :

indexing table ‘idx_baz’…
WARNING: invalid number “”, 0 assumed
WARNING: invalid number “”, 0 assumed
WARNING: invalid number “”, 0 assumed
WARNING: invalid number “”, 0 assumed
*** Oops, indexer crashed! Please send the following report to developers.
Manticore 6.2.0 45680f95d@230804 (columnar 2.2.0 dc33868@230804) (secondary 2.2. 0 dc33868@230804)
-------------- report begins here ---------------
Current document: docid=0, hits=0
Current batch: minid=0, maxid=0
Hit pool start: docid=0, hit=0
-------------- backtrace begins here ---------------
Program compiled with Clang 15.0.7
Configured with flags: Configured with these definitions: -DDISTR_BUILD=bullseye -DUSE_SYSLOG=1 -DWITH_GALERA=1 -DWITH_RE2=1 -DWITH_RE2_FORCE_STATIC=1 -DWITH_ST EMMER=1 -DWITH_STEMMER_FORCE_STATIC=1 -DWITH_NLJSON=1 -DWITH_UNIALGO=1 -DWITH_IC U=1 -DWITH_ICU_FORCE_STATIC=1 -DWITH_SSL=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1 -DDL_ZSTD =1 -DZSTD_LIB=libzstd.so.1 -DWITH_CURL=1 -DDL_CURL=1 -DCURL_LIB=libcurl.so.4 -DW ITH_ODBC=1 -DDL_ODBC=1 -DODBC_LIB=libodbc.so.2 -DWITH_EXPAT=1 -DDL_EXPAT=1 -DEXP AT_LIB=libexpat.so.1 -DWITH_ICONV=1 -DWITH_MYSQL=1 -DDL_MYSQL=1 -DMYSQL_LIB=libm ariadb.so.3 -DWITH_POSTGRESQL=1 -DDL_POSTGRESQL=1 -DPOSTGRESQL_LIB=libpq.so.5 -D LOCALDATADIR=/var/lib/manticore -DFULL_SHARE_DIR=/usr/share/manticore
Built on Linux x86_64 (bullseye) (cross-compiled)
Stack bottom = 0x0, thread stack size = 0x20000
Trying system backtrace:
begin of system symbols:
indexer(_Z12sphBacktraceib+0x22a)[0x5651990307fa]
indexer(_Z7sigsegvi+0xbb)[0x565198f2c4cb]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13140)[0x7f46f3da8140]
indexer(+0xcc5c1b)[0x565198f4dc1b]
indexer(_ZN13CSphIndex_VLN5BuildERKN3sph8Vector_TIP10CSphSourceNS0_13DefaultCopy _TIS3_EENS0_14DefaultRelimitENS0_16DefaultStorage_TIS3_EEEEiiR17CSphIndexProgres s+0xfec)[0x565198f4ad5c]
indexer(_Z7DoIndexRK17CSphConfigSectionPKcRK15CSphOrderedHashIS_10CSphString15CS phStrHashFuncLi256EEP8_IO_FILE+0x1f09)[0x565198f2a249]
indexer(main+0x27b9)[0x565198f2f399]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea)[0x7f46f3be4d0a]
indexer(_start+0x2a)[0x565198f20e3a]
Trying boost backtrace:
0# sphBacktrace(int, bool) in indexer
1# sigsegv(int) in indexer
2# 0x00007F46F3DA8140 in /lib/x86_64-linux-gnu/libpthread.so.0
3# 0x0000565198F4DC1B in indexer
4# CSphIndex_VLN::Build(sph::Vector_T<CSphSource*, sph::DefaultCopy_T<CSphSourc e*>, sph::DefaultRelimit, sph::DefaultStorage_T<CSphSource*> > const&, int, int, CSphIndexProgress&) in indexer
5# DoIndex(CSphConfigSection const&, char const*, CSphOrderedHash<CSphConfigSec tion, CSphString, CSphStrHashFunc, 256> const&, _IO_FILE*) in indexer
6# main in indexer
7# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
8# _start in indexer

-------------- backtrace ends here ---------------

4

you can not make more than 256 fields and can not match these fields either version 6xx or previous versions.

For the WARNING the source of the warning is not clear it could be better to create ticket at GIthub there to put config and data source example that reproduces this warning.

For the crash is the same - please create ticket at Github

5

Thanks for your answer.

Regarded to my sitution, its better for me to step back to an earlier version.
Is there a solution to install a specific version of manticore ? for example the latest version compatible with more than 256 fields ?

Thanks again

6

I sure there is and there was no code that could support more than 256 full-text fields and data format at the index hit structure can not address more than 256 distinct numbers.

That is why it is not clear for me how do you use more fields count in the earlier version or what kind of errors \ warnings do you have now.

7

This warning means that you pass empty string into the INT \ BIGINT attribute and it converts to default value - 0

8

I hav done all modification, now with 208 sql query fields and 228 stored fields (sql_attr_float,sql_attr_bigint,sql_field_string).
And i still have the bug.

My best solution is to go back to older version but i dont know how to install my old version ? Do you know a way to show the version of engine ? Thanks guys for your help

9

I have tried with another mysql table, with the same bug.

Here what i do :

1 - i make sure that the engine is stopped
/etc/init.d/manticore stop
searchd --stop
killall -9 searchd

2 - in index the table
indexer --rotate idx_baz

3 - i finish by launching the engine
searchd -c /etc/manticoresearch/manticore.conf

I have the same bug at step 1

*** Oops, indexer crashed! Please send the following report to developers.
Manticore 6.2.0 45680f95d@230804 (columnar 2.2.0 dc33868@230804) (secondary 2.2.0 dc33868@230804)
-------------- report begins here ---------------
Current document: docid=0, hits=0
Current batch: minid=0, maxid=0
Hit pool start: docid=0, hit=0
-------------- backtrace begins here ---------------
Program compiled with Clang 15.0.7
Configured with flags: Configured with these definitions: -DDISTR_BUILD=bullseye -DUSE_SYSLOG=1 -DWITH_GALERA=1 -DWITH_RE2=1 -DWITH_RE2_FORCE_STATIC=1 -DWITH_STEMMER=1 -DWITH_STEMMER_FORCE_STATIC=1 -DWITH_NLJSON=1 -DWITH_UNIALGO=1 -DWITH_ICU=1 -DWITH_ICU_FORCE_STATIC=1 -DWITH_SSL=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1 -DDL_ZSTD=1 -DZSTD_LIB=libzstd.so.1 -DWITH_CURL=1 -DDL_CURL=1 -DCURL_LIB=libcurl.so.4 -DWITH_ODBC=1 -DDL_ODBC=1 -DODBC_LIB=libodbc.so.2 -DWITH_EXPAT=1 -DDL_EXPAT=1 -DEXPAT_LIB=libexpat.so.1 -DWITH_ICONV=1 -DWITH_MYSQL=1 -DDL_MYSQL=1 -DMYSQL_LIB=libmariadb.so.3 -DWITH_POSTGRESQL=1 -DDL_POSTGRESQL=1 -DPOSTGRESQL_LIB=libpq.so.5 -DLOCALDATADIR=/var/lib/manticore -DFULL_SHARE_DIR=/usr/share/manticore
Built on Linux x86_64 (bullseye) (cross-compiled)
Stack bottom = 0x0, thread stack size = 0x20000
Trying system backtrace:
begin of system symbols:
indexer(_Z12sphBacktraceib+0x22a)[0x5576f892d7fa]
indexer(_Z7sigsegvi+0xbb)[0x5576f88294cb]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13140)[0x7f5529304140]
indexer(+0xcc5d09)[0x5576f884ad09]
indexer(_ZN13CSphIndex_VLN5BuildERKN3sph8Vector_TIP10CSphSourceNS0_13DefaultCopy_TIS3_EENS0_14DefaultRelimitENS0_16DefaultStorage_TIS3_EEEEiiR17CSphIndexProgress+0xfec)[0x5576f8847d5c]
indexer(_Z7DoIndexRK17CSphConfigSectionPKcRK15CSphOrderedHashIS_10CSphString15CSphStrHashFuncLi256EEP8_IO_FILE+0x1f09)[0x5576f8827249]
indexer(main+0x27b9)[0x5576f882c399]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea)[0x7f5529140d0a]
indexer(_start+0x2a)[0x5576f881de3a]
Trying boost backtrace:
0# sphBacktrace(int, bool) in indexer
1# sigsegv(int) in indexer
2# 0x00007F5529304140 in /lib/x86_64-linux-gnu/libpthread.so.0
3# 0x00005576F884AD09 in indexer
4# CSphIndex_VLN::Build(sph::Vector_T<CSphSource*, sph::DefaultCopy_T<CSphSource*>, sph::DefaultRelimit, sph::DefaultStorage_T<CSphSource*> > const&, int, int, CSphIndexProgress&) in indexer
5# DoIndex(CSphConfigSection const&, char const*, CSphOrderedHash<CSphConfigSection, CSphString, CSphStrHashFunc, 256> const&, _IO_FILE*) in indexer
6# main in indexer
7# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
8# _start in indexer

-------------- backtrace ends here ---------------

10

It could be better to create ticket at Github there to put reproducible example - config, data source that recreates indexer crash

However 208 field + 228 stored fields - already more than 256 fields and you also have sql_field_string that creates field + attribute

11

Thanks
but no solutions so far, can someone help me on installing older version of manticore ?

12

You can find all older release packages in https://repo.manticoresearch.com/

1 Like
13

Thanks