Manticore Search On Centos8 Stream using MySQL 5.7

webbgroup · April 12, 2024, 4:38pm

Search Daemon starts up fine. But the indexer doesn’t work or connect.

We are using MySql 5.7 community-client libraries.
and can connect just fine from the command line.

configuration looks something like below, obviously cricital things removed.

source something_data_src
{
  type = mysql
  sql_host = localhost
  sql_user = somethinguser
  sql_pass = <redacted>
  sql_db = something
  sql_port = 3306
  sql_query_pre =
  sql_range_step = 200
}

RPM’s installed:

[manticore@apache-centos81 ~]$ rpm -qa | grep manticore
manticore-server-core-6.2.12_230822.dc5144d35-1.el8.x86_64
manticore-tools-6.2.12_230822.dc5144d35-1.el8.x86_64
manticore-icudata-5.0.3_221123.d2d9e5e56-1.el7.centos.noarch
manticore-common-6.2.12_230822.dc5144d35-1.el8.noarch
manticore-server-6.2.12_230822.dc5144d35-1.el8.x86_64
manticore-backup-1.0.8_23080408.f7638f9-1.noarch
manticore-6.2.12_230822.dc5144d35-1.el8.x86_64
manticore-repo-0.2.0-1.noarch
manticore-devel-6.2.12_230822.dc5144d35-1.el8.noarch
manticore-buddy-1.0.18_23080408.2befdbe-1.noarch
[manticore@apache-centos81 ~]$ rpm -qa | grep mysql
mysql-community-common-5.7.44-1.el7.x86_64
mysql-community-libs-5.7.44-1.el7.x86_64
mysql-community-client-5.7.44-1.el7.x86_64
mysql-selinux-1.0.10-1.el8.noarch
mysql-community-libs-compat-5.7.44-1.el7.x86_64
mysql-community-server-5.7.44-1.el7.x86_64
mysql80-community-release-el7-5.noarch

mysql connectivity validation.

[manticore@apache-centos81 ~]$ mysql -u somethinguser -h localhost something -p 
Enter password: 
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 16547
Server version: 5.7.44 MySQL Community Server (GPL)

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

mysql> \s
--------------
mysql  Ver 14.14 Distrib 5.7.44, for Linux (x86_64) using  EditLine wrapper

Connection id:		16547
Current database:	something
Current user:		somethinguser@localhost
SSL:			Not in use
Current pager:		stdout
Using outfile:		''
Using delimiter:	;
Server version:		5.7.44 MySQL Community Server (GPL)
Protocol version:	10
Connection:		Localhost via UNIX socket
Server characterset:	latin1
Db     characterset:	utf8
Client characterset:	utf8
Conn.  characterset:	utf8
UNIX socket:		/var/lib/mysql/mysql.sock
Uptime:			6 hours 17 min 40 sec

Threads: 1  Questions: 75052  Slow queries: 0  Opens: 545  Flush tables: 1  Open tables: 538  Queries per second avg: 3.312
--------------

indexing attempt.

indexing table 'something_delta_idx'...
ERROR: table 'something_delta_idx': sql_connect: MySQL source wasn't initialized. Wrong name in dlopen? (DSN=mysql://somethinguser:***@localhost:3306/something).
total 0 docs, 0 bytes
total 0.000 sec, 0 bytes/sec, 0.00 docs/sec

[manticore@apache-centos81 ~]$ ldd $(which searchd)
	linux-vdso.so.1 (0x00007ffccbb23000)
	libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007fa392a27000)
	libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007fa39253c000)
	libz.so.1 => /lib64/libz.so.1 (0x00007fa392324000)
	libanl.so.1 => /lib64/libanl.so.1 (0x00007fa392120000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fa391d9e000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fa391b9a000)
	libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fa391805000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa3915ed000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa3913cd000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fa390ff7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa394f3c000)

tomat · April 13, 2024, 12:35pm

could you check your mysql client that it can connect on the same host and port as you use in your index_source as in your mysql client output it said
Connection: Localhost via UNIX socket
but it should be
Connection: 127.0.0.1 via TCP/IP

could you connect as mysql -h localhost -P 3306 -u somethinguser -p ?

webbgroup · April 13, 2024, 1:19pm

Thank you for responding. Yes, see the local mysql connectivity validation included within the original post.

ChatGPT says that I should be seeing a mysqlclient library in the ldd command. Do you know if this is accurate? AND/OR does manticore have native mysql support w/o having to load a library?

The odd situation with our setup. is that we are using the el7 mysql clients even though we are on Centos8 Stream. The reason for this situation is that we are using mysql 5.7, not 8.

tomat · April 13, 2024, 1:25pm

my output similar to your provided

mysql -u test -h localhost

mysql> \s
--------------
Connection:             Localhost via UNIX socket

and that way it uses unix socket but if I use the command I ask you to use it uses TCP as indexer use

mysql -u test -h 127.0.0.1 -P 3306

mysql> \s
--------------
Connection:             127.0.0.1 via TCP/IP

I sure that you test with client one way to connect but daemon uses another way to connect

webbgroup · April 13, 2024, 1:50pm

Both are successful. Additionally, we were using Sphinx previously without any issues connectivity wise.

[user@apache-centos81]$ mysql -u somethinguser -h localhost -P 3306 something -p
Enter password: 
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 5.7.44 MySQL Community Server (GPL)

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> \s
--------------
mysql  Ver 14.14 Distrib 5.7.44, for Linux (x86_64) using  EditLine wrapper

Connection id:		13
Current database:	something
Current user:		somethinguser@localhost
SSL:			Not in use
Current pager:		stdout
Using outfile:		''
Using delimiter:	;
Server version:		5.7.44 MySQL Community Server (GPL)
Protocol version:	10
Connection:		Localhost via UNIX socket
Server characterset:	latin1
Db     characterset:	utf8
Client characterset:	utf8
Conn.  characterset:	utf8
UNIX socket:		/var/lib/mysql/mysql.sock
Uptime:			48 min 24 sec

Threads: 1  Questions: 159  Slow queries: 0  Opens: 178  Flush tables: 1  Open tables: 171  Queries per second avg: 0.054
--------------

mysql> exit
Bye

[user@apache-centos81]$ mysql -u somethinguser -h 127.0.0.1 -P 3306 something -p
Enter password: 
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 15
Server version: 5.7.44 MySQL Community Server (GPL)

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> \s
--------------
mysql  Ver 14.14 Distrib 5.7.44, for Linux (x86_64) using  EditLine wrapper

Connection id:		15
Current database:	something
Current user:		somethinguser@localhost
SSL:			Cipher in use is ECDHE-RSA-AES128-GCM-SHA256
Current pager:		stdout
Using outfile:		''
Using delimiter:	;
Server version:		5.7.44 MySQL Community Server (GPL)
Protocol version:	10
Connection:		127.0.0.1 via TCP/IP
Server characterset:	latin1
Db     characterset:	utf8
Client characterset:	utf8
Conn.  characterset:	utf8
TCP port:		3306
Uptime:			49 min 0 sec

Threads: 1  Questions: 182  Slow queries: 0  Opens: 194  Flush tables: 1  Open tables: 187  Queries per second avg: 0.061
--------------

mysql>

webbgroup · April 13, 2024, 2:27pm

Changed it to the 127.0.0.1 and still nothing.

sql_connect: MySQL source wasn't initialized. Wrong name in dlopen? (DSN=mysql://somethinguser:***@127.0.0.1:3306/something).

Sergey · April 18, 2024, 5:00am

There’s an article about it MySQL source wasn't initialized. Wrong name in dlopen?

Here’s the general way of dealing with it:

run indexer w/o any flags, you’ll see there smth like:

-DMYSQL_LIB=/opt/homebrew/opt/mysql-client/lib/libmysqlclient.21.dylib

or

-DMYSQL_LIB=libmysqlclient.so.21

or in your case:

-DMYSQL_LIB=libmariadb.so.3

Now you need to make sure you have this file in your OS. To find which package provides it in Centos 8 you can do this:

[root@df6e535a83b3 /]# yum whatprovides /usr/lib64/libmariadb.so.3
Last metadata expiration check: 1:18:40 ago on Thu 18 Apr 2024 03:37:09 AM UTC.
mariadb-connector-c-3.1.11-2.el8_3.x86_64 : The MariaDB Native Client library (C driver)
Repo        : appstream
Matched from:
Filename    : /usr/lib64/libmariadb.so.3

It’s important to NOT install a package for another architecture, e.g. it won’t work with this package:

[root@df6e535a83b3 usr]# yum whatprovides libmariadb.so.3
Last metadata expiration check: 0:08:19 ago on Thu 18 Apr 2024 03:37:09 AM UTC.
mariadb-connector-c-3.1.11-2.el8_3.i686 : The MariaDB Native Client library (C driver)
Repo        : appstream
Matched from:
Provide    : libmariadb.so.3

Install it:

yum install mariadb-connector-c-3.1.11-2.el8_3.x86_64

Now indexer should be able to find and use the library:

[root@df6e535a83b3 usr]# ls -la /usr/lib64/libmariadb.so.3
-rwxr-xr-x 1 root root 348808 Dec 17  2020 /usr/lib64/libmariadb.so.3