Creating an index

General Discussions - English
1

I chose Manticore today! Already started the server. I’m reading the documentation.

Question: Is it impossible to create an index from a ready-made MySQL database like in Sphinx? Or do I need to write a script to transfer?

2

Yes, it’s possible - Manticore Search Manual: Data creation and modification > Adding data from external storages > Plain tables creation

https://play.manticoresearch.com/mysql

1 Like
3

Hello, I have the same issue, coming from the Sphinix background.

I have existing tables. I want to index all the columns. Even though not frequent I would like to insert record to the table and take the advantage of RT index.

I’m moving to a new server and thought ok let’s go with Manticore instead of Sphinix.

I moved my MySQL db to new server.
Where do I setup the config file for the indexer, I only want to index some tables? I’m missing info on that and can’t find it in docs.
Please help
Thank you

4

Where do I setup the config file for the indexer, I only want to index some tables?

Nothing changed in terms of that. Setup it as previously and run the same indexer command with the same parameters.

1 Like
5

Thank you is there are sample config somewhere?
Because I can start searchd with the default manticore.conf.
But when I add source and index details to the config file. I can’t start search

I even copied the following to the config, but searchd will not start
searchd {
listen = 127.0.0.1:9312
listen = 127.0.0.1:9306:mysql
listen = 127.0.0.1:9308:http
log = /var/log/manticore/searchd.log
query_log = /var/log/manticore/query.log
pid_file = /var/run/manticore/searchd.pid
data_dir = /var/lib/manticore
}

source sakila_film {
type = mysql
sql_host = 127.0.0.1
sql_user = user
sql_pass = pass123
sql_db = sakila
sql_port = 3306
sql_query_pre = SET NAMES utf8
sql_query = SELECT film_id, title, description,category_id from film
sql_field_string = title
sql_field_string = description
sql_attr_uint = release_year
sql_attr_uint = category_id
}

table sakila_film {
type = plain
source = sakila_film
path = /home/sa/testplain
}

I get following error
Job for manticore.service failed because the control process exited with error code.
See “systemctl status manticore.service” and “journalctl -xeu manticore.service” for details.

But if I have the following in the config file searchd will start.
searchd {
listen = 127.0.0.1:9312
listen = 127.0.0.1:9306:mysql
listen = 127.0.0.1:9308:http
log = /var/log/manticore/searchd.log
query_log = /var/log/manticore/query.log
pid_file = /var/run/manticore/searchd.pid
data_dir = /var/lib/manticore
}

6

it could be better to post searchd.log to make sure what error you have but from config you provided you can not miix the data_dir option (RT mode) with the index definition in the config.

It could be better to get use the the all modes searchd provides via plain table mode and RT mode

7

you could also read the difference at our manual https://manual.manticoresearch.com/Read_this_first#Real-time-mode-vs-plain-mode

8

image
image1888×1102 260 KB

image
image3312×1256 497 KB

9

Thank you @tomat I only want plain mode. Because I already have existing tables. I’m hoping to build plain+delta indexes. We already have that working on a different project with Sphinx. What I’m doing wrong in the log file. I only want to start with 2 plain files.
When I add the source and index details I get the following error. When I tried to start.

sudo systemctl start manticore

Job for manticore.service failed because the control process exited with error code.
See “systemctl status manticore.service” and “journalctl -xeu manticore.service” for details.

Thank you

10

@Sergey Thank you, I only want the plain mode, Is my config wrong? Following is my actual config file.

I get following error
Job for manticore.service failed because the control process exited with error code.
See “systemctl status manticore.service” and “journalctl -xeu manticore.service” for details.

searchd {
listen = 127.0.0.1:9312
listen = 127.0.0.1:9306:mysql
listen = 127.0.0.1:9308:http
log = /var/log/manticore/searchd.log
query_log = /var/log/manticore/query.log
pid_file = /var/run/manticore/searchd.pid
data_dir = /var/lib/manticore
}

source kyc_portal {
type = mysql
sql_host = 127.0.0.1
sql_user = xxxxxx
sql_pass = xxxxx
sql_db = newdb
sql_port = 3306 # optional, default is 3306

sql_query = SELECT record_no AS id,CAPTION AS body,Head_detail AS title,urn FROM unique_tp_data

sql_field_string = title

  #sql_attr_uint    = category_id
  #sql_field_string = title

}

index kyc_portal {
type = plain
source = kyc_portal
path = /var/lib/manticore/kyc_portal

min_word_len = 2

charset_table = non_cjk

}

11

Yes, it’s wrong. You can’t have data_dir and source/index at the same time. You can have:

  • either source and index/table - then it’s called “Plain mode”
  • or data_dir - then it’s called “RT mode”.