manticore docker container crashes when manticore.conf is mapped

General Discussions - English
1

Following the instructions from the official documentation, I ran this command to start a manticore docker container:

docker run -e EXTRA=1 --name manticore -v $(pwd)/data:/var/lib/manticore -p 127.0.0.1:9306:9306 -p 127.0.0.1:9308:9308 -d manticoresearch/manticore

This connection works correctly as verified below:

/tmp$ curl -X POST 'http://127.0.0.1:9308/sql' -d 'mode=raw&query=CREATE TABLE testrt ( title text, content text, gid integer)'
[{"total":0,"warning":"","error":"table 'testrt': CREATE TABLE failed: table 'testrt' already exists"}]

However, when I map a manticore.conf file, then the container crashes on startup.

docker run -e EXTRA=1 --name manticore -v $(pwd)/manticore.conf:/etc/manticoresearch/manticore.conf -v $(pwd)/data:/var/lib/manticore/ -p 127.0.0.1:9306:9306 -p 127.0.0.1:9308:9308 -u manticore -d manticoresearch/manticore

The docker logs shows the following:

/tmp$ docker logs manticore
Manticore 6.2.12 dc5144d35@230822 (columnar 2.2.4 5aec342@230822) (secondary 2.2.4 5aec342@230822)
[Fri Mar  8 05:40:40.090 2024] [1] using config file '/etc/manticoresearch/manticore.conf' (9538 chars)...
starting daemon version '6.2.12 dc5144d35@230822 (columnar 2.2.4 5aec342@230822) (secondary 2.2.4 5aec342@230822)' ...
listening on all interfaces for mysql, port=9306
listening on UNIX socket /var/run/mysqld/mysqld.sock
, error -2: Name or service not known172.17.0.3
shutdown daemon version '6.2.12 dc5144d35@230822 (columnar 2.2.4 5aec342@230822) (secondary 2.2.4 5aec342@230822)' ...
shutdown complete
Manticore 6.2.12 dc5144d35@230822 (columnar 2.2.4 5aec342@230822) (secondary 2.2.4 5aec342@230822)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
Copyright (c) 2017-2023, Manticore Software LTD (https://manticoresearch.com)

To narrow down the problem, I went into the “working” container and copied over the its manticore.conf onto my host machine. Then I stopped the working container and started a new one by mounting that copied manticore.conf file. Specifically note that I have only one manticore container instance running at any time, so this avoids any potential issues related to port conflicts, etc.

Since the manticore.conf contents is exactly the same and in the exact same location, and there’s no other difference (that I’m aware of) in the configuration, I’d expect the container to behave exactly the same. But this isn’t happening. What’s the problem here and how could I resolve it?

Here is the full contents of the manticore.conf (without any modification to the default file contents):

#!/bin/sh
ip=`hostname -i|rev|cut -d\  -f 1|rev`
cat << EOF
searchd {
    # https://manual.manticoresearch.com/Server_settings/Searchd#access_plain_attrs
    # access_plain_attrs = mmap_preread

    # https://manual.manticoresearch.com/Server_settings/Searchd#access_blob_attrs
    # access_blob_attrs = mmap_preread

    # https://manual.manticoresearch.com/Server_settings/Searchd#access_doclists
    # access_doclists = file

    # https://manual.manticoresearch.com/Server_settings/Searchd#access_hitlists
    # access_hitlists = file

    # https://manual.manticoresearch.com/Server_settings/Searchd#agent_connect_timeout
    # agent_connect_timeout =

    # https://manual.manticoresearch.com/Server_settings/Searchd#agent_query_timeout
    # agent_query_timeout =

    # https://manual.manticoresearch.com/Server_settings/Searchd#agent_retry_count
    # agent_retry_count = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#agent_retry_delay
    # agent_retry_delay = 500

    # https://manual.manticoresearch.com/Server_settings/Searchd#attr_flush_period
    # attr_flush_period = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#binlog_flush
    # binlog_flush = 2

    # https://manual.manticoresearch.com/Server_settings/Searchd#binlog_max_log_size
    # binlog_max_log_size = 268435456

    # https://manual.manticoresearch.com/Server_settings/Searchd#binlog_path
    # binlog_path =

    # https://manual.manticoresearch.com/Server_settings/Searchd#client_timeout
    # client_timeout = 300

    # https://manual.manticoresearch.com/Server_settings/Searchd#collation_libc_locale
    # collation_libc_locale = C

    # https://manual.manticoresearch.com/Server_settings/Searchd#collation_server
    # collation_server = libc_ci

    # https://manual.manticoresearch.com/Server_settings/Searchd#data_dir
    data_dir = /var/lib/manticore

    # https://manual.manticoresearch.com/Server_settings/Searchd#docstore_cache_size
    # docstore_cache_size = 16m

    # https://manual.manticoresearch.com/Server_settings/Searchd#expansion_limit
    # expansion_limit = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#grouping_in_utc
    # grouping_in_utc = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#ha_period_karma
    # ha_period_karma = 60

    # https://manual.manticoresearch.com/Server_settings/Searchd#ha_ping_interval
    # ha_ping_interval = 1000

    # https://manual.manticoresearch.com/Server_settings/Searchd#hostname_lookup
    # hostname_lookup =

    # https://manual.manticoresearch.com/Server_settings/Searchd#jobs_queue_size
    # jobs_queue_size =

    # https://manual.manticoresearch.com/Server_settings/Searchd#listen_backlog
    # listen_backlog = 5

    # https://manual.manticoresearch.com/Server_settings/Searchd#listen
    # listen_env = this directive allows to append listeners from environment variables

    listen = 9306:mysql41
    listen = /var/run/mysqld/mysqld.sock:mysql41
    listen = $ip:9312
    listen = 9308:http
    listen = $ip:9315-9325:replication

    # https://manual.manticoresearch.com/Server_settings/Searchd#listen_tfo
    # listen_tfo = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#log
    log = /var/log/manticore/searchd.log

    # https://manual.manticoresearch.com/Server_settings/Searchd#max_batch_queries
    # max_batch_queries = 32

    # https://manual.manticoresearch.com/Server_settings/Searchd#threads
    # threads =

    # https://manual.manticoresearch.com/Server_settings/Searchd#max_filters
    # max_filters = 256

    # https://manual.manticoresearch.com/Server_settings/Searchd#max_filter_values
    # max_filter_values = 4096

    # https://manual.manticoresearch.com/Server_settings/Searchd#max_open_files
    # max_open_files =

    # https://manual.manticoresearch.com/Server_settings/Searchd#max_packet_size
    max_packet_size = 128M

    # https://manual.manticoresearch.com/Server_settings/Searchd#mysql_version_string
    # mysql_version_string =

    # https://manual.manticoresearch.com/Server_settings/Searchd#net_workers
    # net_workers = 1

    # https://manual.manticoresearch.com/Server_settings/Searchd#net_wait_tm
    # net_wait_tm = -1

    # https://manual.manticoresearch.com/Server_settings/Searchd#net_throttle_accept
    # net_throttle_accept = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#net_throttle_action
    # net_throttle_action = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#node_address
    # node_address =

    # https://manual.manticoresearch.com/Server_settings/Searchd#ondisk_attrs_default
    # ondisk_attrs_default = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#persistent_connections_limit
    # persistent_connections_limit =

    # https://manual.manticoresearch.com/Server_settings/Searchd#pid_file
    pid_file = /var/run/manticore/searchd.pid

    # https://manual.manticoresearch.com/Server_settings/Searchd#predicted_time_costs
    # predicted_time_costs = doc=64, hit=48, skip=2048, match=64

    # https://manual.manticoresearch.com/Server_settings/Searchd#preopen_indexes
    # preopen_indexes = 1

    # https://manual.manticoresearch.com/Server_settings/Searchd#qcache_max_bytes
    # qcache_max_bytes = 16Mb

    # https://manual.manticoresearch.com/Server_settings/Searchd#qcache_thresh_msec
    # qcache_thresh_msec = 3000

    # https://manual.manticoresearch.com/Server_settings/Searchd#qcache_ttl_sec
    # qcache_ttl_sec = 60

    # https://manual.manticoresearch.com/Server_settings/Searchd#query_log_format
    query_log_format = sphinxql

    # https://manual.manticoresearch.com/Server_settings/Searchd#query_log_min_msec
    # query_log_min_msec = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#query_log
    # query_log = /var/log/manticore/query.log

    # https://manual.manticoresearch.com/Server_settings/Searchd#query_log_mode
    # query_log_mode = 600

    # https://manual.manticoresearch.com/Server_settings/Searchd#max_connections
    # max_connections =

    # https://manual.manticoresearch.com/Server_settings/Searchd#network_timeout
    # network_timeout = 5

    # https://manual.manticoresearch.com/Server_settings/Searchd#read_buffer
    # read_buffer = 256K

    # https://manual.manticoresearch.com/Server_settings/Searchd#read_buffer_docs
    # read_buffer_docs = 256K

    # https://manual.manticoresearch.com/Server_settings/Searchd#read_buffer_hits
    # read_buffer_hits = 256K

    # https://manual.manticoresearch.com/Server_settings/Searchd#read_unhinted
    # read_unhinted 32K

    # https://manual.manticoresearch.com/Server_settings/Searchd#rt_flush_period
    # rt_flush_period =

    # https://manual.manticoresearch.com/Server_settings/Searchd#rt_merge_iops
    # rt_merge_iops = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#rt_merge_maxiosize
    # rt_merge_maxiosize = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#seamless_rotate
    # seamless_rotate = 1

    # https://manual.manticoresearch.com/Server_settings/Searchd#server_id
    # server_id =

    # https://manual.manticoresearch.com/Server_settings/Searchd#shutdown_timeout
    # shutdown_timeout = 3

    # https://manual.manticoresearch.com/Server_settings/Searchd#shutdown_token
    # shutdown_token =

    # https://manual.manticoresearch.com/Server_settings/Searchd#snippets_file_prefix
    # snippets_file_prefix =

    # https://manual.manticoresearch.com/Server_settings/Searchd#sphinxql_state
    # sphinxql_state =

    # https://manual.manticoresearch.com/Server_settings/Searchd#sphinxql_timeout
    # sphinxql_timeout = 900

    # https://manual.manticoresearch.com/Server_settings/Searchd#ssl_ca
    # ssl_ca =

    # https://manual.manticoresearch.com/Server_settings/Searchd#ssl_cert
    # ssl_cert =

    # https://manual.manticoresearch.com/Server_settings/Searchd#ssl_key
    # ssl_key =

    # https://manual.manticoresearch.com/Server_settings/Searchd#subtree_docs_cache
    # subtree_docs_cache = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#subtree_hits_cache
    # subtree_hits_cache = 0

    # https://manual.manticoresearch.com/Server_settings/Searchd#thread_stack
    # thread_stack =

    # https://manual.manticoresearch.com/Server_settings/Searchd#unlink_old
    # unlink_old = 1

    # https://manual.manticoresearch.com/Server_settings/Searchd#watchdog
    # watchdog = 1

    # https://manual.manticoresearch.com/Server_settings/Searchd#secondary_indexes
    # secondary_indexes = 1
}

common {

    # https://manual.manticoresearch.com/Server_settings/Common#lemmatizer_base
    # lemmatizer_base = /usr/local/share

    # https://manual.manticoresearch.com/Server_settings/Common#progressive_merge
    # progressive_merge =

    # https://manual.manticoresearch.com/Server_settings/Common#json_autoconv_keynames
    # json_autoconv_keynames =

    # https://manual.manticoresearch.com/Server_settings/Common#json_autoconv_numbers
    # json_autoconv_numbers = 0

    # https://manual.manticoresearch.com/Server_settings/Common#on_json_attr_error
    # on_json_attr_error = ignore_attr

    # https://manual.manticoresearch.com/Server_settings/Common#plugin_dir
    # plugin_dir =

}

EOF
2

Can you share your manticore.conf? I can’t reproduce it:

➜  dragondive ls -la
total 8
drwxr-xr-x    4 sn  staff    128  9 Mar 00:47 .
drwxr-x---+ 372 sn  staff  11904  9 Mar 00:48 ..
drwxr-xr-x    5 sn  staff    160  9 Mar 00:48 data
-rw-r--r--    1 sn  staff    193  9 Mar 00:47 manticore.conf

➜  dragondive ls -la data/
total 0
drwxr-xr-x  2 sn  staff   64  9 Mar 00:49 .
drwxr-xr-x  4 sn  staff  128  9 Mar 00:47 ..

➜  dragondive cat manticore.conf
searchd {
    listen = 9306:mysql
    listen = 9312
    listen = 9308
    log = /var/log/manticore/searchd.log
    pid_file = /var/run/manticore/searchd.pid
    data_dir = /var/lib/manticore
}

➜  dragondive docker run -e EXTRA=1 --name manticore -v $(pwd)/manticore.conf:/etc/manticoresearch/manticore.conf -v $(pwd)/data:/var/lib/manticore/ -p 127.0.0.1:9306:9306 -p 127.0.0.1:9308:9308 -u manticore -d manticoresearch/manticore
a3de8ffdaf0756b73e221d195a100d0c7a6ac37d9a1cb6c4d3336725718c29be

➜  dragondive docker logs -n 10 manticore
listening on all interfaces for sphinx and http(s), port=9312
listening on all interfaces for sphinx and http(s), port=9308
prereading 0 tables
preread 0 tables in 0.000 sec
accepting connections
[BUDDY] started v1.0.18 '/usr/share/manticore/modules/manticore-buddy/bin/manticore-buddy --listen=http://0.0.0.0:9312  --threads=8' at http://127.0.0.1:37117
[BUDDY] Loaded plugins:
[BUDDY]   core: empty-string, backup, emulate-elastic, insert, select, show, cli-table, plugin, test, insert-mva
[BUDDY]   local:
[BUDDY]   extra:
➜  dragondive

➜  dragondive mysql -P9306 -h0
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 6.2.12 dc5144d35@230822 (columnar 2.2.4 5aec342@230822) (secondary 2.2.4 5aec342@230822) git branch manticore-6.2.12...origin/manticore-6.2.12

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>
3

Well, there’s at least this check for the custom config https://github.com/manticoresoftware/docker/blob/6.2.12/docker-entrypoint.sh#L177 . Not sure how relevant it is to the issue though.

4

Hello @Sergey

Thanks for your response and in trying to recreate the problem.

The manticore.conf file I used is the same as the “default” config file that the container uses. That’s not the one I want to use eventually, I chose to mount that as an intermediate step to identify the problem.

I’m not sure if this makes any difference, but I use the docker engine in WSL2 (Ubuntu 22.04 on Windows 11 host).

I will try out those steps you mentioned and see if that solves the problem. Unfortunately it will take me a while to get back to this because I need to deal with something else for the next few days.

5

I did this again today, and now it works.

Unfortunately, I’m not sure what was causing the previous problem. Other than doing a complete system restart, I did not change anything since last time. If I come across this problem again, I will try to analyze it more and provide more information.