Docker Manticore setup

I am trying to setup Manticore search within a docker container and I am running into an issues. I am able to build manticore, web node and a mysql container with no problem. Once the container is built it searches for the MySQL database. After that has been completed I am running into this issue

2023-11-28 16:04:38 using config file '/etc/manticoresearch/manticore.conf'...
2023-11-28 16:04:38 indexing table 'cw_issue_pages'...
2023-11-28 16:04:38 collected 0 docs, 0.0 MB
2023-11-28 16:04:38 total 0 docs, 0 bytes
2023-11-28 16:04:38 total 0.238 sec, 0 bytes/sec, 0.00 docs/sec
2023-11-28 16:04:38 total 0 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
2023-11-28 16:04:38 total 10 writes, 0.006 sec, 0.1 kb/call avg, 0.6 msec/call avg
2023-11-28 16:04:38 WARNING: failed to open pid_file '/var/run/manticore/'.
2023-11-28 16:04:38 WARNING: tables NOT rotated.

This is my docker-composer.yml file

    image: "mysql"
    command: --default-authentication-plugin=mysql_native_password --sql_mode="STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION"    
    container_name: "mysql-container"
      - 3306:3306
      - ./data/mysql/schema.sql:/docker-entrypoint-initdb.d/schema.sql
      - MYSQL_ROOT_PASSWORD=password 
      - MYSQL_DATABASE=database
      - MYSQL_USER=user
      - MYSQL_PASSWORD=password
    container_name: manticore
    image: manticoresearch/manticore
    command: "gosu manticore indexer --all --rotate"
    restart: always
      - db
      - 9306:9306
      - 9308:9308
      nproc: 65535
         soft: 65535
         hard: 65535
        soft: -1
        hard: -1
      - ./data/manticore/lib:/var/lib/manticore
      - ./data/manticore/indexes:/etc/manticore/data
      - ./data/manticore/log:/var/log/manticore
      - ./data/manticore/local_cw_manticore.conf:/etc/manticoresearch/manticore.conf 
      - ./data/manticore/stopword/magazine-stopwords.txt:/etc/manticoresearch/magazine-stopwords.txt

I do have this at the top of local_cw_manticore.conf

searchd {
    listen =
    listen =
    log = /var/log/manticore/searchd.log
    query_log = /var/log/manticore/query.log
    pid_file = /var/run/manticore/

If someone has run into this issue or has some insight into what is happening that would be amazing. We have been running into this issue for the past week now and can’t seem to find a solution to it

It seems like you container is just set to run ‘indexer’

command: "gosu manticore indexer --all --rotate"

Where really the command should be running ‘searchd’ - that is the real daemon, that answers queries etc.

When you call ‘indexer --rotate’ it is expecting that searchd is running (and its searchd that creates the pid file, that can’t be found)

Once you’ve got a container running searchd, then figure how to run indexer. Normally would run it with ‘cron’. You could perhaps run cron in the same container as searchd, but it would normally be cleaner to run it in a sidecar container.

The indexer and cron containers, would have to have a shared data volumn, and also be able to communicate (via pid file and signals)

We actully use GitHub - aptible/supercronic: Cron for containers to as the cron daemon in our containerized manticore setup.
So technically would be one container running supercronic (which periodically runs indexer), and the seperate container that runs searchd.

indexer has to be run in a running container, please read GitHub - manticoresoftware/docker: Official docker for Manticore Search and use docker exec

Please also read about the new features related with indexing in docker - GitHub - manticoresoftware/docker: Official docker for Manticore Search that allow building a plain table on container startup and using an internal cron service.