How do I run the indexers of the manticore servers from any web server? The web servers will have a kind of script that will manage the building of indexes. I’m kind of lost how to run the indexers without programming some kind of SSH. Is there any other way than creating a SSH script?
Still no option to call indexer via TCP? Old-style Sphinx-ish indexes inherently depend on separate Cron process updating them. This is especially problematic in containerized environments like Docker/Kubernetes. Running both Cron and searchd in one container is problematic because running two processes is problematic by itself. Normally I could run Cron outside in another container but without remote indexer it’s not possible. And running sshd instead of Cron won’t help either.
Also it makes writing automated integration tests a trouble because integration test needs to find a way to trigger reindexing after arrange-part before asserting that the search is working.
I was considering switching to Manticore’s new “RT” indexes but that’s really a completely different design. I would have to rewrite all infra code. Now I am wondering if I should switch to RT index or code a good Cron+searchd daemon runner.
I feel rather unintelligent and obtrusive as I am talking with myself. But maybe other Googlers could find it useful, so…
I was dwelling for several days upon xmlpipe/csvpipe, sophisticated main+delta scheme, trying to generate ids, solving merging duplicates issues and dealing with Cron setup. And then I just migrated to Manticore-specific RT indexes using only HTTP, very small config file and ready-made Docker image. Took me a couple of hours maybe overall. I was coming from Sphinx and trying old Sphinx approach. But this new approach seems quite a lot easier.
root@7656cbf91f92:/var/lib/manticore# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
mantico+ 1 0.0 0.0 768404 9224 ? Ssl 03:34 0:00 searchd --nodetach
root 37 0.0 0.0 40480 8004 pts/0 Ss+ 03:34 0:00 mysql
root 42 0.0 0.0 20256 3852 pts/1 Ss 03:35 0:00 bash
root 644 0.0 0.0 30104 2844 ? Ss 03:38 0:00 cron
root 648 0.0 0.0 36152 3208 pts/1 R+ 03:39 0:00 ps aux
root@7656cbf91f92:/var/lib/manticore# tail -f /var/log/cron.log
So you can build your own image FROM manticoresearch/manticore and add cron to it.
And then I just migrated to Manticore-specific RT indexes using only HTTP, very small config file and ready-made Docker image. Took me a couple of hours maybe overall
That’s exactly why we’ve been always focusing more on the RT part of the project since the beginning and added the HTTP JSON interface. Plain indexation is fine, but it made more sense back in the days when Sphinx was more of an extension to mysql/postgres/whatever.