Hi all, I ran OPTIMIZE INDEX on my rt-index(460 million records)
request processed quickly, then check SHOW INDEX STATUS:
the number of disk segments still shows 21
what am I doing wrong and how to monitor it correctly?
Check in searchd log (/var/log/manticore/searchd.log by default). If it got to merging it should read smth like:
[Tue Jun 14 05:28:11.501 2022] [247] rt: index t: optimized progressive chunk(s) 1 ( left 1 ) in 106ms 895us
The first number is how many chunks were merged (if you have 6 in total it will be 5). The second one - chunks left after merging.
Take into account:
- optimize doesn’t always merge down to a single chunk
- there’s auto optimize, which is ON by default
- the number of chunks to leave after merging is controlled by:
-
optimize_cutoffin sectionsearchdof config Manticore Search Manual: Server settings > Searchd -
optimize_cutoffindex’s setting - Manticore Search Manual: Creating an index > Local indexes > Plain and real-time index settings -
OPTIMIZE ... OPTION cutoff=Nwhen run manually, otherwise the number of chunks to leave equals tooptimize_cutoff, which is by default number of CPU cores * 2. I’m guessing you have more than 11 cores, hence no actual merging happened.