Concurrent FLUSH and OPTIMIZE

General Discussions - English
1

Hi, I have 3 related questions, I hope it’s not too much to ask. They are all about:

  • Debian,
  • Manticore 6.2.12, with columnar
  • Buddy, plugins: core: empty-string, backup, emulate-elastic, insert, select, show, cli-table, plugin, test, insert-mva
  • RT index created with CREATE TABLE
  • binlog is off (in .conf)
  • .conf: access_plain_attrs=mlock , access_blob_attrs=mlock , access_doclists=mlock , access_hitlists=mlock
  • only MySQL interface is enabled

#1: Disable automatic FLUSH
I know about rt_flush_period and rt_mem_limit, but i couldn’t find the setting to disable both and let me control manually when to FLUSH (so only when i run FLUSH TABLE).

#2: Concurrent FLUSH + FLUSH
What happens when manticore is doing an automatic FLUSH (rt_flush_period, rt_mem_limit) and while its running, I keep doing write queries, and (while it’s still running) I also manually FLUSH TABLE? Will it be ok?

#3: Concurrent FLUSH + OPTIMIZE
What happens when manticore is doing an automatic FLUSH (rt_flush_period, rt_mem_limit) and while its running, I keep doing write queries, and (while it’s still running) I also manually OPTIMIZE TABLE rt_idx OPTION sync=1; ? Will it be ok?

Bonus Question: FLUSH and UPDATE/DELETE queries
I read in the docs that a FLUSH creates a new disk chunk. That gives me the feeling that it’s about INSERT/REPLACE. How about UPDATE/DELETE? Maybe I understood something wrong, pointing me to a correct docs would be appreciated.

Thanks in advance.

2

There are 2 different kinds of flushes (achieved by ‘flush table’ and ‘flush ramchunk’). First one doesn’t produce any disk chunk, but just saves accumulated data into .ram file. And that is ruled by rt_flush_period. After data is saved, binlog with this data can be released. Second ‘flush ramchunk’ produces disk-chunk and abandon .ram file. As result index has no ram chunk at all.
you can use freeze <table> to disable any disk savings. This command will once perform flush, and then set internal flag to prohibit further modifications. Then you can insert/replace - up to rt-mem-limit; this data will be stored only in binlog; no .ram, no disk chunks will be automatically saved. When rt-mem-limit reached, no more modifications possible.

Concurrent flushing is not possible. If one flush request came right during another - it will be scheduled next right after current flush finishes.

OPtimizing doesn’t affect flushing. That is separate process doing with existing disk chunks. Writing new documents will produce new ram/disk chunks. Deleting from optimizing chunks will be kept and applied to resulting chunk also. Same for updates.

Flush is nothing about update/delete. That is - flush table will just save current state. Any modifying query will wait until saving completed, and then apply.

3

Thank you for the explanation