I am updating my Manticore from 2.4.1 to 3.1.2. and i have a problems with the using of new features of killlists.
I have three indexes: delta, main and archive. I am using tsv to update them and killists to keep data updated. For example, killist of delta contained all data in delta and removed data and it affected main and archive in the distributed index search. I have a frequent update of a data and it’s very important for me.
Now if i rotate delta and then main, all data from delta will be permanetly deleted from main, until i rotate delta. And main contains no data and if i rotate delta with max id from main i will lose my data with previous strategy.
could you also try to use RT index instead of delta - main schema?
You could attach main plain index to RT index and reset all collected data then push new \ delta data into that RT index on top of existed data.
Then in case many disk chunks collected issue OPTIMIZE command for that RT index or reindex main from scratch with indexer and attach it again into RT index
And one more question. Does killlist_target work in the rt index to suppress removed data in the big archive index? And what do you mean on attach main to RT?
Does killlist_target work in the rt index to suppress removed data
killlist_target got applied to any kind of indexes either to plain or RT
However for RT index you could remove document from RT index itself via DELETE statement without managing indirect killists.
what do you mean on attach main to RT?
You could attach plain index to RT index with ATTACH statement that allows you to faster indexing your data into plain index with indexer (in case you has hug amount of data and want to index it fastest possible way) then rotate that plain index into daemon then issue statement
This way you might reindex your data from scratch in case it convenient for you then ATTACH that reindex data to existed RT index. And after that apply all new data not to the delta index (as you do now) but directly to that RT index.
in the RT index configuration, where archive_idx is usual plain index, how will it work? Will killlist data be updated if i insert or delete value in rt?