Manticore for LLM

General Discussions - English
1

I wonder if anyone is looking into using the Manticore index for LLM and how these would compare/differ.

We are working on a LLM model for our data and it seems we have to create a new DB/index for all our documents basically duplicating what Manticore already does. LLM are based on words distance and presence so their logic cannot be too far off a fulltext engine.

Wonder if there is a way to integrate the 2 and maybe build a LLM index using Manticore … :slight_smile:

Any ideas ?

2

I think Manticore and LLM can work together in the form of a 2-tier search:

  1. first off, search in Manticore with the query running in quorum/proximity mode to find just potential results
  2. pass the candidate documents to an LLM along with the initial query to prepare a final answer

It should be possible while indexing to pass all full-text data into an LLM library to build an LLM index along with a Manticore full-text index and then do the above.

3

Hi Sergey,

this is exactly what we are doing so all good, my query (sorry I did not made it too clear) was more in terms of indices compatibility, to me it seems like the Fulltext index and the LLM index are calculating a lot of the same stuff (i.e. words proximity is one of them) so I wondered if there was any way to build just 1 index for both uses or convert one index into the other. LLMs use a moving window approach which is one of the main areas where the 2 differ.

I was just wondering I anyone had looked into this and had any ideas to share.
As we are exploring I will update this thread with results, ideas suggestions as we do along.

Thanks
Roberto