Advice moving from plain index to RT index

lybrary · June 5, 2026, 8:00pm

I have so far only used plain indexes. I have a decent understanding of how they work. In my case, the indexer pulls data in from a MySQL database, indexes it, and then it is searched via SQL statements from a PHP script. I now want to migrate to RT indexes because I want to eventually add semantic search via the hybrid_search functionality and have Manticore handle embeddings.

Is there a guide or how-to to start from a working config file for plain indexes, and add a RT version of these plain indexes? What needs to change in the config file? What needs to change in the PHP scripts in terms of keeping the index up-to-date and in terms of searching?

I gather there is an ATTACH SQL command that allows one to populate a RT table from a source definition in the config file. Then there is –print-rt but I don’t understand how they differ. I understand bits and pieces but am looking for a complete tutorial of how to migrate from plain to RT.

Sergey · June 9, 2026, 10:27am

Here’s the guide:

Migrating from a plain table to an RT table in Manticore Search

This is a practical migration guide for a setup where:

MySQL is the original data source.
indexer currently builds a plain table from a source block in manticore.conf.
The application searches Manticore through the MySQL protocol, for example from PHP/PDO.
You want to move to an RT table so you can update documents in real time and later use semantic / hybrid search features.

The examples below use the same architecture: MySQL remains the primary application database, and Manticore Search is queried over the MySQL protocol. The migration changes how Manticore stores and receives documents; it does not require rewriting ordinary search queries from scratch.

Manual references:

1. Start from the plain-table config

A typical plain setup has a MySQL source and a plain table:

source docs_src
{
    type = mysql
    sql_host = 127.0.0.1
    sql_port = 3306
    sql_user = test
    sql_pass =
    sql_db = test

    sql_query_pre = SET NAMES utf8mb4
    sql_query = \
        SELECT id, title, content, category_id, UNIX_TIMESTAMP(updated_at) AS updated_ts \
        FROM documents

    # Columns not declared as attributes become full-text fields.
    sql_attr_uint = category_id
    sql_attr_timestamp = updated_ts
}

table docs_plain
{
    type = plain
    source = docs_src
    path = /var/lib/manticore/docs_plain
}

In this example:

id is the document id.
title and content are full-text fields because they are selected by sql_query and are not declared as attributes.
category_id and updated_ts are attributes because they are declared with sql_attr_*.

If your existing config uses sql_field_string, remember that this creates both a full-text field and a stored string attribute. Mirror that in the RT table if the application expects the value to be returned as an attribute.

2. Add an RT table next to the plain table

This step assumes the same mode your plain table already uses: config-file mode, with source and table blocks in manticore.conf and no data_dir in the searchd section.

In that mode, add a new RT table with the same text fields and attributes:

table docs_rt
{
    type = rt
    path = /var/lib/manticore/docs_rt

    rt_field = title
    rt_field = content

    rt_attr_uint = category_id
    rt_attr_timestamp = updated_ts
}

Keep the plain table in the same config for the migration step. You can remove it later after the RT table is populated and the application has switched over.

Do not add data_dir to this config. Manticore has two different configuration styles:

Config-file mode: tables are declared in manticore.conf. This is the natural mode for migrating an existing plain table with ATTACH TABLE plain TO TABLE rt.
data_dir mode: tables are created with SQL/API, for example CREATE TABLE docs_rt (...). In this mode you cannot keep config-defined plain table declarations in the same searchd config.

So, if your current working setup is a plain table in manticore.conf, stay in config-file mode for the migration. Add the RT table block next to the plain table block, run ATTACH, then later remove the plain table block if you no longer need it.

The actual switch to data_dir/RT mode comes later, after the plain table has been converted to an RT table. That part uses IMPORT TABLE and is covered below.

3. Rebuild the plain table once

Before attaching, rebuild or rotate the plain table the same way you normally do:

indexer -c /etc/manticoresearch/manticore.conf --rotate docs_plain

At this point the daemon should have both tables available: the rebuilt plain table and the empty RT table.

4. Recommended initial migration: `ATTACH TABLE`

For the first bulk load, ATTACH TABLE is usually the simplest path.

ATTACH TABLE docs_plain TO TABLE docs_rt WITH TRUNCATE;

What it does:

Converts the existing plain table files into a disk chunk of the RT table.
Is normally fast because it renames/reuses files rather than re-inserting every document.
With WITH TRUNCATE, first empties the RT table, so the attached plain table becomes the only data in the RT table.
After a successful attach, the source plain table becomes unavailable until you rebuild it again.

A typical migration check looks like this:

SELECT COUNT(*) AS plain_before FROM docs_plain;

plain_before
4

SELECT COUNT(*) AS rt_before FROM docs_rt;

rt_before
0

ATTACH TABLE docs_plain TO TABLE docs_rt WITH TRUNCATE;

Query OK

SELECT COUNT(*) AS rt_after FROM docs_rt;

rt_after
4

SELECT id, category_id FROM docs_rt WHERE MATCH('Manticore') LIMIT 10;

id  category_id
1   10
2   10
4   30

SHOW TABLES;

Table    Type
docs_rt  rt

Notice that after ATTACH, the plain table disappeared from SHOW TABLES. This is expected: the plain table data became part of the RT table.

Important restrictions from the manual:

The target RT table must be empty or have the same settings as the source table.
If the target RT table is empty, fields, attributes, and text-processing settings are copied from the source table and take effect.
The source table must have phrase_boundary_step = 0 and stopword_step = 1.

5. Switch the daemon to RT mode with `IMPORT TABLE`

After section 4, the plain table has already been attached to docs_rt. The data is now in an RT table, but the daemon is still using the old config-file style.

To switch to RT mode with data_dir, import those attached RT table files into a data_dir daemon.

The sequence is:

Stop searchd.
Edit manticore.conf for RT mode:
- remove the source block;
- remove the table docs_plain block;
- remove the config-defined table docs_rt block;
- add data_dir to the searchd section.
Example:
```
searchd
{
    listen = 127.0.0.1:9306:mysql
    listen = 127.0.0.1:9308:http

    log = /var/log/manticore/searchd.log
    query_log = /var/log/manticore/query.log
    pid_file = /var/run/manticore/searchd.pid

    data_dir = /var/lib/manticore
}
```
Do not leave the old plain table declarations in this config. data_dir mode and config-defined plain tables do not mix.
Start searchd with the new RT-mode config.
Import the attached RT table by pointing IMPORT TABLE at the old RT table file prefix:
```
IMPORT TABLE docs_rt FROM '/old/plain-mode/path/docs_rt';
```
For example, if the old RT table was declared as:
```
path = /var/lib/manticore-old/docs_rt
```
then the import command is:
```
IMPORT TABLE docs_rt FROM '/var/lib/manticore-old/docs_rt';
```
Keep the old table files in place until the import succeeds. IMPORT TABLE copies them into data_dir.

Check that the table is available:

SHOW TABLES;

Table    Type
docs_rt  rt

SELECT COUNT(*) FROM docs_rt;

count(*)
4

SELECT id, category_id FROM docs_rt WHERE MATCH('Manticore') LIMIT 10;

id  category_id
1   10
2   10
4   30

At this point the table is managed by the RT-mode daemon. New documents and changes should go through SQL/API writes to docs_rt, not through indexer.

IMPORT TABLE is for RT and percolate tables. A plain table must be converted first, which is exactly what ATTACH TABLE docs_plain TO TABLE docs_rt does.

6. What changes in PHP search code?

For ordinary full-text search, only the table name changes:

SELECT id, category_id
FROM docs_rt
WHERE MATCH('manticore')
LIMIT 20;

The connection method is the same if the RT-mode daemon listens on the same MySQL port. Existing PHP/PDO code can keep using prepared statements and result handling as before; the real change is on the write side, covered next.

7. What changes for keeping the table up to date?

This is the main operational change.

With a plain table, MySQL is the source of truth and Manticore is updated by running indexer again.

With an RT table, Manticore is updated through SQL writes:

INSERT / REPLACE for new or changed documents.
UPDATE for attribute-only changes.
DELETE for deleted documents.

Manticore does not automatically watch MySQL. If you insert a row into MySQL, the RT table will not change unless your application or a background job also writes to Manticore.

For example, inserting a new row into MySQL changes only MySQL. The RT table will still have the old document set until the application or a worker sends the corresponding write to Manticore.

Use REPLACE when adding a new document or replacing full-text fields:

REPLACE INTO docs_rt (id, title, content, category_id, updated_ts)
VALUES (
  5,
  'New semantic search document',
  'This row was added to MySQL after the RT attach.',
  40,
  1767621600
);

Recommended application pattern

When your application creates or changes a document in MySQL, also write the corresponding document to Manticore:

$replace = $pdoManticore->prepare("
    REPLACE INTO docs_rt (id, title, content, category_id, updated_ts)
    VALUES (?, ?, ?, ?, ?)
");

$replace->execute([
    $id,
    $title,
    $content,
    $categoryId,
    $updatedTimestamp,
]);

For an attribute-only change, UPDATE is fine:

UPDATE docs_rt SET category_id = 41 WHERE id = 5;

For a full-text field change, prefer REPLACE, because text fields are part of the indexed document:

REPLACE INTO docs_rt (id, title, content, category_id, updated_ts)
VALUES (5, 'Updated title', 'Updated body text', 42, 1767625200);

For a delete:

DELETE FROM docs_rt WHERE id = 5;

Reliability note

In production, avoid treating the Manticore write as an optional side effect. Common approaches are:

Write to MySQL, then write to Manticore; if the Manticore write fails, put the document id into a retry queue.
Use an outbox table in MySQL and have a worker apply pending changes to Manticore.
Periodically run a reconciliation job that compares recently updated MySQL ids with Manticore.

This is especially useful during migration, when you may want a safe fallback path.

8. Suggested migration checklist

Add a matching RT table block next to the plain table.
Rebuild the plain table, then run ATTACH TABLE docs_plain TO TABLE docs_rt WITH TRUNCATE.
Stop searchd, switch the config to data_dir mode, and start it again.
Run IMPORT TABLE docs_rt FROM '/old/plain-mode/path/docs_rt'.
Check counts and a few representative searches.
Point PHP searches to docs_rt.
Add the RT write path: REPLACE, UPDATE, and DELETE.
Keep retry/reconciliation in place until MySQL and Manticore stay in sync.
Remove the old plain-table indexer job only after the RT path is verified.