limit query error | difference between sphinx and manticore


I am migrating production environment from Sphinx to Manticore search engine. Over mysql protocol and SphinxSE when I run SQL through MySQL SphinxSE for Sphinx table with limit 10000 I successfully get the result set. However, when I run same query through MySQL SphinxSE for Manticore table I get error:
#1430 - There was a problem processing the query on the foreign data source. Data source error: bad searchd response length (length=23911790)

When I connect to Manticore server and run the same query adopted for Manticore MySQL CLI it works with the same 10000 limit.

These testing steps suggest there is a difference or some Manticore limitation comparing it with sphinx when I search using MySQL protocol over SphinxSE. I haven’t tested it yet if there would be any difference using HTTP protocol and Manticore PHP library.

For instance, this query:
SELECT * FROM manticore_ads AS mnt WHERE sph.query = '(@(name,categoryName,description) ps 5);mode=extended2;fieldweights=name,8,description,3,categoryName,1,adIdTag,6,goldTag,9,verifiedTag,5,premiumTag,0;sort=extended:WEIGHT() DESC,posted DESC;offset=0;limit=10000;maxmatches=19999;ranker=expr:(sum((4lcs+10wlccs+(min_hit_pos==1)+word_count…

From the selector (@(name,categoryName,description) ps 5) when I remove field description it sometimes help to get the results, depends on the keyword.

When I do the same query from MySQL SphinxSE using sphinx table, both of the queries work. Also mystery thing is I don’t even have 10000 rows with the given keyword (also now with other keywords). For a keyword from the example I have 7713 rows in both, Sphinx and Manticore database. Also, there is no unique limit where I’m sure all queries will work.

I implemented logging in PHP for every query that doesn’t work and then from database I’m taking it and testing from phpMyAdmin. Some queries are crashing at 4000 limit.

For my platform, sphinx was returning 10000 rows. In this case I’m forced to do some custom unnecessary logic to check every time if query is successful. If it is not, I need to concatenate result set for multiple executed queries until I get 10000 rows and I need to do decrement limit every time I detect query was not successful (from 10000 possibly I could have 6 iterations to decrease it to 4000) and then I need to do dynamically calculation to concat 2x4000 rows + 2000 rows.

I quess I’m missing something on DevOps side or in manticore configuration/SphinxSE. Anyone faces such an issues migrating from sphinx to manticore?

Any help would be appreciated.

What’s the difference between “Over mysql protocol and SphinxSE” and “through MySQL SphinxSE” ?