Query timeout for local distributed indexes


#1

As per the documentation I understood that agent_query_timeout value works for distributed index having remote agents, but does agent_query_timeout work for local distributed indexes as well ?

I have few hundreds of millions of rows , the queries I fire are taking too long time to return result,
Is there a way to have query_timeout for local distributed indexes so that i wouldn’t have to wait for all threads to complete the query and gather result


#2

Hi

agent_query_timeout works only for agents. For locals (actually it will work for remote queries too) use per query OPTION max_query_time instead (https://docs.manticoresearch.com/latest/html/sphinxql_reference/select_syntax.html?highlight=max_query_time#option)


#3

Thanks @Sergey, After reading the documentation I found that using max_query_time may result in unstable non repeatable results, instead I could use max_predicted_time which gives repeatable results,

The documentation doesn’t properly mention how predicted_time_costs value can be configured

In the example section

predicted_time_costs = doc=128, hit=96, skip=4096, match=128

like doc=128 , now what does that mean.
How do I ensure that those values will give best performance for my index

What do each of the value present here mean ?
How does this affect performance?


#4

doc/hit/skip/match means how expensive (in nanoseconds) each of the factors is (fetching a doc, fetching a hit, making a skip and processing a match), but:

  • it depends on the hardware and it’s complicated to find the values close to true. Some statistics techniques (e.g. linear regression) may be used to find the optimal values if you have big enough training data set (query log), but our old experiments show that the quality is still not sufficient and the predicted time is often very different from the real time.
  • there are other contributing factors not included in the formula, that’s probably why the regression didn’t work out

So I would say that unless it’s really important for you and you are ready for doing some maths just use the defaults and check manually if you are satisfied by the result.