Still getting to grips with the Manticore 5 (getting ready to move some sites from 3.6)
… curious about the new ‘implicit cutoff’ feature. When does it apply, and in particular when need to make sure specify cutoff=0.
… I know can always inspect SHOW META afterwards and find out if a cutoff was applied.
Not so much worried about the fact that total_found may not be accurate, more its effect on the actual query results. Normally with cutoff is generally only ‘safe’ if ORDER BY id ASC.
For example, would an implicit cutoff ever be applied when sorting by weight ? - because if it is, it could affect the results - ie could get just the first found, not necessarily the top ranking.
Blindly applying cutoff=0 always seems to defeat the work. But we do need it if want total_found, but do we also need to do it when worried about the quality of results?
One point noticed, is the implicit cutoff, is changing the results of a query
like select * from user where images > 1000 limit 5
… technically, as there is no order-by, ordering is ‘undefined’, but you used to be able to somewhat rely on it being implicitly ORDER BY id ASC
So the changed results are not ‘wrong’ as such, they still match the query, just different to before.
RT>select * from user where images > 1000 limit 5; show meta;
+------+--------+---------+----------------------------+----------------+
| id | images | last | realname | nickname |
+------+--------+---------+----------------------------+----------------+
| 1 | 5847 | 7247486 | Gary Rogers | Inukshuk |
| 8 | 1335 | 7259295 | Martyn Pattison | MartyBartfast |
| 11 | 1506 | 7235790 | Andy Stephenson | Happy Humphrey |
| 22 | 1071 | 2923444 | Brendan and Ruth McCartney | birders |
| 40 | 2404 | 692870 | Pam Brophy | Pam Brophy |
+------+--------+---------+----------------------------+----------------+
5 rows in set (0.001 sec)
+----------------+-------+
| Variable_name | Value |
+----------------+-------+
| total | 5 |
| total_found | 5 |
| total_relation | gte |
| time | 0.000 |
+----------------+-------+
4 rows in set (0.000 sec)
RT>select * from user where images > 1000 limit 5 option cutoff=0; show meta;
+-------+--------+---------+------------------+---------------+
| id | images | last | realname | nickname |
+-------+--------+---------+------------------+---------------+
| 1 | 5847 | 7247486 | Gary Rogers | Inukshuk |
| 5295 | 1110 | 6204695 | Ian Rainey | IanR |
| 25920 | 1317 | 5304387 | louise price | Loup |
| 5299 | 3306 | 7160325 | Danny P Robinson | Danny R |
| 8 | 1335 | 7259295 | Martyn Pattison | MartyBartfast |
+-------+--------+---------+------------------+---------------+
5 rows in set (0.001 sec)
+----------------+-------+
| Variable_name | Value |
+----------------+-------+
| total | 5 |
| total_found | 566 |
| total_relation | eq |
| time | 0.000 |
+----------------+-------+
4 rows in set (0.001 sec)
My main question is still could the implicit cutoff affect queries when relying on implicit weight ordering?
So far when using MATCH() always seem to get total_relation=eq, but not sure covering all the cases
where is no implicit id ordering as id could be regular columnar attribute and it could be better not to fetch every matched document from disk based storage to just make order without user direct intent