How it could look in terms of SQL syntax and general functionality:
CALL PQ extension
Just use values from an existing index rather than processing an array of given documents:
CALL PQ('pq', (SELECT <single_field> FROM any_index WHERE conditions), options);
It would be mandatory that the sub-select returns exactly one text/string column. Internally we would also get each document’s id to display it if
1 as docs option is enabled.
MATERIALIZED VIEW + PQ rules
In some other databases there’s a what’s called materialized view, i.e. you have a table and have a query and say “I want this query to be applied to this table continuously (or on demand) and the result should be put to another table”.
It looks very similar to the idea of this topic. But in our case we already have the concept of PQ index, i.e. not one query, but multiple, so it might look like this:
CREATE MATERIALIZED VIEW index_pq_stream AS
SELECT f text, s string, i int FROM source_index // leave only these fields in the MV
WHERE PQ(pq_index) // PQ(idx) is like MATCH(), but means "use all rules from this PQ index"
AND MATCH(...) AND another_attribute > 123 // defines additional filtering besides the PQ filtering, may be also useful for sharding to parallelize processing etc.
This functionality would be great to have, but given we are quite limited in development resources we would need to make sure that a significant number of users would benefit from this or there’s a company willing to sponsor the development.
If you think the above makes sense and have in mind some use cases for that with more details I’d appreciate if you elaborated more on them.