Managing Memory Usage for Float Vectors in Manticore + Query Performance for Bulk Vector Fetches

benwills · July 9, 2025, 7:45am

I’m exploring options to better manage float_vector memory usage, especially after the recent release with vector search improvements. I have a couple specific questions related to memory handling and performance.

–

Memory Management:

Does the per-table rt_mem_limit setting apply to the new float vector columns? Or are vectors handled/stored in memory outside of that constraint?

If not governed by rt_mem_limit, is there another way to control or restrict the memory usage for vector data? Any insight into how vector storage is partitioned between memory and disk would help with planning.

–

Vector Retrieval at Scale (K-Means Prep):

I’m evaluating a new feature where I need to retrieve all vectors for a filtered set of document IDs, based on time-based filters like “added in the last 48 hours.”

Assuming the vectors are on NVMe storage and I pass in a known list of document IDs (e.g., via id IN (...)), what kind of query time should I roughly expect when retrieving 10,000–50,000 vectors? The use case is to then take those and calculate k-means clustering on that set of vectors outside of Manticore.

Even just a rough range (eg: number of seconds) would help determine if this use case is feasible in Manticore or if I should consider a different form of storage for these vectors. I understand this is unlikely to be a sub-second query, and that’s okay as this operation will be run on a schedule and cached vs user-triggered.

–

Thanks.

Ben

Sergey · July 15, 2025, 10:10am

Hello @benwills

Sorry for the late reply.

Does the per-table rt_mem_limit setting apply to the new float vector columns?

Yes, it does. While a vector is in a RAM chunk, it’s stored in its raw form (without quantization), so it’s affected by rt_mem_limit. Once the RAM chunk is flushed to disk, the vector gets quantized and added to the HNSW index, which turns it into completely different data structures. At that point, quantization and dimensionality become the key factors for RAM optimization.

what kind of query time should I roughly expect when retrieving 10,000–50,000 vectors?

I believe it’s the same when retrieving 10–50K strings/mva with similar size per field.

Even just a rough range (eg: number of seconds) would help determine if this use case is feasible in Manticore or if I should consider a different form of storage for these vectors

It’s easier to test it using manticore-load or another tool. For example, this will create a dummy table with 100,000 documents, each containing a 512-dimensional float_vector.

manticore-load \
--drop \
--batch-size=1000 \
--threads=5 \
--total=100000 \
--init="create table test ( title text, image_vector float_vector knn_type='hnsw' knn_dims='512' hnsw_similarity='l2' )" \
--load="insert into test values ( <increment>, '<text/1/2>', (<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>,<float/0/1>) )"