Server size an data storage and index for 500M rows data

General Discussions - English
1

Hi,
It is possible using Manticore with this application context :

  • fast query response under 100 ms
  • 500 to 100 conccurents users
  • 500 Millions rows data for 400 GO

What server architecture do you advise me, indexing strategy ? its is possible to index the data in 1 hour max.
If one of you hav already faced this kind of challenge please ur feedbacks would be appreciated.

2

Hello @johna

It heavily depends on:

  • the hardware
  • the queries

Manticore can scale out, so the more nodes you have:

  • the more data you can index per second
  • the more concurrent users you can serve
  • the lower response time you can provide

I believe Craigslist that use Manticore have more than 500M rows and 1000 concurrent users and it works fine for them.

3

Thanks,
for example if i use 4 servers like this:
CPU: Intel Xeon Gold 6312U - 24 cœurs/48 threads - 2,4 GHz/3,6 GHz
Memory: From 128 GB DDR4 ECC to 1 TB DDR4 ECC
Storage: NVMe SSD, SATA HDD, SATA SSD
Public Bandwidth: From 1 Gbps to 5 Gbps
Private Bandwidth: Guaranteed from 2 Gbps to 6 Gbps

Its ok for my 500M rows and 100 - 500 concurrents users ?
Should i use parralelized index or create a cluster with one node for each box ?

One more time thanks for your help