Sql_attr_string compressed


#1

Hello together

maybe not possible but … save fields with sql_attr_string attribute compressed in memory.
This would save a lot of memory and uncompress can be done by mySql…

regards Peter


#2

Hi Peter

Sorry for the delays. I’ve discussed this with dev team. In general we’d prefer implementing docstore instead (since it would take longer, but not significantly), so long texts could be stored on disk, not in memory. The disadvantage of compressing string attrs is that it will be impossible to sort/group by them any more. Also we tend to believe most store not very long texts in string attrs and therefore it may make the effect of the compression very insignificant or even negative.
Can you share more details on your situation and why you think in your case it would make sense?


#3

Hello sergey

last year a make a test to save around 30 Mio. websites (only the text) in a Sql_attr_string and it works perfect… but the memory usage was 190 GB, and of course the loadtime was a couple of minutes :wink: The benefit was two things: No need to run a mySql Server and direct snipping on the match. Now i have 40 Mio. websites…

So i need the string-attr only to store the text, not for searching and this works.
My website texts are in compressed format on mySql Server - and the idea is to load the text in compressed format and then by the match uncompress it for snipping.

btw. using QL
btw2. sorry about my english, my german is better :wink:

regards Peter


#4

Ouups, i made a misstake :wink:

I’m talking about sql_field_string, not the sql_attr_string …

regards Peter