We defined an RT table in plain mode with auto-embeddings as:
table rt_products {
type = rt
path = /var/lib/manticore/rt
rt_field = name
rt_field = spc
rt_attr_float_vector = emb_vector
knn = {“attrs”:[{“name”:“emb_vector”,“type”:“hnsw”,“hnsw_similarity”:“L2”,“hnsw_m”:16,“quantization”:“8bit”,“hnsw_ef_construction”:200,“model_name”:“sentence-transformers/all-MiniLM-L6-v2”,“from”:“name,spc”}]}
}
Now we are inserting 10 records into the table, the spc field is a text field and contains a document (converted to text), this is an example of the insert statement:
INSERT INTO rt_products (name, spc) VALUES (‘0.18 % W/V SODIUM CHLORIDE AND 4 % W/V GLUCOSE IN
T’, ’ \n\nIRISH MEDICINES BOARD ACTS 1995 AND 2006\n\nMEDICINAL PRODUCTS(CONT
ROL OF PLACING ON THE MARKET)REGULATIONS,2007\n\n(S.I. NO.540 OF 2007)\n\nPA0179/003/006A\n\nCase No: 2041821\n\nThe Irish Medicines Board in exercise
of the powers conferred on it by the above mentioned Regulations hereby grants to\n\nB. BRAUN MEDICAL LIMITED\n\n3 NAAS ROAD INDUSTRIAL PARK, DUBLIN
12, IRELAND\n\nan authorisation, subject to the provisions of the said Regulations, in respect of the product\n\n0.18 % W/V SODIUM CHLORIDE AND 4 % W/
V GLUCOSE INTRAVENOUS INFUSION BP SOLUTION FOR INFUSION, 500 ML AND 1000 ML\n\nThe particulars of which are set out in Part I and Part II of the attac
hed Schedule. The authorisation is also subject to the general conditions as\n\nmay be specified in the said Regulations as listed on the reverse of t
his document.\n\nThis authorisation, unless previously revoked, shall continue in force from 01/04/2008.\n\nSigned on behalf of the Irish Medicines Bo
ard this\n\n________________\n\nA person authorised in that behalf by the said Board.\n\nIRISH MEDICINES BOARD\n\n____________________________________
\n\n_Date Printed 08/04/2008**\n\n_CRN 2041821\n\n_page number: 1
\n\nPART II\n\nSUMMARY OF PRODUCT CHARACTERISTICS\n\n1 NAME OF THE MEDICINAL PRODUCT\n\n0.18 % w/v Sodium Chloride and 4 % w/v Glucose Intravenous Inf
usion BP, Solution for Infusion.\n\n2 QUALITATIVE AND QUANTITATIVE COMPOSITION\n\nFor a full list of excipients, see section 6.1.\n\n3 PHARMACEUTICAL
FORM\n\nSolution for infusion\n\nA clear, colourless, sterile and non-pyrogenic aqueous solution.\n\n4 CLINICAL PARTICULARS\n\n4.1 THERAPEUTIC INDICAT
IONS\n\nFor use in prophylactic and replacement therapy requiring the use of glucose and sodium chloride.\n\n4.2 POSOLOGY AND METHOD OF ADMINISTRATION
\n\n4.2.1 DOSAGE\n\nIn accordance with fluid, electrolyte and energy requirements:\n\nNot more than 40 ml/kg body weight per day, corresponding to 1.7
g glucose/kg body weight per day.\n\n_Infusion and drop rate:\n\nNot more than 5 mg/kg body weight per hour, corresponding to 0.2 g glucose/kg body
weight per hour, or not more\n\nthan 1.7 drops/kg weight per minute.\n\nThe solution contains per\n\n500 ml\n\n1000 ml\n\nSodium chloride\n\n0.90 g\n
n1.80 g\n\nAnhydrous Glucose\n\n20.00 g\n\n40.00 g\n\nas Glucose Monohydrate\n\nEach litre provides:\n\nElectrolytes:\n\nSodium\n\n31\n\nmmol/l\n\nChl
oride\n\n31\n\nmmol/l\n\nCalorific value:\n\n668 kJ/1 = 160 kcal/1\n\nTheoretical osmolarity:\n\n284\n\nmOsm/l\n\nTitration acidity (to pH 7.4):\n\n<
0.5 mmol/l\n\npH:\n\n3.5 – 5.5\n\nIRISH MEDICINES BOARD\n\n**__
\n\n_Date Printed 08/04/2008**\n\n_CRN 2041821\n\n_page number: 2\n\n4.2.2 METHOD AND ROUTE OF ADMINISTRATION\n\nIntrave
nous infusion.\n\n4.3 CONTRAINDICATIONS\n\n0.18 % w/v Sodium Chloride and 4 % w/v Glucose Intravenous Infusion BP, solution for Infusion should not be
used\n\nin cases of:\n\n- Hyperhydration states.\n\n- Hypotonic dehydration with hyponatraemia.\n\n- Hypokalaemia.\n\n- Impaired kidney function.\n\n
This container contains a significant volume of air. To avoid risk of air embolism, this product must not be\n\nadministered by pressure infusion.\n\n
4.4 SPECIAL WARNINGS AND PRECAUTIONS FOR USE\n\n0.18 % w/v Sodium Chloride and 4 % w/v Glucose Intravenous Infusion B.P. Solution for Infusion should
only be\n\nadministered with caution in cases of insulin refractory hyperglycaemia necessitating the administration of more than 6\n\nunits of insulin
/hour.\n\nIt is necessary to monitor the serum electrolytes and water balance.\n\nIn post-operative and post-traumatic conditions and in conditions of
impaired glucose tolerance: only administer with\n\nmonitoring of blood glucose level.\n\nSpecial attention should be paid to regular monitoring of t
he serum potassium concentration.\n\n4.5 INTERACTION WITH OTHER MEDICINAL PRODUCTS AND OTHER FORMS OF INTERACTION\n\nThe solution should not be admini
stered through the same infusion equipment simultaneously, before or after an\n\nadministration of blood because of the possibility of pseudo-agglutin
ation.\n\n4.6 PREGNANCY AND LACTATION\n\n0.18 % w/v Sodium Chloride and 4 % w/v Glucose Intravenous Infusion BP Solution for Infusion can be used as\n
\nindicated.\n\n4.7 EFFECTS ON ABILITY TO DRIVE AND USE MACHINES\n\nNot applicable.\n\n4.8 UNDESIRABLE EFFECTS\n\nNone to be expected if used in accordance with the instructions given.\n\nIRISH MEDICINES BOARD\n\n**__\n\n_Date Printed 08/04/2008_\n\n_CRN 2041821_\n\n_page number: 3_\n\n4.9 OVERDOSE\n\n4.9.1 SYMPTOMS\n\nOverdose may result in overhydration with increased skin tension, venous congestion, oedema - possibly also lung or\n\nbrain oedema, hypokalaemia and acid-base imbalances and hyperglycaemia.\n\nClinical symptoms of water intoxication may also occur, such as nausea, vomiting, spasms.\n\n4.9.2 EMERGENCY TREATMENT; ANTIDOTES\n\nImmediate cessation of infusion, administration of diuretics with continuous monitoring of serum electrolytes,\n\ncorrection of electrolyte and acid-base imbalances, administration of insulin if necessary.\n\n5 PHARMACOLOGICAL PROPERTIES\n\n5.1 PHARMACODYNAMIC PROPERTIES\n\nThe solution contains equimolar proportions of sodium and chloride corresponding to one fifth the physiological\n\nconcentration in the plasma. In addition this solution also contains 4 % (w/v) of carbohydrate in the form of glucose.\n\nThe solution is mainly intended to supply free water in cases of dehydration.\n\n5.2 PHARMACOKINETIC PROPERTIES\n\nThe total sodium content of the body is ca. 80 mmol/kg of which ca. 97 % is extracellular and ca. 3 % intracellular.\n\nThe daily turnover is ca. 100-180 mmol (corresponding to 1.5-2.5 mmol/kg body weight).\n\nChloride is exchanged for hydrogen carbonate in the tubule system and is, thus, involved in the regulation of the acid\n\nbase balance.\n\nOn infusion glucose is first distributed in the intravascular space and then is taken up into the intracellular space.\n\nIn glycolysis glucose is metabolized to pyruvate or to lactate. Lactate can be partially re-introduced into the glucose\n\nmetabolism (Cori cycle). Under aerobic conditions pyruvate is completely oxidized to carbon dioxide and water. The\n\nfinal products of the complete oxidation of glucose are eliminated via the lungs (carbon dioxide) and the kidneys\n\n(water).\n\n5.3 PRECLINICAL SAFETY DATA\n\nThere are no pre-clinical data of relevance to the prescriber which are additional to those already stated in other\n\nsections of the SPC.\n\n6 PHARMACEUTICAL PARTICULARS\n\n6.1 LIST OF EXCIPIENTS\n\nWater for injections\n\n6.2 INCOMPATIBILITIES\n\nWhen mixing with other medicaments it should be remembered that 0.18 % w/v Sodium Chloride and 4 % w/v\n\nGlucose Intravenous Infusion BP Solution for Infusion has an acid pH which can cause precipitation in the mixture.\n\nIRISH MEDICINES BOARD\n\n__\n\n_Date Printed 08/04/2008_\n\n_CRN 2041821_\n\n_page number: 4_\n\n6.3 SHELF LIFE\n\nUnopened: 3 years\n\nOnce opened: Use immediately. Discard any unused contents.\n\n6.4 SPECIAL PRECAUTIONS FOR STORAGE\n\nDo not store above 25°C. Store in original container.\n\n6.5 NATURE AND CONTENTS OF CONTAINER\n\nContainers of low-density polyethylene with integral on-welded closure of the same material. The closure contains a\n\nrubber disc.\n\nContents: 500 ml, 1000 ml.\n\n6.6 SPECIAL PRECAUTIONS FOR DISPOSAL AND OTHER HANDLING\n\nSingle-dose container. Discard unused contents.\n\nOnly to be used if the solution is clear, free from visible solid particles and the container or its closure do not show\n\nvisible signs of damage.\n\n7 MARKETING AUTHORISATION HOLDER\n\nB. Braun Medical Limited\n\n3 Naas Road Industrial Park\n\nDublin 12\n\n8 MARKETING AUTHORISATION NUMBER\n\nPA 179/3/6\n\n9 DATE OF FIRST AUTHORISATION/RENEWAL OF THE AUTHORISATION\n\nDate of first authorisation: 01 April 1983\n\nDate of last renewal: 01 April 2008\n\n10 DATE OF REVISION OF THE TEXT\n\nApril 2008\n\nIRISH MEDICINES BOARD\n\n_____________________________________________________________________________________\n\n_Date Printed 08/04/2008_\n\n_CRN 2041821_\n\n_page number: 5_\n ');
The problem we have is the time it takes to insert records into the table, about 8 minutes for 10 records, we assume this is caused by the vector calculation (?)
Has anyone performed any benchmarking on this operation so that we can compare ?
Are we doing something wrong ?
As it stands building a table with 5M+ records like the above would not be doable, maybe this is the wrong approach.