not all columns found in some lines when importing CSV

General Discussions - English
1

Hi,

I think there is a bug in the indexer.
When I’m building my indexes from CSV files (just "cat " in source), I often get this error:
ERROR: index ‘de_trigrams_chunk_x’: source ‘de_trigrams_chunk_x’: not all columns found (found=1, total=3, line=607654).

the example line is
607653,“eine bestehende Reklamation”,41

when I add another “space” in the text, it could be imported.
I have this problem around 3-5 times per file.
Any idea?

Alex

2

it could be better to create a ticket at Github there to put reproducible example of this issue (config file along with csv file)

3

Can’t reproduce

snikolaev@dev:~$ cat csv_issue.conf
searchd {
    listen = 9315:mysql41
    log = searchd.log
    pid_file = searchd.pid
    binlog_path =
}

source src {
    type = csvpipe
    csvpipe_command = echo "607653,\"eine bestehende Reklamation\",41"
    csvpipe_field = f
    csvpipe_attr_uint = a
}

index idx {
    type = plain
    source = src
    path = idx
    stored_fields = f
}

snikolaev@dev:~$ indexer -c csv_issue.conf --all --rotate
Manticore 4.2.0 15e927b@211223 release (columnar 1.11.4 327b3d4@211223)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
Copyright (c) 2017-2021, Manticore Software LTD (https://manticoresearch.com)

using config file 'csv_issue.conf'...
indexing index 'idx'...
collected 1 docs, 0.0 MB
creating lookup: 0.0 Kdocs, 100.0% done
sorted 0.0 Mhits, 100.0% done
total 1 docs, 27 bytes
total 0.049 sec, 549 bytes/sec, 20.35 docs/sec
total 3 reads, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
total 15 writes, 0.000 sec, 0.0 kb/call avg, 0.0 msec/call avg
rotating indices: successfully sent SIGHUP to searchd (pid=2376702).

mysql> select * from idx;
+--------+------+-----------------------------+
| id     | a    | f                           |
+--------+------+-----------------------------+
| 607653 |   41 | eine bestehende Reklamation |
+--------+------+-----------------------------+
1 row in set (0.00 sec)