Thousands of warnings send() failed: 32: Broken pipe

On Saturday our server recorded thousands of these entries in the searchd.log causing the disk to fill up.
Manticore seems to work normally but we don’t know what caused these warnings.


[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: last message repeated 100 times
[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: send() failed: 32: Broken pipe
[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: last message repeated 100 times
[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: send() failed: 32: Broken pipe
[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: last message repeated 100 times
[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: send() failed: 32: Broken pipe
[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: last message repeated 100 times
[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: send() failed: 32: Broken pipe
[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: last message repeated 100 times
[Sat Feb 25 11:41:06.009 2023] [1456978] WARNING: send() failed: 32: Broken pipe
[Sat Feb 25 11:41:06.010 2023] [1456978] WARNING: last message repeated 100 times
[Sat Feb 25 11:41:06.010 2023] [1456978] WARNING: send() failed: 32: Broken pipe
[Sat Feb 25 11:41:06.010 2023] [1456978] WARNING: last message repeated 100 times
[Sat Feb 25 11:41:06.010 2023] [1456978] WARNING: send() failed: 32: Broken pipe
[Sat Feb 25 11:41:06.010 2023] [1456978] WARNING: last message repeated 100 times

What version of Manticore are you running?

Manticore 5.0.2 348514c86@220530 dev

What’s in the “searchd” section of the config?

searchd {
    listen = 0.0.0.0:9308:http_readonly
    log = /var/log/manticore/searchd.log
    query_log = /var/log/manticore/query.log
    query_log_mode = 666
    pid_file = /var/run/manticore/searchd.pid
    binlog_path = # disable logging
    query_log_format = plain # sphinxql,plain
    ssl_cert = /etc/ssl/certs/ssl-cert-snakeoil.pem
    ssl_key = /etc/ssl/private/ssl-cert-snakeoil.key
}

OK, I can’t reproduce it in 5.0.2 the same way I could reproduce this warning previously in 3.x.x (debug sleep N + kill -9 to break the connection). If you know a way to reproduce it, please create an issue on GitHub.