integration of Manticore with Filebeat

General Discussions - English
1

==========version==========
Manticore 6.2.0
filebeat version 8.9.0 (amd64)
CentOS Linux release 7.6.1810
==========conf============
##here is my Beats conf
[root@10-15-118-41 opt]# cat /etc/filebeat/filebeat.yml

"filebeat.inputs:
- type: filestream
id: testmanticore
enabled: true
paths:
- /opt/datacollect.log
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 1
setup.template.name: “waipan”
setup.template.pattern: “waipan”
setup.ilm.enabled: false
output.elasticsearch:
hosts: [“http://10.15.118.41:9308”]
index: “waipan”
allow_older_versions: true
processors:
- add_host_metadata:
when.not.contains.tags: forwarded

##here is manticore conf

searchd {
listen = 127.0.0.1:9312
listen = 0.0.0.0:9306:mysql
listen = 0.0.0.0:9308:http
log = /var/log/manticore/searchd.log
query_log = /var/log/manticore/query.log
pid_file = /run/manticore/searchd.pid
data_dir = /var/lib/manticore
}

=========error messages=======
##syslog

Aug 8 15:51:50 10-15-118-41 filebeat: {“log.level”:“error”,“@timestamp”:“2023-08-08T15:51:50.281+0800”,“log.logger”:“publisher_pipeline
_output”,“log.origin”:{“file.name”:“pipeline/client_worker.go”,“file.line”:148},“message”:“Failed to connect to backoff(elasticsearch(ht
tp://10.15.118.41:9308)): Connection marked as failed because the onConnect callback failed: error loading template: failed to put data
stream: could not put data stream: Put "http://10.15.118.41:9308/_data_stream/waipan\”: EOF. Response body: ",“service.name”:“filebeat”
,“ecs.version”:“1.6.0”}

##http error

http://10.15.118.41:9308/_data_stream/waipan
{“error”:{“type”:“index_not_found_exception”,“reason”:“no such index [_data_stream]”,“index”:“_data_stream”},“status”:404}

========need help=========
I clearly used the custom index name ‘waipan’ in Filebeat, but why am I getting an ‘index _data_stream’ does not exist error? What went wrong? Could you please help me? Thank you.

2

could you create ticket at Github where put reproducible example?

4

okay i’ll try it

5