Dear all,
I am successfully running a Manticore instance on Kubernetes using Manticore’s latest Docker Image (3.5.4). Unfortunately, I’m facing an issue trying to set up a replication Cluster.
I have created two pods (running on two differents nodes) named manticore1 and manticore2, with the following config:
searchd
{
listen = 9306:mysql41
listen = 9312
listen = 9322:replication
log = /var/log/manticore/query.log
query_log = /var/log/manticore/searchd.log
pid_file = /var/run/manticore/searchd.pid
preopen_indexes = 1
binlog_path = /var/lib/manticore
data_dir = /var/lib/manticore
collation_server = utf8_general_ci
max_packet_size = 64M
max_open_files = max
}
The ports 9306, 9312 and 9322 are opened on each pod’s service.
I am able to connect to each of them using their “kubernetes” name :
mysql -h manticore1
mysql -h manticore2
On the first node, I am doing this :
- CREATE CLUSTER xxx;
- ALTER CLUSTER xxx ADD index1;
- ALTER CLUSTER xxx ADD index2;
When I issue the following query on Manticore2, it hangs and in the end I have this error:
JOIN CLUSTER profiles AT ‘manticore1:9312’;
ERROR 1064 (42000): replication connection failed: 7 ‘error in node state, must reinit’
Unfortunately, I can’t issue an IP address in the config file, is it mandatory?
Did I forget something?
Best regards,
JF
PS : Here is the output of SHOW STATUS on the first node :
mysql> show status;
+---------------------------------------------+-------------------------------------------+
| Counter | Value |
+---------------------------------------------+-------------------------------------------+
| uptime | 745 |
| connections | 1716 |
| maxed_out | 0 |
| version | 3.5.4 13f8d08d@201211 release |
| mysql_version | 3.5.4 13f8d08d@201211 release |
| command_search | 0 |
| command_excerpt | 0 |
| command_update | 0 |
| command_keywords | 0 |
| command_persist | 0 |
| command_status | 2 |
| command_flushattrs | 0 |
| command_sphinxql | 0 |
| command_ping | 0 |
| command_delete | 0 |
| command_set | 0 |
| command_insert | 0 |
| command_replace | 0 |
| command_commit | 1712 |
| command_suggest | 0 |
| command_json | 0 |
| command_callpq | 0 |
| command_clusterpq | 0 |
| command_getfield | 1 |
| agent_connect | 0 |
| agent_tfo | 0 |
| agent_retry | 0 |
| queries | 0 |
| dist_queries | 0 |
| workers_total | 2 |
| workers_active | 3430 |
| workers_clients | 1714 |
| work_queue_length | 1716 |
| query_wall | 0.000 |
| query_cpu | OFF |
| dist_wall | 0.000 |
| dist_local | 0.000 |
| dist_wait | 0.000 |
| query_reads | OFF |
| query_readkb | OFF |
| query_readtime | OFF |
| avg_query_wall | 0.000 |
| avg_query_cpu | OFF |
| avg_dist_wall | 0.000 |
| avg_dist_local | 0.000 |
| avg_dist_wait | 0.000 |
| avg_query_reads | OFF |
| avg_query_readkb | OFF |
| avg_query_readtime | OFF |
| qcache_max_bytes | 16777216 |
| qcache_thresh_msec | 3000 |
| qcache_ttl_sec | 60 |
| qcache_cached_queries | 0 |
| qcache_used_bytes | 0 |
| qcache_hits | 0 |
| cluster_name | profiles |
| cluster_profiles_state_uuid | b929325a-451b-11eb-9d65-47cbb4afc64e |
| cluster_profiles_conf_id | 1 |
| cluster_profiles_status | primary |
| cluster_profiles_size | 1 |
| cluster_profiles_local_index | 0 |
| cluster_profiles_node_state | synced |
| cluster_profiles_nodes_set | |
| cluster_profiles_nodes_view | 127.0.0.1:9312,127.0.0.1:9322:replication |
| cluster_profiles_indexes_count | 2 |
| cluster_profiles_indexes | digsty,suggests |
| cluster_profiles_local_state_uuid | b929325a-451b-11eb-9d65-47cbb4afc64e |
| cluster_profiles_protocol_version | 9 |
| cluster_profiles_last_applied | 2 |
| cluster_profiles_last_committed | 2 |
| cluster_profiles_replicated | 2 |
| cluster_profiles_replicated_bytes | 288 |
| cluster_profiles_repl_keys | 2 |
| cluster_profiles_repl_keys_bytes | 64 |
| cluster_profiles_repl_data_bytes | 86 |
| cluster_profiles_repl_other_bytes | 0 |
| cluster_profiles_received | 2 |
| cluster_profiles_received_bytes | 178 |
| cluster_profiles_local_commits | 0 |
| cluster_profiles_local_cert_failures | 0 |
| cluster_profiles_local_replays | 0 |
| cluster_profiles_local_send_queue | 0 |
| cluster_profiles_local_send_queue_max | 1 |
| cluster_profiles_local_send_queue_min | 0 |
| cluster_profiles_local_send_queue_avg | 0.000000 |
| cluster_profiles_local_recv_queue | 0 |
| cluster_profiles_local_recv_queue_max | 2 |
| cluster_profiles_local_recv_queue_min | 0 |
| cluster_profiles_local_recv_queue_avg | 0.500000 |
| cluster_profiles_local_cached_downto | 1 |
| cluster_profiles_flow_control_paused_ns | 0 |
| cluster_profiles_flow_control_paused | 0.000000 |
| cluster_profiles_flow_control_sent | 0 |
| cluster_profiles_flow_control_recv | 0 |
| cluster_profiles_flow_control_interval | [ 100, 100 ] |
| cluster_profiles_flow_control_interval_low | 100 |
| cluster_profiles_flow_control_interval_high | 100 |
| cluster_profiles_flow_control_status | OFF |
| cluster_profiles_cert_deps_distance | 1.000000 |
| cluster_profiles_apply_oooe | 0.000000 |
| cluster_profiles_apply_oool | 0.000000 |
| cluster_profiles_apply_window | 1.000000 |
| cluster_profiles_commit_oooe | 0.000000 |
| cluster_profiles_commit_oool | 0.000000 |
| cluster_profiles_commit_window | 1.000000 |
| cluster_profiles_local_state | 4 |
| cluster_profiles_local_state_comment | Synced |
| cluster_profiles_cert_index_size | 2 |
| cluster_profiles_cert_bucket_count | 4 |
| cluster_profiles_gcache_pool_size | 1688 |
| cluster_profiles_causal_reads | 0 |
| cluster_profiles_cert_interval | 0.000000 |
| cluster_profiles_open_transactions | 0 |
| cluster_profiles_open_connections | 0 |
| cluster_profiles_ist_receive_status | |
| cluster_profiles_ist_receive_seqno_start | 0 |
| cluster_profiles_ist_receive_seqno_current | 0 |
| cluster_profiles_ist_receive_seqno_end | 0 |
| cluster_profiles_incoming_addresses | 127.0.0.1:9312,127.0.0.1:9322:replication |
| cluster_profiles_cluster_weight | 1 |
| cluster_profiles_desync_count | 0 |
| cluster_profiles_evs_delayed | |
| cluster_profiles_evs_evict_list | |
| cluster_profiles_evs_repl_latency | 0/0/0/0/0 |
| cluster_profiles_evs_state | OPERATIONAL |
| cluster_profiles_gcomm_uuid | b928f5ce-451b-11eb-af8b-a2db647dffd6 |