ElasticSearch 7.8.1集群搭建

标签:tin   xpl   false   初始   ast   多个   restart   als   max   

通往集群的大门

集群由什么用?

高可用

  高可用(High Availability)是分布式系统架构设计中必须考虑的因素之一,它通常是指,通过设计减少系统不能提供服务的时间。如果系统每运行100个时间单位,会有1个时间单位无法提供服务,我们说系统的可用性是99%。

负载均衡

  将流量均衡的分布在不同的节点上,每个节点都可以处理一部分负载,并且可以在节点之间动态分配负载,以实现平衡。

高性能

  将流量分发到不同机器,充分利用多机器多CPU,从串行计算到并行计算提供系统性能。

ES集群的基本核心概念

Cluster集群

  一个ElasticSearch集群由一个或多个节点(Node)组成,每个集群都有一个共同的集群名称作为标识。

Node节点

  一个ElasticSearch实例即一个Node,一台机器可以有多个实例,正常使用下每个实例应该会部署在不同机器上。ElasticSearch的配置文件中可以通过node.master、node.data来设置节点类型。

  node.master:表示节点是否具有称为主节点的资格

    true代表的是有资格竞选主节点

    false代表的是没有资格竞选主节点

  node.data:表示节点是否存储数据

Node节点组合

主节点+数据节点(master+data)

    节点即有称为主节点的资格,又存储数据

node.master: true
node.data: true

数据节点(data)

  节点没有成为主节点的资格,不参与选举,只会存储数据

node.master: false
node.data: true

客户端节点(client)

  不会成为主节点,也不会存储数据,主要是针对海量请求的时候,可以进行负载均衡

node.master: false
node.data: false

分片

  每个索引有一个或多个分片,每个分片存储不同的数据。分片可分为主分片(primary shard)和复制分片(replica shard),复制分片是主分片的拷贝。默认每个主分片有一个复制分片,一个索引的复制分片的数量可以动态地调整,复制分片匆匆不与它的主分片在同一个节点上。

搭建ES集群

搭建步骤

  • 拷贝ES7.8.1安装包3份,分别命名es-a,es-b,es-c
  • 分别修改elasticsearch.yml文件
  • 分别启动a、b、c三个节点
  • 打开浏览器输入:ip:port/_cat/health?v,如果返回的nodt.total是3,代表集群搭建成功

配置文件

#集群名称,三台集群,要配置相同的集群名称!!!
cluster.name: my-application
#节点名称
node.name: node-1 #是不是有资格主节点
node.master: true
#是否存储数据
node.data: true
#最?集群节点数
node.max_local_storage_nodes: 3 #?关地址
network.host: 0.0.0.0
#端?
http.port: 9200
#内部节点之间沟通端?
transport.tcp.port: 9300
#es7.x 之后新增的配置,写?候选主节点的设备地址,在开启服务后可以被选为主节点
discovery.seed_hosts: ["localhost:9300","localhost:9400","localhost:9500"]
#es7.x 之后新增的配置,初始化?个新的集群时需要此配置来选举master
cluster.initial_master_nodes: ["node-1", "node-2","node-3"] #数据和存储路径
path.data: /Users/louis.chen/Documents/study/search/storage/a/data
path.logs: /Users/louis.chen/Documents/study/search/storage/a/logs

注意

  因为我是一台服务器上,搞了3搞ES启动端口号不同,拷贝了3份es,具体还要跟实际情况相应调整

真实配置-a

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
# 集群名称
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# 集群名称
node.name: node-1
# 是不是有资格主节点
node.master: true
# 是否存储数据
node.data: true
# 最大集群节点数,因为3个集群,所以配置3
node.max_local_storage_nodes: 3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
# 数据存储路径
path.data: /var/soft/es7.8.1/elasticsearch-7.8.1/data
# 日志存储路径
# Path to log files:
#
path.logs: /var/soft/es7.8.1/elasticsearch-7.8.1/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
# 网关地址
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
# 端口
http.port: 9200
# 内部节点之间沟通端口
transport.tcp.port: 9300
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
# es7.x之后新增的配置,写入候选主节点的设备地址,在开启服务后可以被选为主节点
discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9400", "127.0.0.1:9500"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
# es7.x之后新增的配置,初始化一个新的集群时需要此配置来选举master
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -------------------------bootstrap.system_call_filter: false----------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
bootstrap.system_call_filter: false
http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization
http.cors.allow-credentials: true

真实配置-b

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
# 集群名称
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# 集群名称
node.name: node-2
# 是不是有资格主节点
node.master: true
# 是否存储数据
node.data: true
# 最大集群节点数,因为3个集群,所以配置3
node.max_local_storage_nodes: 3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
# 数据存储路径
path.data: /var/soft/es-b/data
# 日志存储路径
# Path to log files:
#
path.logs: /var/soft/es-b/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
# 网关地址
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
# 端口
http.port: 9201
# 内部节点之间沟通端口
transport.tcp.port: 9400
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
# es7.x之后新增的配置,写入候选主节点的设备地址,在开启服务后可以被选为主节点
discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9400", "127.0.0.1:9500"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
# es7.x之后新增的配置,初始化一个新的集群时需要此配置来选举master
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -------------------------bootstrap.system_call_filter: false----------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
bootstrap.system_call_filter: false
http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization
http.cors.allow-credentials: true

只修改了,2个端口、数据和日志存储路径

真实配置-c

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
# 集群名称
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# 集群名称
node.name: node-3
# 是不是有资格主节点
node.master: true
# 是否存储数据
node.data: true
# 最大集群节点数,因为3个集群,所以配置3
node.max_local_storage_nodes: 3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
# 数据存储路径
path.data: /var/soft/es-c/data
# 日志存储路径
# Path to log files:
#
path.logs: /var/soft/es-c/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
# 网关地址
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
# 端口
http.port: 9202
# 内部节点之间沟通端口
transport.tcp.port: 9500
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
# es7.x之后新增的配置,写入候选主节点的设备地址,在开启服务后可以被选为主节点
discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9400", "127.0.0.1:9500"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
# es7.x之后新增的配置,初始化一个新的集群时需要此配置来选举master
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -------------------------bootstrap.system_call_filter: false----------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
bootstrap.system_call_filter: false
http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-headers : X-Requested-With,X-Auth-Token,Content-Type,Content-Length,Authorization
http.cors.allow-credentials: true

只修改了,2个端口、数据和日志存储路径

踩坑录(必看

坑一

技术图片

 

坑二

  因为我是在一台linux上搭建的集群,然后用端口号9200、9201、9202区分开来的,es-b、es-c是拷贝es-a的,但是es-a的data目录有数据,需要将es-b和es-c的data目录清空,然后重启es即可!!!

启动

技术图片

 

集群搭建成功

技术图片

配置kibana

打开配置kibana.yml,添加elasticsearch.hosts: ["http://192.168.199.170:9200", "http://192.168.199.170:9201", "http://192.168.199.170:9202"]

技术图片

 

然后重启即可

 

ElasticSearch 7.8.1集群搭建

标签:tin   xpl   false   初始   ast   多个   restart   als   max   

原文地址:https://www.cnblogs.com/chenyanbin/p/13493920.html

版权声明:完美者 发表于 2020-08-13 12:29:31。
转载请注明:ElasticSearch 7.8.1集群搭建 | 完美导航

暂无评论

暂无评论...