Elasticsearch+Logstash+Filebeat+Kibana部署【7.1.1版本】-EW帮帮网

root@localhost ~]# hostnamectl set-hostname es
[root@localhost ~]# bash
[root@es ~]# rz
rz waiting to receive.**
Sending: elasticsearch-7.1.1-x86_64.rpm
 100% [====================] 00:00:07 346845802 bytes

[root@es ~]# yum install -y java
[root@es ~]# rpm -ivh elasticsearch-7.1.1-x86_64.rpm 
警告：elasticsearch-7.1.1-x86_64.rpm: 头V4 RSA/SHA512 Signature, 密钥 ID d88e42b4: N
准备中...                          ################################# [100%]
Creating elasticsearch group... OK
Creating elasticsearch user... OK
正在升级/安装...
   1:elasticsearch-0:7.1.1-1          ################################# [100%]
### NOT starting on installation, please execute the following statements to configu to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service
Created elasticsearch keystore in /etc/elasticsearch

配置主机名解析

[root@es elasticsearch]# scp /etc/hosts 192.168.44.12:/etc/
The authenticity of host '192.168.44.12 (192.168.44.12)' can't be established.
ECDSA key fingerprint is SHA256:mcPIFK1593usLRK8zsfuNNsBn39Jt2MC2L/GP1fhA1I.
ECDSA key fingerprint is MD5:00:85:24:3b:89:ba:ab:7e:35:49:f7:21:d2:0e:c7:f7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.44.12' (ECDSA) to the list of known hosts.
root@192.168.44.12's password: 
hosts                                             100%  238   371.6KB/s   00:00    
[root@es elasticsearch]# scp /etc/hosts 192.168.44.13:/etc/
The authenticity of host '192.168.44.13 (192.168.44.13)' can't be established.
ECDSA key fingerprint is SHA256:mcPIFK1593usLRK8zsfuNNsBn39Jt2MC2L/GP1fhA1I.
ECDSA key fingerprint is MD5:00:85:24:3b:89:ba:ab:7e:35:49:f7:21:d2:0e:c7:f7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.44.13' (ECDSA) to the list of known hosts.
root@192.168.44.13's password: 
hosts                                             100%  238   372.4KB/s   00:00    
[root@es elasticsearch]# scp /etc/hosts 192.168.44.110:/etc/
^C[root@es elasticsearch]# scp /etc/hosts 192.168.44.10:/etc/
The authenticity of host '192.168.44.10 (192.168.44.10)' can't be established.
ECDSA key fingerprint is SHA256:mcPIFK1593usLRK8zsfuNNsBn39Jt2MC2L/GP1fhA1I.
ECDSA key fingerprint is MD5:00:85:24:3b:89:ba:ab:7e:35:49:f7:21:d2:0e:c7:f7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.44.10' (ECDSA) to the list of known hosts.
root@192.168.44.10's password: 
hosts                                             100%  238   421.9KB/s   00:00    
[root@es elasticsearch]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.44.12 kibana
192.168.44.11 es
192.168.44.10 logstash
192.168.44.13 web1
[root@es elasticsear

配置elasticsearch

[root@es elasticsearch]# cat elasticsearch.yml | grep -Ev "^#"
node.name: es
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 192.168.44.11
http.port: 9200
cluster.initial_master_nodes: ["es"]

启动elasticsearch服务

[root@es elasticsearch]# systemctl start elasticsearch
[root@es elasticsearch]# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
   Active: active (running) since 四 2025-07-17 20:08:13 CST; 1h 0min ago
     Docs: http://www.elastic.co
 Main PID: 1737 (java)
   CGroup: /system.slice/elasticsearch.service
           ├─1737 /usr/share/elasticsearch/jdk/bin/java -Xms1g -Xmx1g -XX:+UseCon...
           └─1817 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_6...

7月 17 20:08:13 es systemd[1]: Started Elasticsearch.
7月 17 20:08:13 es elasticsearch[1737]: OpenJDK 64-Bit Server VM warning: Opti...e.
Hint: Some lines were ellipsized, use -l to show in full.

Kibana安装与配置

[root@kibana ~]# cat /etc/kibana/kibana.yml | grep -Ev "^#"
server.port: 5601
server.host: "192.168.44.12"
server.name: "kibana"
elasticsearch.hosts: ["http://192.168.44.11:9200"]
kibana.index: ".kibana"
i18n.locale: "zh-CN"

filebeat安装与配置

[root@web1 ~]# cat /etc/filebeat/filebeat.yml
- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/messages
    #- c:\programdata\elasticsearch\logs\*
output.logstash:
  # The Logstash hosts
  hosts: ["192.168.44.10:5044"]

logstash安装与配置

安装JAVA8、将logstash软件包拷贝至elk主机执行安装、优化logstash命令

[root@logstash ~]# yum install -y java
[root@logstash ~]# yum localinstall -y logstash-7.1.1.rpm
[root@logstash ~]# ln -s /usr/share/logstash/bin/logstash /usr/local/bin/

输出给logstash

创建配置文件

使用tags标签

input {
  beats {
    port => 5044
  }
}
output {
  if "access" in [tags] {
  	  elasticsearch {
   		 hosts => ["192.168.44.11:9200"]
   		 index => "access-%{+YYYY.MM.dd}"
  	}
  }
  if "error" in [tags] {
  	  elasticsearch {
   		 hosts => ["192.168.44.11:9200"]
   		 index => "error-%{+YYYY.MM.dd}"
  	}
  }
###日志进行标准输出，观察日志获取的过程###
  stdout {
  		codec => rubydebug
  }
}

使用filter过滤器

input {
        file {
                path => "/var/log/messages"
                start_position => "beginning"
        }
        beats {
                port => 5044
        }
}
filter {
  if [host][name] {
    mutate { add_field => { "hostname" => "%{[host][name]}" } }
  }
  else if [agent][hostname] {
    mutate { add_field => { "hostname" => "%{[agent][hostname]}" } }
  }
  else {
    mutate { add_field => { "hostname" => "%{host}" } }
  }
}
output {
  if [hostname] == "logstash" {
        elasticsearch {
                hosts => ["192.168.44.11:9200"]
                index => "system-log-%{+YYYY.MM.dd}"
        }
  }
  else if [hostname] == "web1" {
        elasticsearch {
                hosts => ["192.168.44.11:9200"]
                index => "web1-log-%{+YYYY.MM.dd}"
        }
  }
  stdout {
        codec => rubydebug
  }
}
~

kibana输出示例

收集不同主机及对应应用的不同日志

[root@logstash conf.d]# cat /etc/logstash/conf.d/pipline.conf
input {
        file {
                path => "/var/log/messages"
                start_position => "beginning"
        }
        beats {
                port => 5044
        }
}
filter {
  if [host][name] {
    mutate { add_field => { "hostname" => "%{[host][name]}" } }
  }
  else if [agent][hostname] {
    mutate { add_field => { "hostname" => "%{[agent][hostname]}" } }
  }
  else {
    mutate { add_field => { "hostname" => "%{host}" } }
  }
}
output {
  if [hostname] == "logstash" {
        elasticsearch {
                hosts => ["192.168.44.11:9200"]
                index => "system-log-%{+YYYY.MM.dd}"
        }
  }
  else if [hostname] == "web1" {
    if "system" in [tags] {
        elasticsearch {
                hosts => ["192.168.44.11:9200"]
                index => "web1-log-%{+YYYY.MM.dd}"
        }
  }
   if "nginx-access" in [tags] {
        elasticsearch {
                hosts => ["192.168.158.79:9200"]
                index => "web1-nginx-access-log-%{+YYYY.MM.dd}"
        }
    }
    if "nginx-error" in [tags] {
        elasticsearch {
                hosts => ["192.168.158.79:9200"]
                index => "web1-nginx-error-log-%{+YYYY.MM.dd}"
        }
    }
  }
  stdout {
        codec => rubydebug
  }
}

同时更改web1中filebeat的配置

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/messages
  tags: "system"
    #- c:\programdata\elasticsearch\logs\*
- type: log
  enabled: true
  path:
    - /var/log/nginx/access.log
  tags: "nginx-access"
- type: log
  enabled: true
  path:
    - /var/log/nginx/error.log
  tags: "nginx-error"

kibana输出示例

输出给es

直接更改filebeat的配置文件

[root@web1 ~]# vi /etc/filebeat/filebeat.yml
#注释掉logstash块，然后在es块做以下书写
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["192.168.44.11:9200"]
  indices:
   - index: "nginx1-access-logs"
     when:
      contains:
       { "message": "GET"}
   - index: "nginx1-error-logs"
     when:
      contains:
       { "message": "error" }
setup.ilm.enabled: false
setup.template.name: "nginx1"
setup.template.pattern: "nginx1-*"
  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

重启filebeat服务之后查看kibana输出示例

结果出现了nginx1-error-logs 和 nginx1-access-logs的日志

Elasticsearch+Logstash+Filebeat+Kibana部署【7.1.1版本】

集群模式

Elasticsearch安装与配置

将elasticsearch软件包拷贝至elk主机执行安装、安装JAVA8

配置主机名解析

配置elasticsearch

启动elasticsearch服务

Kibana安装与配置

filebeat安装与配置

logstash安装与配置

安装JAVA8、将logstash软件包拷贝至elk主机执行安装、优化logstash命令

输出给logstash

创建配置文件

使用tags标签

使用filter过滤器

kibana输出示例

收集不同主机及对应应用的不同日志

同时更改web1中filebeat的配置

kibana输出示例

输出给es

直接更改filebeat的配置文件

重启filebeat服务之后查看kibana输出示例

网站公告

今日签到

热门文章

最新发布

Elasticsearch+Logstash+Filebeat+Kibana部署【7.1.1版本】

集群模式

Elasticsearch安装与配置

将elasticsearch软件包拷贝至elk主机执行安装 、安装JAVA8

配置主机名解析

配置elasticsearch

启动elasticsearch服务

Kibana安装与配置

filebeat安装与配置

logstash安装与配置

安装JAVA8、将logstash软件包拷贝至elk主机执行安装 、优化logstash命令

输出给logstash

创建配置文件

使用tags标签

使用filter过滤器

kibana输出示例

收集不同主机及对应应用的不同日志

同时更改web1中filebeat的配置

kibana输出示例

输出给es

直接更改filebeat的配置文件

重启filebeat服务之后查看kibana输出示例

网站公告

今日签到

热门文章

最新发布

将elasticsearch软件包拷贝至elk主机执行安装、安装JAVA8

安装JAVA8、将logstash软件包拷贝至elk主机执行安装、优化logstash命令