Prometheus+Grafana实现对服务的监控

Prometheus+Grafana实现对服务的监控

前言：Prometheus+Grafana实现监控会更加全面，监控的组件更多
Prometheus官网 https://prometheus.io/docs/prometheus/latest/getting_started/
Grafana官网 https://grafana.com/docs/

一、安装Prometheus+Grafana

这里采用docker安装

1.1 在服务器建3个文件夹

/mydata/prometheus/config
/mydata/prometheus/data
/mydata/grafana/storage
/mydata/grafana/conf

1.2 同步时间和目录授权

1. 打开vmware顶部菜单栏： 虚拟机==>设置==>选项==>VMware Tool==>将客户机时间与主机同步
   2.授权data目录： chown -R 777 /mydata/prometheus/data

1.3 Prometheus的配置文件准备

把prometheus的配置文件prometheus.yml放进 /mydata/prometheus/config
目录，prometheus.yml来源，可以去上面官网下载一个prometheus，然后把他的prometheus.yml拖进/mydata/prometheus/config

1.4 docker安装

docker run -d --name=prometheus \
-p 9090:9090 \
-v /mydata/prometheus/config/prometheus.yml:/etc/prometheus/prometheus.yml \
-v /mydata/prometheus/data:/prometheus \
-v /etc/localtime:/etc/localtime:ro \
prom/prometheus

查看：http://192.168.174.198:9090/query 能看到输出，则安装成功

1.5 安装grafana

docker run -d -p 3000:3000 --name=grafana \
-v /mydata/grafana/storage:/var/lib/Grafana \
-v /mydata/grafana/conf/defaults.ini:/usr/share/grafana/conf/defaults.ini \
-v /etc/localtime:/etc/localtime:ro \
grafana/grafana

defaults.ini我是用docker cp 复制出来的，也可以下载一个grafana，粘进去。目的是想汉化，修改defaults.ini中的default_language = zh-Hans 。但是实际并没有全部汉化，部分汉化
至此可以打开 grafana的地址：http://192.168.174.198:3000能看到页面就算安装成功

1.6 给grafana添加promethus数据源

选择grafana左侧的菜单:
Data source ==> 点击右上角的 add new data source
==>选择prometheus ==> 在Connection 地址数据我们的promethus的地址http://192.168.174.198:9090 ==>点击最下方的 save&test

二、监控docker

2.1 安装cadvisor

docker run -d \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --volume=/dev/disk/:/dev/disk:ro \
  -v /etc/localtime:/etc/localtime:ro \
  --publish=8181:8080 \
  --detach=true \
  --name=cadvisor \
  google/cadvisor:latest

鉴于8080容易冲突，我这映射到docker对外端口8181

2.2 修改prometheus.yml

在prometheus.yml添加如下，用于监控docker详情

- job_name: 'docker'
    static_configs:
      - targets: ['192.168.174.198:8181']
        labels:
          instance: docker

此时prometheus.yml的内容如下：

global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"
    static_configs:
      - targets: ["192.168.174.198:9090"]
        labels:
          app: "prometheus"
  - job_name: 'docker'
    static_configs:
      - targets: ['192.168.174.198:8181']
        labels:
          instance: docker

2.3 配置监控面板

打开grafana面板，点击右上角的+选择 import dashboard，在弹出的页面输入193，监控docker的面板用193就挺好。
下面的prometheus 选择刚才的prometheus 即可打开对docker的监控画面

三、监控linux服务器

3.1 安装node-exporter

docker run -d -p 9100:9100 --name=node-exporter \
-v "/proc:/host/proc:ro" \
-v "/sys:/host/sys:ro" \
-v "/:/rootfs:ro" \
-v /etc/localtime:/etc/localtime:ro \
--net="host" \
prom/node-exporter

3.2 添加prometheus.yml配置

在上面的基础上添加

  - job_name: 'node-explore'
    static_configs:
      - targets: ['192.168.174.198:9100']
        labels:
          instance: node-explore

3.3 配置grafana面板

打开grafana面板，点击右上角的+选择 import dashboard，在弹出的页面输入9276，监控模板可以自己去试探，我用9276号模板挺方便。
下面的prometheus 选择刚才的prometheus 即可打开对linux服务器的监控画面

四、prometheus监控微服务(springboot)

4.1导包

 <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
        <groupId>io.micrometer</groupId>
        <artifactId>micrometer-registry-prometheus</artifactId>
    </dependency>

我的springcloud 2023.0.3 /springcloud alibaba 2023.0.1.2 /springboot3.3.5
这里引入的actuator和prometheus的maven的版本分别是：3.3.5、1.9.8

4.2 yml配置

management:
  server:
  # 这个端口是prometheus与微服务通信的端口
    port: 8100
  endpoints:
    web:
      exposure:
        # 从技术上更改端点的暴露 -- 通过HTTP公开所有的端点，可通过 /actuator/{ID} 去查看，如 /actuator/beans
        include: "*"
      base-path: /actuator
    jmx:
      exposure:
        include: "*"
  endpoint:
    prometheus:
      enabled: true
    health:
      show-details: always
    httptrace:
      enabled: true
    metrics:
      enabled: true
      export:
        prometheus:
          enabled: true
  prometheus:
    metrics:
      export:
        enabled: true

4.3 主启动类添加

@Value("${spring.application.name}")
private String application;

@Bean
MeterRegistryCustomizer<MeterRegistry> configurer(){
    return (registry)->registry.config().commonTags("application",application);
}

4.4 修改prometheus.yml配置

  - job_name: 'gateway'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['192.168.20.109:8100']
        labels:
          instance: springboot

最终的prometheus.yml内容如下：

global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["192.168.174.198:9090"]
       # The label name is added as a label `label_name=<label_value>` to any timeseries scraped from this config.
        labels:
          app: "prometheus"
  - job_name: 'docker'
    static_configs:
      - targets: ['192.168.174.198:8181']
        labels:
          instance: docker
  - job_name: 'node-explore'
    static_configs:
      - targets: ['192.168.174.198:9100']
        labels:
          instance: node-explore
  - job_name: 'admin'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['192.168.20.109:8096']
        labels:
          instance: springboot
  - job_name: 'auth'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['192.168.20.109:8097']
        labels:
          instance: springboot

  - job_name: 'order'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['192.168.20.109:8098']
        labels:
          instance: springboot
  - job_name: 'storage'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['192.168.20.109:8099']
        labels:
          instance: springboot
  - job_name: 'gateway'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['192.168.20.109:8100']
        labels:
          instance: springboot

4.5 配置grafana面板

打开grafana面板，点击右上角的+选择 import dashboard，在弹出的页面输入12900，监控springboot我用的是12900。 下面的prometheus 选择刚才的prometheus 即可打开对springboot服务器的监控画面

五、常用grafana监控组件与模板id总结

微服务性能监控springboot：12900、4701、10280
docker环境性能监控：893 、193
nacos性能监控：13221
mysql性能监控：9362
elasticsearch：266
Redis监控模板：11835
Jmeter： 5496
linux服务器 ：12633 、9276、8919