DevOps Tools/Prometheus

Overview | Prometheus Server | Prometheus Node Exporter

Overview

Metric Collector manager. Uses a scrape method by default, so the prometheus server needs to have connectivity to each node exporter.

CentOS 7 Installation

Create dependencies

~$ sudo yum install -y wget curl
~$ sudo groupadd --system prometheus
~$ sudo useradd -s /sbin/nologin --system -g prometheus prometheus
~$ sudo mkdir /var/lib/prometheus
~$ for i in rules rules.d files_sd; do sudo mkdir -p /etc/prometheus/${i}; done
~$ mkdir -p /tmp/prometheus && cd /tmp/prometheus

Download and setup

~$ curl -s https://api.github.com/repos/prometheus/prometheus/releases/latest | grep browser_download_url | grep linux-amd64 | cut -d '"' -f 4 | wget -qi -
~$ tar xvf prometheus*.tar.gz
~$ cd prometheus*/
~$ sudo mv prometheus promtool /usr/local/bin/
~$ sudo mv prometheus.yml  /etc/prometheus/prometheus.yml
~$ sudo mv consoles/ console_libraries/ /etc/prometheus/
~$ cd ~/
~$ rm -rf /tmp/prometheus

Create config

~$ sudo vim /etc/prometheus/prometheus.yml

prometheus config

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']

Setup permissions

~$ for i in rules rules.d files_sd; do sudo chown -R prometheus:prometheus /etc/prometheus/${i}; done
~$ for i in rules rules.d files_sd; do sudo chmod -R 775 /etc/prometheus/${i}; done
~$ sudo chown -R prometheus:prometheus /var/lib/prometheus/ 

Setup Firewall rules

~$ sudo firewall-cmd --zone=public --permanent --add-port=9090/tcp 
~$ sudo firewall-cmd --reload

Create systemd service

~$ sudo vim /etc/systemd/system/prometheus.service

systemd service

[Unit]
Description=Prometheus
Documentation=https://prometheus.io/docs/introduction/overview/
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=prometheus
Group=prometheus
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.listen-address=0.0.0.0:9090 \
  --web.external-url=

SyslogIdentifier=prometheus
Restart=always

[Install]
WantedBy=multi-user.target

~$ sudo systemctl daemon-reload
~$ sudo systemctl start prometheus
~$ sudo systemctl enable prometheus

Test

Test connection to the server running Prometheus on port 9090