What is Prometheus and How to install and configure it on Linux server ?

What is Prometheus?

Prometheus is a popular Opensource system monitoring and alerting tool Originally developed by SoundCloud, then made open source and accepted as the second project in the Cloud Native Computing Foundation (CNCF) in 2016. It is similar in design to Google’s Borgmon monitoring system, and a relatively modest system can handle collecting hundreds of thousands of metrics every second.

Prometheus Architecture diagram :

Prometheus is a multi-component system some of which are optional

  • Client libraries (Instrumenting application code)
  • Push gateway (To support metric collections from short-lived jobs)
  • Special purpose exporters (for services like HAProxy, StatsD, Graphite, etc.)
  • Alert manager (to handle alerts)
  • Additional support tools

How does it work?

Prometheus is meant to be used to collect metrics from an exposed HTTP endpoint. It stores all the collected metrics locally and runs rules over the data to aggregate and record new time series from existing data or generate alerts. API consumers like Grafana can be used to visualize

  • Queries produced by PromQL (Prometheus Query Language) enables users to select and aggregate time-series data in real-time.
  • PromQL (Prometheus Query Language) also allows you to set up triggers that will send push notifications to other sources, like Slack, PagerDuty, or Email.

Minimum Requirements to Install Prometheus :

  • 2 CPU cores
  • 4 GB of memory
  • 20 GB of free disk space

Installation Process :

  • This guide explains how to install and configure the latest Prometheus on a Linux VM.
sudo useradd --no-create-home --shell /bin/false prometheus
sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
sudo chown prometheus:prometheus /etc/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus
curl -LO https://github.com/prometheus/prometheus/releases/download/v2.28.1/prometheus-2.28.1.linux-amd64.tar.gz
tar -xvf prometheus-2.28.1.linux-amd64.tar.gz
mv prometheus-2.18.1.linux-amd64 prometheus-files
sudo cp prometheus-files/prometheus /usr/local/bin/
sudo cp prometheus-files/promtool /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool
sudo cp -r prometheus-files/consoles /etc/prometheussudo cp -r prometheus-files/console_libraries /etc/prometheussudo chown -R prometheus:prometheus /etc/prometheus/consolessudo chown -R prometheus:prometheus /etc/prometheus/console_libraries
sudo cat << EOF > /etc/prometheus/ prometheus.yml
global:
 external_labels:
  region: us-central
  monitor: infra
alerting:
 alertmanagers:
 - static_configs:
    - targets:
     - “localhost:9093”
rule_files:
 - “rules.yml”
scrape_configs:
 - job_name: prometheus
   scrape_interval: 5s
   static_configs:
    - targets:
       - “localhost:9090”
 - job_name: nsq_exporter
   scrape_interval: 5s
   static_configs:
   - targets:
     - “<Target_Server_IP>:9127”
EOF
sudo cat << EOF > /etc/systemd/system/prometheus.service
[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target
[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
  --config.file /etc/prometheus/prometheus.yml \
  --storage.tsdb.path /monitoring/prometheus/prometheus_data/ \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.external-url=https://prom.example.com \
  --web.listen-address=:9090 \
  --web.enable-lifecycle \
  --web.enable-admin-api \
  --log.level=info
[Install]
WantedBy=multi-user.target
EOF
  • Reload the system service to register the prometheus service and start the prometheus service
sudo systemctl daemon-reload
sudo systemctl start prometheus
sudo systemctl status prometheus

mkdir /etc/alertmanager/
sudo cat << EOF > /etc/alertmanager/alertmanager.yml
route:
group_by: [‘alertname’, ‘cluster’, ‘service’]
group_wait: 30s
group_interval: 2m
repeat_interval: 3m
receiver: slack_general
routes:
- match:
severity: slack
receiver: slack_general
receivers:
- name: slack_general
  slack_configs:
  - api_url:‘https://hooks.slack.com/services/T596DT44S/xxxxxxxxxxxxxxxxxxxxx'
channel: ‘#monitoring-alerts’
send_resolved: true
EOF
mkdir /var/lib/alertmanager
sudo cat << EOF > /etc/systemd/system/alertmanager.service
[Unit]
Description=AlertManager Server Service
Wants=network-online.target
After=network-online.target[Service]
User=root
Group=root
Type=Simple
ExecStart=/usr/local/bin/alertmanager \
--config.file /etc/alertmanager/alertmanager.yml
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl start alertmanager
systemctl enable alertmanager
  • Port 9100 opened in server firewall as Prometheus reads metrics on this port.
  • Setup Node Exporter Binary
cd /tmpcurl -LO https://github.com/prometheus/node_exporter/releases/download/v0.18.1/node_exporter-0.18.1.linux-amd64.tar.gz
sudo mv node_exporter-0.18.1.linux-amd64/node_exporter /usr/local/bin
sudo useradd -rs /bin/false node_exporter
sudo cat << EOF > /etc/systemd/system/node_exporter.service
[Unit]
Description=Node Exporter
After=network.target
[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter

Configure the Server as Target on Prometheus Server

  • Now that we have the node exporter up and running on the server, we have to add this server as a target on the Prometheus server configuration.
  • Job name can be your server hostname or IP for identification purposes.
- job_name: ‘node_exporter_metrics’
   scrape_interval: 5s
   static_configs:
    - targets: [‘TARGET_SERVER_IP:9100’]
sudo systemctl restart prometheus

Leave your thought here

Your email address will not be published.