Prometheus is the defacto monitoring tool for Cloud native applications and microservices. You can’t talk Docker and Kubernetes infrastructure monitoring without mentioning Prometheus. To achieve complete monitoring, alerting and visualization, Grafana usually comes into the mix.

Below are the steps to install Prometheus monitoring tool on RHEL 8.

Step 1: Add system user and group for Prometheus

Let’s kick off the installation of Prometheus on RHEL 8 by creating a dedicated user that will run and manage Prometheus service. This is a system user that doesn’t have access to console/shell login.

sudo groupadd --system prometheus
sudo useradd -s /sbin/nologin --system -g prometheus prometheus

Note that this user doesn’t have /bin/bash shell, that’s why we used -s /sbin/nologin.

Step 2: Set NTP Server

To avoid any time drift, configure NTP server on Prometheus server to provide accurate time.

How to Configure NTP Server Using Chrony on RHEL / CentOS 8

Step 3: Create data directory for Prometheus

Once the system user and group has been created, proceed to create a directory that will be used to store Prometheus data. This includes the metrics collected from the agents being monitored.

sudo mkdir /var/lib/prometheus

You can choose to use a different path, e.g separate partition.

Step 4: Create configuration directories for Prometheus

Prometheus primary configuration files directory is /etc/prometheus/. It will have some sub-directories.

for i in rules rules.d files_sd; do
 sudo mkdir -p /etc/prometheus/${i};
done

Step 5: Download Prometheus on CentOS 8 / RHEL 8

We need to download the latest release of Prometheus archive and extract it to get binary files. You can check releases from Prometheus releases Github page.

You can use curl or wget to download from the command line.

curl -s https://api.github.com/repos/prometheus/prometheus/releases/latest \
  | grep browser_download_url \
  | grep linux-amd64 \
  | cut -d '"' -f 4 \
  | wget -qi -

Extract the file and move it to directory in your $PATH

tar xvf prometheus-*.tar.gz
cd prometheus-*/
sudo cp prometheus promtool /usr/local/bin/

Also copy consoles and console_libraries to /etc/prometheus directory:

sudo cp -r consoles/ console_libraries/ /etc/prometheus/

Step 6: Create a Prometheus configuration file.

Prometheus configuration file will be located under /etc/prometheus/prometheus.yml. Create simple configurations using content:

# Global config
global: 
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.  
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.  
  scrape_timeout: 15s  # scrape_timeout is set to the global default (10s).

# A scrape configuration containing exactly one endpoint to scrape:# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']

Make changes to the file to fit your initial setting and save the file.

Step 7: Create systemd Service unit

To be able to manage Prometheus service with systemd, you need to explicitly define this unit file.

Create a file

sudo vi /etc/systemd/system/prometheus.service

Add the following contents to it.

[Unit]
Description=Prometheus
Documentation=https://prometheus.io/docs/introduction/overview/
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
Environment="GOMAXPROCS=2"
User=prometheus
Group=prometheus
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.listen-address=0.0.0.0:9090 \
  --web.external-url=

SyslogIdentifier=prometheus
Restart=always

[Install]
WantedBy=multi-user.target

Remember to edit the line:

Environment="GOMAXPROCS=2

Replace 2 with the number of vcpus on your server.

Set correct directory permissions.

for i in rules rules.d files_sd; do
  sudo chown -R prometheus:prometheus /etc/prometheus/${i};
done
for i in rules rules.d files_sd; do
  sudo chmod -R 775 /etc/prometheus/${i};
done
sudo chown -R prometheus:prometheus /var/lib/prometheus/

Start Prometheus service.

sudo systemctl daemon-reload
sudo systemctl start prometheus

Check status using systemctl status prometheus command:

$ systemctl status prometheus.service 
 ● prometheus.service - Prometheus
    Loaded: loaded (/etc/systemd/system/prometheus.service; disabled; vendor preset: disabled)
    Active: active (running) since Fri 2019-01-11 00:36:17 EAT; 4min 44s ago
      Docs: https://prometheus.io/docs/introduction/overview/
  Main PID: 7576 (prometheus)
     Tasks: 9 (limit: 11510)
    Memory: 19.8M
    CGroup: /system.slice/prometheus.service
            └─7576 /usr/local/bin/prometheus --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/prometheus --web.console.templat>
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.709504055Z caller=main.go:244 build_context="(go=go1.11.3, user=root@b>
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.709529058Z caller=main.go:245 host_details="(Linux 4.18.0-32.el8.x86_6>
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.709557341Z caller=main.go:246 fd_limits="(soft=1024, hard=4096)"
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.709576706Z caller=main.go:247 vm_limits="(soft=unlimited, hard=unlimit>
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.712203022Z caller=main.go:561 msg="Starting TSDB …"
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.712231744Z caller=web.go:429 component=web msg="Start listening for co>
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.717664176Z caller=main.go:571 msg="TSDB started"
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.71771626Z caller=main.go:631 msg="Loading configuration file" filename>
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.718015628Z caller=main.go:657 msg="Completed loading of configuration >
 Jan 11 00:36:17 rhel8.local prometheus[7576]: level=info ts=2019-01-10T21:36:17.71803238Z caller=main.go:530 msg="Server is ready to receive web reque>

Step 8: Configure firewalld

I’ll allow access to Prometheus management interface port 9090 from my trusted network using Firewalld rich rules.

sudo firewall-cmd --permanent --add-rich-rule 'rule family="ipv4" \
source address="192.168.122.0/24" port protocol="tcp" port="9090" accept'

sudo firewall-cmd --reload

If you want to allow from any IP, use:

sudo firewall-cmd --add-port=9090/tcp --permanent
sudo firewall-cmd --reload

Open Prometheus Server IP/Hostname and port 9090

install prometheus rhel8 centos8 03

You now have Prometheus Server installed on CentOS 8 / RHEL 8 Linux system. Also check our Prometheus monitoring guides below.

Monitoring Ceph Cluster with Prometheus and Grafana

Monitoring Ceph Cluster with Prometheus and Grafana

How to Monitor Linux Server Performance with Prometheus and Grafana in 5 minutes

How to Monitor BIND DNS server with Prometheus and Grafana

How to Monitor Redis Server with Prometheus and Grafana in 5 minutes

Monitoring MySQL / MariaDB with Prometheus in five minutes 

How to Monitor Apache Web Server with Prometheus and Grafana in 5 minutes