<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Monitoring and Alerts on SILO</title><link>https://silo.pgsty.com/operations/monitoring/</link><description>Recent content in Monitoring and Alerts on SILO</description><generator>Hugo</generator><language>en</language><atom:link href="https://silo.pgsty.com/operations/monitoring/index.xml" rel="self" type="application/rss+xml"/><item><title>Monitoring and Alerting using Prometheus</title><link>https://silo.pgsty.com/operations/monitoring/collect-minio-metrics-using-prometheus/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://silo.pgsty.com/operations/monitoring/collect-minio-metrics-using-prometheus/</guid><description>&lt;p&gt;&lt;a id="monitoring-and-alerting-using-prometheus"&gt;&lt;/a&gt;
&lt;a id="minio-metrics-collect-using-prometheus"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://youtu.be/A3vCDaFWNNs?ref=docs"&gt;Monitoring with MinIO and Prometheus: Overview&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://youtu.be/Oix9iXndSUY?ref=docs"&gt;Monitoring with MinIO and Prometheus: Lab&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;MinIO publishes cluster, node, bucket, and resource metrics using the &lt;a href="https://prometheus.io/docs/concepts/data_model/#data-model"&gt;Prometheus Data Model&lt;/a&gt;. The procedure on this page documents the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Configuring a Prometheus service to scrape and display metrics from a MinIO deployment&lt;/li&gt;
&lt;li&gt;Configuring an Alert Rule on a MinIO Metric to trigger an AlertManager action&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These instructions use &lt;a href="https://silo.pgsty.com/operations/monitoring/metrics-v2/#minio-metrics-v2"&gt;version 2 metrics.&lt;/a&gt; For more about metrics API versions, see &lt;a href="https://silo.pgsty.com/operations/monitoring/metrics-and-alerts/#minio-metrics-and-alerts"&gt;Metrics and alerts.&lt;/a&gt;&lt;/p&gt;</description></item><item><title>Monitoring and Alerting using InfluxDB</title><link>https://silo.pgsty.com/operations/monitoring/monitor-and-alert-using-influxdb/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://silo.pgsty.com/operations/monitoring/monitor-and-alert-using-influxdb/</guid><description>&lt;p&gt;&lt;a id="monitoring-and-alerting-using-influxdb"&gt;&lt;/a&gt;
&lt;a id="minio-metrics-influxdb"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;MinIO publishes cluster and node metrics using the &lt;a href="https://prometheus.io/docs/concepts/data_model/"&gt;Prometheus Data Model&lt;/a&gt;. &lt;a href="https://www.influxdata.com/?ref=minio"&gt;InfluxDB&lt;/a&gt; supports scraping MinIO metrics data for monitoring and alerting.&lt;/p&gt;
&lt;p&gt;The procedure on this page documents the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Configuring an InfluxDB service to scrape and display metrics from a MinIO deployment&lt;/li&gt;
&lt;li&gt;Configuring an Alert on a MinIO metric&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="alert alert-info" role="alert"&gt;
&lt;p&gt;&lt;strong&gt;Prerequisites&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;This procedure requires the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;An existing InfluxDB deployment configured with one or more &lt;a href="https://docs.influxdata.com/influxdb/v2.4/notification-endpoints/"&gt;notification endpoints&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;An existing MinIO deployment with network access to the InfluxDB deployment&lt;/li&gt;
&lt;li&gt;An &lt;a href="https://silo.pgsty.com/reference/minio-mc/#command-mc"&gt;&lt;code&gt;mc&lt;/code&gt;&lt;/a&gt; installation on your local host configured to &lt;a href="https://silo.pgsty.com/reference/minio-mc/mc-alias-set/#alias"&gt;access&lt;/a&gt; the MinIO deployment&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These instructions use &lt;a href="https://silo.pgsty.com/operations/monitoring/metrics-v2/#minio-metrics-v2"&gt;version 2 metrics.&lt;/a&gt; For more about metrics API versions, see &lt;a href="https://silo.pgsty.com/operations/monitoring/metrics-and-alerts/#minio-metrics-and-alerts"&gt;Metrics and alerts.&lt;/a&gt;&lt;/p&gt;</description></item><item><title>Publish Server or Audit Logs to an External Service</title><link>https://silo.pgsty.com/operations/monitoring/minio-logging/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://silo.pgsty.com/operations/monitoring/minio-logging/</guid><description>&lt;p&gt;&lt;a id="publish-server-or-audit-logs-to-an-external-service"&gt;&lt;/a&gt;
&lt;a id="minio-logging"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;MinIO publishes all &lt;a href="https://silo.pgsty.com/reference/minio-server/#command-minio.server"&gt;&lt;code&gt;minio server&lt;/code&gt;&lt;/a&gt; operations to the system console. Reading these logs depends on how the server process is managed. For example, if the server is managed through a &lt;code&gt;systemd&lt;/code&gt; script, you can read the logs using &lt;code&gt;journalctl -u SERVICENAME.service&lt;/code&gt;. Replace &lt;code&gt;SERVICENAME&lt;/code&gt; with the name of the MinIO service.&lt;/p&gt;
&lt;p&gt;MinIO also supports publishing server logs and audit logs to an HTTP webhook.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://silo.pgsty.com/operations/monitoring/minio-logging/#minio-logging-publish-server-logs"&gt;Server logs&lt;/a&gt; contain the same &lt;a href="https://silo.pgsty.com/reference/minio-server/#command-minio.server"&gt;&lt;code&gt;minio server&lt;/code&gt;&lt;/a&gt; operations logged to the system console. Server logs support general monitoring and troubleshooting of operations.&lt;/li&gt;
&lt;li&gt;&lt;a href="https://silo.pgsty.com/operations/monitoring/minio-logging/#minio-logging-publish-audit-logs"&gt;Audit logs&lt;/a&gt; are more granular descriptions of each operation on the MinIO deployment. Audit logging supports security standards and regulations which require detailed tracking of operations.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;MinIO publishes logs as a JSON document as a &lt;code&gt;PUT&lt;/code&gt; request to each configured endpoint. The endpoint server is responsible for processing each JSON document. MinIO requires explicit configuration of each webhook endpoint and does &lt;em&gt;not&lt;/em&gt; publish logs to a webhook by default.&lt;/p&gt;</description></item><item><title>Healthcheck API</title><link>https://silo.pgsty.com/operations/monitoring/healthcheck-probe/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://silo.pgsty.com/operations/monitoring/healthcheck-probe/</guid><description>&lt;p&gt;&lt;a id="healthcheck-api"&gt;&lt;/a&gt;
&lt;a id="minio-healthcheck-api"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;MinIO exposes unauthenticated endpoints for probing node uptime and cluster &lt;a href="https://silo.pgsty.com/operations/concepts/erasure-coding/#minio-ec-parity"&gt;high availability&lt;/a&gt; for simple healthchecks. These endpoints return an HTTP status code indicating whether the underlying resource is healthy or satisfies read/write quorum. MinIO exposes no other data through these endpoints.&lt;/p&gt;
&lt;h2 id="node-liveness"&gt;Node Liveness&lt;/h2&gt;
&lt;p&gt;Use the following endpoint to test if a MinIO server is online:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-shell" data-lang="shell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl -I https://minio.example.net:9000/minio/health/live
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Replace &lt;code&gt;https://minio.example.net:9000&lt;/code&gt; with the DNS hostname of the MinIO server to check.&lt;/p&gt;</description></item><item><title>Metrics version 2</title><link>https://silo.pgsty.com/operations/monitoring/metrics-v2/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://silo.pgsty.com/operations/monitoring/metrics-v2/</guid><description>&lt;p&gt;&lt;a id="metrics-version-2"&gt;&lt;/a&gt;
&lt;a id="minio-metrics-v2"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;MinIO publishes cluster and node metrics using the &lt;a href="https://prometheus.io/docs/concepts/data_model/"&gt;Prometheus Data Model&lt;/a&gt;. You can use any scraping tool to pull metrics data from MinIO for further analysis and alerting.&lt;/p&gt;
&lt;h2 id="version-2-endpoints"&gt;Version 2 Endpoints&lt;/h2&gt;
&lt;p&gt;Metrics version 2 provides metrics organized into three categories:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://silo.pgsty.com/operations/monitoring/metrics-v2/#minio-available-cluster-metrics"&gt;Cluster Metrics&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://silo.pgsty.com/operations/monitoring/metrics-v2/#minio-available-bucket-metrics"&gt;Bucket Metrics&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://silo.pgsty.com/operations/monitoring/metrics-v2/#minio-available-resource-metrics"&gt;Resource Metrics&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Each v2 endpoint returns all metrics for its category. For example, scraping the following endpoint returns all cluster metrics:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-shell" data-lang="shell"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;http://HOSTNAME:PORT/minio/v2/metrics/cluster
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The base endpoint alone, &lt;code&gt;/minio/v2/metrics/&lt;/code&gt;, returns cluster metrics.&lt;/p&gt;</description></item><item><title>Monitor a MinIO Server with Grafana</title><link>https://silo.pgsty.com/operations/monitoring/grafana/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://silo.pgsty.com/operations/monitoring/grafana/</guid><description>&lt;p&gt;&lt;a id="monitor-a-minio-server-with-grafana"&gt;&lt;/a&gt;
&lt;a id="minio-grafana"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://grafana.com/"&gt;Grafana&lt;/a&gt; allows you to query, visualize, alert on and understand your metrics no matter where they are stored.&lt;/p&gt;
&lt;h2 id="prerequisites"&gt;Prerequisites&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;An existing &lt;a href="https://prometheus.io/docs/prometheus/latest/installation/"&gt;Prometheus deployment&lt;/a&gt; with backing &lt;a href="https://prometheus.io/docs/alerting/latest/overview/"&gt;Alert Manager&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;An existing MinIO deployment with network access to the Prometheus deployment&lt;/li&gt;
&lt;li&gt;&lt;a href="https://grafana.com/grafana/download"&gt;Grafana installed&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="alert alert-info" role="alert"&gt;
&lt;p&gt;&lt;strong&gt;Grafana dashboards use metrics version 2&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The MinIO Grafana dashboards use &lt;a href="https://silo.pgsty.com/operations/monitoring/metrics-v2/#minio-metrics-v2"&gt;metrics version 2&lt;/a&gt;. For more about metrics API versions, see &lt;a href="https://silo.pgsty.com/operations/monitoring/metrics-and-alerts/#minio-metrics-and-alerts"&gt;Metrics and alerts.&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Version 3 metrics require creating your own dashboard. For more information about dashboards, see &lt;a href="https://grafana.com/docs/grafana/latest/dashboards/"&gt;the Grafana documentation.&lt;/a&gt;&lt;/p&gt;</description></item></channel></rss>