DevOps Monitoring and Logging Syllabus

What is Prometheus and what model does it use to collect metrics?

Prometheus is an open-source, CNCF monitoring and alerting toolkit that stores time-series data. It uses a pull (scrape) model, periodically scraping metrics over HTTP from instrumented targets/exporters at configured endpoints.

What query language does Prometheus use?

PromQL (Prometheus Query Language), used to select and aggregate time-series data for graphs, tables, and alerting rules.

What are the four core Prometheus metric types?

Counter (monotonically increasing), Gauge (can go up or down), Histogram (samples bucketed with cumulative counts plus sum and count), and Summary (client-side quantiles with sum and count).

What is a Prometheus exporter, and give a common example?

An exporter is a sidecar/agent that exposes metrics from a system in Prometheus format so Prometheus can scrape them. A common example is Node Exporter, which exposes host/OS metrics (CPU, memory, disk, network).

What is the role of Alertmanager in the Prometheus ecosystem?

Alertmanager receives alerts fired by Prometheus rules and handles deduplication, grouping, silencing, inhibition, and routing notifications to receivers like email, Slack, or PagerDuty.

How is the per-second rate of a Prometheus counter computed in PromQL?

With the rate() function over a range vector, e.g. rate(http_requests_total[5m]), which gives the average per-second increase of the counter over the time window.

What is Grafana primarily used for?

Grafana is an open-source analytics and visualization platform used to query, visualize, alert on, and explore metrics, logs, and traces through customizable interactive dashboards, supporting many data sources.

Name several data sources Grafana can connect to.

Prometheus, Graphite, InfluxDB, Elasticsearch, Loki, MySQL/PostgreSQL, CloudWatch, and many others; Grafana is data-source agnostic.

How do Grafana and Prometheus typically work together?

Prometheus collects and stores time-series metrics and serves them via PromQL; Grafana connects to Prometheus as a data source to build dashboards and visualizations, and can also handle alerting on top of those metrics.

What is Grafana Loki?

Loki is Grafana's horizontally scalable log aggregation system inspired by Prometheus. It indexes only metadata labels (not full log content), making it cost-efficient, and is queried with LogQL.

What is Nagios and what kind of monitoring is it known for?

Nagios is a long-established open-source monitoring tool for infrastructure, networks, hosts, and services. It is known for host/service up-down (availability) checks and alerting, traditionally using a polling, agent-based or agentless plugin model.

How does Nagios extend its monitoring capabilities?

Through plugins—external executable check scripts/programs that test a host or service and return a status code, which Nagios interprets. NRPE (Nagios Remote Plugin Executor) allows running checks on remote hosts.

What status states does a Nagios service check return?

OK (0), WARNING (1), CRITICAL (2), and UNKNOWN (3).

What is Datadog?

Datadog is a commercial, cloud-based SaaS monitoring and observability platform that unifies metrics, traces, and logs (plus APM, security, and synthetics) across infrastructure and applications, collected primarily via the Datadog Agent.

What is the Datadog Agent and how does it differ from the Prometheus model?

The Datadog Agent is software installed on hosts/containers that collects metrics, traces, and logs and pushes them to Datadog's SaaS backend. Unlike Prometheus's pull model, Datadog's agent uses a push model to a hosted service.

What does APM (Application Performance Monitoring) provide, as offered by tools like Datadog?

APM provides distributed tracing and code-level visibility into application requests—latency breakdowns per service/endpoint, error tracking, and bottleneck identification across a microservices request path.

What is logging in software systems?

Logging is the practice of recording timestamped, discrete event records emitted by applications and infrastructure, capturing what happened (messages, errors, state, context) for debugging, auditing, and analysis.

What are common log severity levels from least to most severe?

TRACE, DEBUG, INFO, WARN, ERROR, FATAL (with some systems adding NOTICE/CRITICAL/ALERT/EMERGENCY as in syslog).

What is structured logging and why is it valuable?

Structured logging emits logs in a consistent, machine-parseable format (e.g., JSON) with key-value fields rather than free text. It is valuable because it enables reliable parsing, searching, filtering, and aggregation.

What is log aggregation?

Log aggregation is the process of collecting logs from many distributed sources into a centralized system where they can be stored, indexed, searched, and analyzed together.

What is the difference between logs, metrics, and traces?

Logs are discrete timestamped event records (high detail, high volume); metrics are aggregated numeric measurements over time (low cost, trend-focused); traces follow a single request's path across multiple services to show end-to-end flow and latency.

What is log retention and why does it matter?

Log retention is the policy defining how long logs are kept before deletion or archival. It matters for balancing storage cost against debugging needs, compliance/audit requirements, and security investigations.

List several benefits of logging.

Faster debugging and root-cause analysis, audit trails and compliance, security and intrusion detection, understanding user behavior, post-incident forensics, and correlating events across distributed systems.

What is log correlation and how is it commonly enabled across services?

Log correlation links related log entries from different services/components for a single request or transaction. It is commonly enabled by propagating a shared correlation ID or trace ID across service calls.

What does the acronym ELK Stack stand for, and what is each component's role?

Elasticsearch (search and analytics engine that stores and indexes data), Logstash (server-side data processing pipeline that ingests, transforms, and ships data), and Kibana (visualization and exploration UI for Elasticsearch data).

What is the Elastic Stack and how does Beats fit in?

The Elastic Stack is the ELK Stack plus Beats. Beats are lightweight single-purpose data shippers installed on edge hosts (e.g., Filebeat for logs, Metricbeat for metrics) that forward data to Logstash or directly to Elasticsearch.

In the ELK pipeline, what is the typical data flow?

Data is collected (often by Beats), sent to Logstash for parsing/filtering/enrichment (or ingested directly), stored and indexed in Elasticsearch, and then visualized and queried in Kibana.

What underlying technology powers Elasticsearch's fast full-text search?

Elasticsearch is built on Apache Lucene and uses an inverted index, which maps terms to the documents containing them, enabling fast full-text search.

What is Splunk and how does it differ from the ELK Stack?

Splunk is a commercial, proprietary platform for searching, monitoring, and analyzing machine-generated data/logs, queried with its SPL language. It differs from the open-source ELK Stack mainly by being a licensed commercial product (typically priced by data volume ingested) with enterprise support out of the box.

What query language does Splunk use, and what is a defining feature of it?

Splunk uses SPL (Search Processing Language). A defining feature is the pipe (|) syntax that chains search and transformation commands, similar to Unix pipelines, to filter and process events.

What is Graylog and what backend does it typically rely on?

Graylog is an open-source centralized log management platform for collecting, indexing, and analyzing logs. It typically relies on Elasticsearch/OpenSearch for log storage and search and MongoDB for configuration and metadata, ingesting logs via inputs such as the GELF format and Syslog.