Monitoring বনাম Observability
| Monitoring | Observability |
|---|---|
| Known unknowns | Unknown unknowns |
| Predefined dashboards | Ad-hoc investigation |
| "Is it working?" | "Why is it not working?" |
3 Pillars of Observability ⭐
- Metrics — numbers over time (CPU, RPS)
- Logs — discrete events
- Traces — request journey through system
Metrics — Prometheus ⭐
# Scrape config
scrape_configs:
- job_name: 'myapp'
static_configs:
- targets: ['app:3000']
# App exports metrics (Prometheus format)
http_requests_total{method="GET",path="/api/users"} 1234
http_request_duration_seconds{path="/api/users"} 0.05
Metric Types
- Counter — only increase (requests total)
- Gauge — up/down (memory use)
- Histogram — bucketed distribution (latency)
- Summary — quantile (p50, p99)
PromQL
# Rate per second
rate(http_requests_total[5m])
# p99 latency
histogram_quantile(0.99,
rate(http_duration_seconds_bucket[5m]))
# Error rate
sum(rate(http_requests_total{status=~"5.."}[5m]))
/ sum(rate(http_requests_total[5m]))
Grafana
Prometheus (ও অন্যান্য) থেকে dashboard তৈরি। Alerting built-in।
Golden Signals (Google SRE)
- Latency (request time)
- Traffic (RPS)
- Errors (rate)
- Saturation (resource utilization)
USE Method
Utilization, Saturation, Errors — per resource (CPU, memory, disk, network)।
RED Method
Rate, Errors, Duration — per service।
Logging — ELK Stack ⭐
App → Logstash/Filebeat → Elasticsearch → Kibana
// Structured logging (JSON)
{
"ts": "2024-01-15T10:30:00Z",
"level": "ERROR",
"service": "order-api",
"trace_id": "abc123",
"user_id": 42,
"msg": "Payment failed",
"err": "..."
}
Structured Logging Best
- JSON format (searchable)
- Correlation ID / Trace ID
- Context (user, request, service)
- Appropriate level (DEBUG/INFO/WARN/ERROR)
- No PII / secrets in logs
- Centralize (ELK, Loki, Datadog, Splunk)
Log Levels
- DEBUG — dev only
- INFO — normal operations
- WARN — recoverable issue
- ERROR — failure affecting user
- FATAL — app crashing
Distributed Tracing ⭐
Microservices-এ request কোন path-এ গেছে। Service-by-service latency।
Request → Gateway (50ms)
↓
Auth (20ms)
↓
Order (100ms)
↓
Payment (200ms)
↓
DB (150ms)
Total: 520ms
Tools
Jaeger, Zipkin, AWS X-Ray, Datadog APM, New Relic, Dynatrace, Honeycomb।
OpenTelemetry ⭐
Vendor-neutral instrumentation standard। Trace + metric + log।
import { trace } from '@opentelemetry/api';
const tracer = trace.getTracer('my-app');
const span = tracer.startSpan('processOrder');
span.setAttribute('order.id', orderId);
try {
await process();
} finally {
span.end();
}
SLI / SLO / SLA ⭐
| Term | অর্থ |
|---|---|
| SLI (Indicator) | Measured metric (latency p99) |
| SLO (Objective) | Internal target (99.9% uptime) |
| SLA (Agreement) | Customer contract (99.5% or refund) |
Error Budget
SLO: 99.9% uptime Monthly budget: 43 minutes downtime Burn rate — গতি অনুযায়ী alert
Alerting Strategy
- Alert on symptoms, not causes
- Actionable (clear runbook)
- Multi-level (page, Slack, email)
- Escalation (on-call rotation)
- Avoid alert fatigue
- Runbook per alert
Alert Tools
PagerDuty, Opsgenie, VictorOps, Alertmanager (Prometheus)।
Dashboards
- Service overview (golden signals)
- Business metrics (orders/min, revenue)
- Infrastructure (nodes, k8s cluster)
- Dependency health
APM Tools (All-in-one)
Datadog, New Relic, Dynatrace, AppDynamics, Elastic APM।