Imagine managing a fleet of airplanes. You wouldn’t let them fly without dashboards showing speed, altitude, and fuel levels. Similarly, in Docker Swarm, monitoring and logging are the dashboards of your cluster - they help you track container health, resource usage, and application logs to ensure smooth operations.

Observability Foundations

1. Why Monitoring and Logging Matter

• Monitoring: Tracks metrics like CPU, memory, network, and container health.
• Logging: Captures application and system events for debugging and auditing.
• Visibility: Without monitoring and logging, issues remain hidden until failure.

2. Monitoring in Swarm

• Third‑Party Tools:
  • Prometheus + Grafana: Metrics collection and visualization.
  • cAdvisor: Container resource monitoring.
  • ELK Stack (Elasticsearch, Logstash, Kibana): Centralized logging and analytics.

Service Inspection:

docker service ps web

Docker Stats: Built‑in command for resource usage.

docker stats

3. Logging in Swarm

• Centralized Logging:
  • Use ELK stack or Fluentd to aggregate logs across nodes.
  • Ensures logs are searchable and persistent.

Service Logs: View logs for services.

docker service logs web

Docker Logs: View logs for individual containers.

docker logs container_id

4. Best Practices

• Use centralized monitoring and logging for clusters.
• Set up alerts for critical metrics (CPU spikes, memory leaks).
• Retain logs for auditing and compliance.
• Monitor both system metrics (nodes, networks) and application metrics (response times, errors).

Things to Remember

• Monitoring and logging are the eyes and ears of a Swarm cluster.
• Docker provides basic commands, but production requires advanced tools.
• Centralized solutions like Prometheus and ELK are industry standards.

Hands‑On Lab

Step 1: Monitor Resource Usage

docker stats

Step 2: View Service Logs

docker service logs web

Step 3: Deploy Prometheus + Grafana in Swarm

version: '3'
services:
  prometheus:
    image: prom/prometheus
    ports:
      - "9090:9090"
  grafana:
    image: grafana/grafana
    ports:
      - "3000:3000"

Step 4: Deploy ELK Stack for Logging

version: '3'
services:
  elasticsearch:
    image: elasticsearch:7.9.2
    ports:
      - "9200:9200"
  kibana:
    image: kibana:7.9.2
    ports:
      - "5601:5601"

Practice Exercise

1. Deploy a service frontend with 3 replicas.
2. Use docker service logs frontend to view logs.
3. Deploy Prometheus and Grafana in the cluster.
4. Create a dashboard showing CPU and memory usage of frontend.
5. Reflect on how monitoring and logging improve reliability.

Visual Learning Model

Swarm Monitoring & Logging
   ├── Docker Stats → basic metrics
   ├── Service Logs → container/application logs
   ├── Prometheus + Grafana → metrics visualization
   └── ELK Stack → centralized logging & analytics

The Hackers Notebook

Swarm monitoring and logging provide visibility into cluster health and application performance. Docker offers basic commands (stats, logs), but production environments rely on centralized solutions like Prometheus, Grafana, and ELK. With proper monitoring and logging, Swarm clusters become reliable, auditable, and easier to manage.