Monitoring Application Logs with ELK on Kubernetes (Nginx)
Monitoring logs is crucial for any app. This setup uses Nginx as a sample app, Filebeat to collect logs, Logstash to process them, Elasticsearch to store, and Kibana to visualize all on Docker Desktop Kubernetes.
Accessing App and Kibana
-
Ingress: Access Nginx at
http://elk.local/app-1and Kibana athttp://elk.local/kibana. -
Port-forward:
kubectl port-forward svc/app-1-service 8080:80 -n elk-demo1 kubectl port-forward svc/kibana 5601:5601 -n elk-demo1
ELK Components
-
Nginx: Serves requests, generates logs in
/var/log/nginx.kubectl get pods -n elk-demo1 kubectl logs <nginx-pod> -n elk-demo1 -
Filebeat: Collects container logs and forwards to Logstash.
kubectl logs <filebeat-pod> -n elk-demo1 -
Logstash: Parses and forwards logs to Elasticsearch.
kubectl logs deployment/logstash -n elk-demo1 -
Elasticsearch: Stores logs; check indices:
curl http://localhost:9200/_cat/indices?v -
Kibana: Visualizes logs; check pod logs if issues arise:
kubectl logs deployment/kibana -n elk-demo1
Log Flow
- Nginx writes logs 2. Filebeat collects 3. Logstash parses 4. Elasticsearch stores 5. Kibana visualizes.
Troubleshooting
- Port-forward fails: Ensure pods are
Running. - No logs in Kibana: Verify indices in Elasticsearch and Filebeat config.
- Filebeat not sending: Check file paths and permissions.
Key Commands
kubectl get pods -n elk-demo1
kubectl logs <pod> -n elk-demo1
kubectl port-forward svc/<service> <local>:<target> -n elk-demo1
curl http://localhost:9200/_cat/indices?v
Summary: Nginx logs are collected by Filebeat, processed by Logstash, stored in Elasticsearch, and visualized via Kibana. Access via Ingress or port-forwarding. This lightweight stack is ideal for learning ELK
