Monitor Kubernetes Logs, Metrics, and Metadata with ELK and Beats Part -2

 Installing Filebeat Agent

Filebeat is the agent that we are going to use to ship logs to Logstash.



The configMap that holds the Filebeat configuration, we specify output.logstash:hosts: '${LOGSTASH_URL}' that it needs to ship the log data to Logstash. We’re specifying the Service short URL since both resources live in the same namespace.

We are using a DaemonSet for this deployment. A DaemonSet ensures that an instance of the Pod is running each node in the cluster.
To deploy Filebeat, we need to create a service account, a cluster role, and a cluster role binding.

After install the FileBeat , Logstash ,Elasticsearch and Kibana will show on the Openshift Topology.

Monitor OpenShift with Elastic Observability

Now that we have Filebeat shipping OpenShift logs to Elasticsearch, we can open up Kibana and start monitoring system health from Elastic Observability.

1. Create Index Patten 
Login to kibana using elastic username and password. Then Navigate to the Stack Management under Kibana section select Index Patterns.


If we able to successfully deployed Filebeat and Logstash, it should show index as shown below. Type the Index pattern name same as index and click Next step.

















Select the time field dropdown @timestamp and click the Create index pattern button ,shown as below. 


After Create it will show the lists of every field in the logstash-filebeat* index and the field's associated core type as recorded by Elasticsearch. 

2. Discover Logs
Then Navigate to the Analytics then Discover.







Comments

Post a Comment

Popular posts from this blog

Implement an event bus on Kubernetes with RabbitMQ using MassTransit in Microservice Architecture Part - 2

Image
Event-driven microservice architecture, services communicate each-other using event messages. When business events occur, producers publish them with messages.  At the same time, other services consume them through event listeners. The main benefits of event-driven systems are asynchronous behavior and loosely coupled structures. In this article, we are going to talk about a simple publisher/subscriber scenario deployed on Kubernetes cluster. To impalement publisher and subscriber we use Masstransit. In this tutorial, we will use Red Hat® OpenShift® version 4.6  Kubernetes container platform to deploy RabbitMQ cluster , publisher and consumer application.  Prerequisites You should have a basic understanding of RabbitMQ, Docker, Kubernetes, Openshift, ASP.NET Core WebAPI, and .NET Core. Read following articles before start. Openshift Architecture https://darshanadinushal.blogspot.com/2021/02/openshift-with-kubernetes-openshift-v4.html Apache Kafka and RabbitMQ https://darshanadinushal.b
Read more

Provision Red Hat OpenShift Cluster On AWS

Image
OpenShift Red Hat OpenShift Container Platform unites developers and IT operations on a single platform to build, deploy, and manage applications consistently across hybrid cloud and multi-cloud infrastructures. OpenShift Container Platform — platform as a service built around docker containers orchestrated and managed by Kubernetes on a foundation of Red Hat Enterprise Linux. OpenShift helps businesses achieve greater value by delivering modern and traditional applications with shorter development cycles and lower operating costs. For more OpenShift Architecture read my previous blog post. OpenShift On AWS Architecture Openshift Kubernetes Engine comes with different installation on different platforms like AWS, Azure, Google Cloud, Bare-metal and Vmware. In this post I have simplified each and every steps of the process provision OpenShift Container Platform 4.7 on AWS Infrastructure from scratch. In this article we are going to provision Three Master and Three worker node cluster on
Read more

Redis Enterprise on Openshift to Manage Distributed Chase in Microservices

Image
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. In this blog we describe how to use Radis chase to manage distributed chase between microservices. Install Redis Enterprise Clusters Using Operators on Openshift Redis Enterprise Software is distributed, in-memory NoSQL database server. Redis Enterprise Software extends open source Redis and delivers stable high performance, zero-downtime linear scaling and high availability cluster. Prerequisites A minimum of 3 nodes which support the following requirements: RAM - At least 3GB with 4GB recommended Persistent Storage - At least 10GB of free space with 20GB recommended. A kubernetes version of 1.9 or higher oc adm policy add-scc-to-group redis-enterprise-scc system:serviceaccounts:my-project Install the Redis Operator Log in to your Openshift account as a cluster admin or super admin (so you have access to all the default projects in openshift). Create a new project, fil
Read more