Monitor Kubernetes Logs, Metrics, and Metadata with ELK and Beats Part -3

 Installing MetricBeat Agent

Metricbeat helps you monitor your servers and the services they host by collecting metrics from the operating system and services.




You deploy Metricbeat as a DaemonSet to ensure that there’s a running instance on each node of the cluster. These instances are used to retrieve most metrics from the host, such as system metrics, Docker stats, and metrics from all the services running on top of Kubernetes.

Kube-state-metrics

kube-state-metrics is a simple service that listens to the Kubernetes API server and generates metrics about the state of the objects. It is not focused on the health of the individual Kubernetes components, but rather on the health of the various objects inside, such as deployments, nodes and pods.

Download the yaml file from my git repository.


Run the following command to execute "kube-state-metrics".

oc create -f kube-state-metrics.yaml

After install the Kube-state-metrics , Logstash , FileBeat , Logstash ,Elasticsearch and Kibana will show on the Openshift Topology.


Monitor OpenShift with Elastic Observability

Now that we have Filebeat shipping OpenShift logs to Elasticsearch, we can open up Kibana and start monitoring system health from Elastic Observability.

1. Create Index Patten 
Login to kibana using elastic username and password. Then Navigate to the Stack Management under Kibana section select Index Patterns.

If we able to successfully deployed Filebeat and Logstash, it should show index as shown below. Type the Index pattern name same as index and click Next step.


After Create it will show the lists of every field in the logstash-filebeat* index and the field's associated core type as recorded by Elasticsearch. 



2. Discover Logs
Then Navigate to the Analytics then Discover.


Add Filter for Kubernetes module to extracts the logs . 


Add a filter for the event. 


Then expand the one log record add the necessary fields for more information.
for example , "kubernetes.event.metadata.name" , "kubernetes.event.metadata.namespace" ,"kubernetes.even.message" etc.


You can save the Queries by click the save button on left top.

Deploy any kind of sample application to your project using S2i or any other option. In here I am using source to image option to deploy the application. This is a .Net core sample application ,you can see the source code from my git repository (link). 

After successfully deployed your application, navigate to the elastic search and set the time "20 secounds" and Refresh the logs. Once there is no logs ,try to scale up the application pod once scale up at the same time Refresh the logs.


You can see how the pods are assign on the Kubernetes.

Same way scale down the application pods.
 


Delete the Elasticserarch Index.

Navigate to the Stack Management => Index Management and delete the indices.

Enjoy!!! stay safe.




Comments

Post a Comment

Popular posts from this blog

Implement an event bus on Kubernetes with RabbitMQ using MassTransit in Microservice Architecture Part - 2

Image
Event-driven microservice architecture, services communicate each-other using event messages. When business events occur, producers publish them with messages.  At the same time, other services consume them through event listeners. The main benefits of event-driven systems are asynchronous behavior and loosely coupled structures. In this article, we are going to talk about a simple publisher/subscriber scenario deployed on Kubernetes cluster. To impalement publisher and subscriber we use Masstransit. In this tutorial, we will use Red Hat® OpenShift® version 4.6  Kubernetes container platform to deploy RabbitMQ cluster , publisher and consumer application.  Prerequisites You should have a basic understanding of RabbitMQ, Docker, Kubernetes, Openshift, ASP.NET Core WebAPI, and .NET Core. Read following articles before start. Openshift Architecture https://darshanadinushal.blogspot.com/2021/02/openshift-with-kubernetes-openshift-v4.html Apache Kafka and RabbitMQ https://darshanadinushal.b
Read more

Provision Red Hat OpenShift Cluster On AWS

Image
OpenShift Red Hat OpenShift Container Platform unites developers and IT operations on a single platform to build, deploy, and manage applications consistently across hybrid cloud and multi-cloud infrastructures. OpenShift Container Platform — platform as a service built around docker containers orchestrated and managed by Kubernetes on a foundation of Red Hat Enterprise Linux. OpenShift helps businesses achieve greater value by delivering modern and traditional applications with shorter development cycles and lower operating costs. For more OpenShift Architecture read my previous blog post. OpenShift On AWS Architecture Openshift Kubernetes Engine comes with different installation on different platforms like AWS, Azure, Google Cloud, Bare-metal and Vmware. In this post I have simplified each and every steps of the process provision OpenShift Container Platform 4.7 on AWS Infrastructure from scratch. In this article we are going to provision Three Master and Three worker node cluster on
Read more

Redis Enterprise on Openshift to Manage Distributed Chase in Microservices

Image
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. In this blog we describe how to use Radis chase to manage distributed chase between microservices. Install Redis Enterprise Clusters Using Operators on Openshift Redis Enterprise Software is distributed, in-memory NoSQL database server. Redis Enterprise Software extends open source Redis and delivers stable high performance, zero-downtime linear scaling and high availability cluster. Prerequisites A minimum of 3 nodes which support the following requirements: RAM - At least 3GB with 4GB recommended Persistent Storage - At least 10GB of free space with 20GB recommended. A kubernetes version of 1.9 or higher oc adm policy add-scc-to-group redis-enterprise-scc system:serviceaccounts:my-project Install the Redis Operator Log in to your Openshift account as a cluster admin or super admin (so you have access to all the default projects in openshift). Create a new project, fil
Read more