RabbitMQ Architecture
RabbitMQ is one of the most widely used open-source message brokers. It was originally based on the Advanced Message Queuing Protocol (AMQP).In the RabbitMQ architecture main components are Producer ,Exchange ,Queue and Consumer application.
In here we will take a use case example to understand better how RabbitMQ components are work each other.
Use Case: In this system when user create an doctor appointment ,system should be able to trigger an email to the relevant parties. In this cluster lets say we have couple of queues patient-notification(notify to the patient) , doctor-notification(notify to the doctor) , admin-notification(notify to the hospital administration).
Producer:
Producing means nothing more than sending a messages .A producer pushes messages to exchanges.
Ex: Producer is a web base application, once user create an appointment ,producer send a message to queue ,In this message object include email details.
Exchange
It is basically a routing rule for the messages. Messages are not published directly to a queue; instead, the producer sends messages to an exchange. An exchange is responsible for routing the messages to different queues with the help of bindings and routing keys. A binding is a link between a queue and an exchange. There are four types of Exchanges.
Direct: The message is routed to the queues whose binding key exactly matches the routing key of the message. Ex: If the queue is bound to the exchange with the binding key patient-notification, a message published to the exchange with a routing key patient-notification is routed to that queue. It will send email to patient.
Fanout: A fanout exchange routes messages to all of the queues bound to it. Ex: When the user create an appointment ,It will route to all the queues ,that means send an email to patient ,doctor and hospital administration.
Topic: The topic exchange does a wildcard match between the routing key and the routing pattern specified in the binding. Ex: If the user cancel the appointment ,system only needs to send an email to doctor and hospital administration.so message push to doctor-notification ,admin-notification.
Headers: Headers exchanges use the message header attributes for routing.
Queue
A queue is a line of things waiting to be handled, starting at the beginning of the line and processing it in sequential order.
A message queue is a queue of messages sent between applications. It includes a sequence of work objects that are waiting to be processed.
A message is the data transported between the sender and the receiver application; it's essentially a byte array with some headers at the top.
The basic architecture of a message queue is simple; there are client applications called producers that create messages and deliver them to the message queue. Another application, called a consumer, connects to the queue and gets the messages to be processed. Messages placed onto the queue are stored until the consumer retrieves them. Once consumer consumed the message successfully ,consumer send ack(acknowledgment) to the queue ,then RabbitMQ will delete the message from the queue.
A message queue provides an asynchronous communications protocol, which is a system that puts a message onto a message queue and does not require an immediate response to continuing processing. Message Queue contains FIFO (first in first out) if there is no priority queue.
Consumer
Consumer also known as a subscriber. It reads messages from the queues and send the ack to the RabbitMQ.
Ex: In our use case consumer is application , it will get the email message object from the queue and process the object check the validation of the email object ,send a email to the end user.
|
|
|
|
Basic
|
RabbitMQ
is solid , mature ,general purpose message broker. It support Publish/
Subscribe and asynchronous processing.
|
|
Price
|
open
source; it's free to use.
|
|
Primary
use
|
A
common use case for RabbitMQ is to handle background jobs or long-running
task, such as file scanning, image scaling or PDF conversion.
System
simply needs to notify another part of the system (send an email/SMS
notification)
|
|
Message
handling
|
RabbitMQ,
messages are stored until a receiving application connects and receives a
message off the queue.
RabbitMQ
use the RAM to store the message , when the message queue does not have any
event/message it can be fast.(Openshift/Kubernetes use Persistent Storage)
|
|
Protocol
|
It
supports several protocols such as AMQP, MQTT, STOMP
|
|
Scaling
|
RabbitMQ
is mostly designed for vertical scaling (scale by adding more power ,CPU).
|
|
Routing
|
RabbitMQ has route messages in
complex ways to the consumers.
direct, topic, fanout, and header
exchanges
|
|
Monitor
|
RabbitMQ has a built-in user-friendly
interface that lets you monitor and handle your RabbitMQ server from a web
browser.
|
|
|
|
Kafka Architecture
Kafka is an open-source event streaming platform first developed by LinkedIn, then donated to the Apache software foundation. Apache Kafka aims to provide the user with a unified, high latency, high throughput platform for handling real-time data.
As we talk about running Kafka on Kubernetes - Kafka runs as a cluster of nodes called Kafka brokers. Kafka was developed first as a messaging queue and works as a pub-sub model. It’s used as a popular message queue for distributed systems, and is commonly used to stream data in the Internet of Things use cases.
|
|
|
|
Basic
|
Apache
Kafka is a message bus optimized for high-ingress data streams and replay. It
is event streaming platform.
Publish
/ subscribe and asynchronous processing
|
|
Price
|
open
source; it's free to use.
|
|
Primary
use
|
Use
Kafka when we need to move a large amount of data, process data in real-time
or analyze data over a time period.
An
example is when we want to track user activity on a web-shop and generate
suggested items to buy.
|
|
Message
handling
|
The
message queue in Kafka is persistent. Kafka actually stores all of its
records to disk and does not keep anything in RAM.
That
mean message is not removed once it’s consumed ,It can be replayed or
consumed multiple times.
|
|
Protocol
|
Kafka
uses a custom protocol on top of TCP/IP
|
|
Scaling
|
Kafka
is built from the ground up with horizontal scaling (scale by adding more
machines)
|
|
Routing
|
Kafa has a very simple routing
approach
|
|
Monitor
|
Kafka does not have build in tool to
monitor.
|
Comments
Post a Comment