Working With RabbitMQ Unacked Messages Simplified 101

Companies are making a shift towards the adoption of microservice-based architecture for modern applications. Implementing decoupled modules within the application is necessary for smooth and uninterrupted running no matter how much volumes of data is transferred. Message brokers allow applications to communicate and decouple from each other.

RabbitMQ is a widely used open-source message broker that helps in scaling the application by deploying a message queuing mechanism in between the two applications. It offers temporary storage for data preventing data loss. When a message gets transferred from publisher to consumer, the publisher gets a confirmation that the message is successfully delivered. But what happens when this is not the case? What if a publisher doesn’t get acknowledgment? These messages are unacknowledged. The RabbitMQ Unacked Messages get piled up in the queue which creates difficulty in a later stage.

RabbitMQ works as an intermediary platform that ensures the message is delivered to the right destination. In this article, you will learn about RabbitMQ, its basic terminologies, how to remove RabbitMQ Unacked Messages, and read about what’s the cause of RabbitMQ Unacked Messages.

What is RabbitMQ?

RabbitMQ is an open-source message broker service provider software that is written in the Erlang programming language. It is the first one to implement the AQMP (Advanced Message Queuing Protocol). RabbitMQ is commonly called message-oriented middleware because it acts as a medium between publish or message provide and consumer or message consumer. RabbitMQ supports many programming languages and can run on various Cloud environments and operating systems. It comes with an extended plug-in architecture to deliver support for MQ Telemetry Transport (MQTT), Streaming Text Oriented Messaging Protocol (STOMP), and other protocols.

RabbitMQ offers both Command-line tools and web-based GUI for monitoring and managing operations. It’s lightweight and easy to deploy on Cloud and premises. It’s known for scalability and high availability because it deploys distributed and federated configurations to meet business requirements.

Basic concepts of RabbitMQ:

• Queue: A Queue is a sequential data structure that is a medium through which messages are transferred and stored.
• Producer or Publisher: It is the one who sends messages to a queue.
• Consumer: A consumer is the one who subscribes to and receives messages from the broker and uses that message for other defined operations.
• Broker: It’s a message broker that provides storage space for storing data. The broker act as middleware where data is consumed or received by another application connecting with the broker.
• Exchange: It’s an entry point for the broker as it receives messages from the producer and routes them to the appropriate queue.
• Channel: It’s a lightweight connection to a broker via a shared TCP connection.

Key Features of RabbitMQ

Some of the main features of RabbitMQ are listed below:

• Tools and Plugins: RabbitMQ offers many tools and plugins support for continuous integration, operational metrics, and integration to other systems.
• Asynchronous Messaging: RabbitMQ supports multiple messaging protocols, delivery acknowledgment options, message queuing, routes, and various exchange types.
• Distributed Deployment: RabbitMQ allows users to deploy queues as clusters for high throughput and availability.

What are RabbitMQ Unacked Messages?

RabbitMQ Uacked Messages are unacknowledged messages. In RabbitMQ, when many messages are delivered to the consumer or the target. But according to protocols, it is not guaranteed that message delivery will always be successful. So to solve this, Publishers and Consumers require a mechanism for delivery and processing confirmation. This is where Acknowledgement and Unacknowledgment. A message is ready when it is waiting to be processed. Whenever a consumer connects to the queue it receives a batch of messages to process. Meanwhile, the consumer is working on the messages the amount is given in prefetch size and they get the message unacked. RabbitMQ Unacked Messages are the messages that are not Acknowledged.  

If a consumer fails to acknowledge messages, the RabbitMQ will keep sending new messages until the prefetch value set for the associated channel is equal to the number of RabbitMQ Unacked Messages count. If RabbitMQ Unacked Messages are available then it will make the “struck” messages available again, and the client process will automatically reconnect. RabbitMQ Unacked Messages are read by the consumer but the consumer has never sent an ACK or confirmation to the RabbitMQ broker to say that it has finished processing it.

How to Remove Rabbit Unacked Messages 

Now that you have understood RabbitMQ and what are RabbitMQ Unacked Messages. In this section, you will learn the steps to remove RabbitMQ Unacked Messages. It is tiresome and inefficient when there is a long queue of RabbitMQ Unacked Messages. You will learn to remove RabbitMQ Unacked Messages using the command line and GUI as well. It will end up in a poor environment if you don’t deal with these RabbitMQ Unacked Messages. The following steps to remove RabbitMQ Unacked Messages are listed below:

Using Command Line

• First, let’s list down the queues with Unacked connections using the following command given below.

rabbitmqctl list_queues name messages messages_unacknowledged

• You can find the channel name that has RabbitMQ Unacked Messages. It will also allow you to view the connection that is associated with the channel. 

rabbitmqctl list_channels connection messages_unacknowledged

• You can close the channel using the following command given below.

rabbitmqctl close_connection "<rabbit@aditya-tla.1659610565.0284.0 >" "Closing Connection"

• You can also kill the process manually for the consumer that is associated with the channel. 
• To verify that you don’t have any RabbitMQ Unacked Messages but these same messages should be in the “Ready” state. The following command is given below.

rabbitmqctl list_queues name messages messages_unacknowledged

• If you want to remove RabbitMQ Unacked Messages from the queue then you can purge them using either of the following commands given below:

rabbitmqctl purge_queue Phono
rabbitmqadmin purge queue name=Phono

Using GUI

• First. open up your RabbitMQ web-based GUI console.
• Here, navigate to the Channel tab where you can have all the details related to Channels and RabbitMQ Unacked Messages.
• Go to the Connections tab to close them.

• Here, you can view the list of connections from the Connections tab. You can view further details of the connection by clicking one of them. Your GUI Console will look similar to the one shown below in the image.

• As you view the details of one connection, here, you can view the number of RabbitMQ unacked messages of each channel as shown in the image below.

• Then, click on the Force Close button to close the connection.
• Now, once the connection is closed. You can see that all the RabbitMQ Unacked Messages will come in a Ready state. 

• Next, select a queue to purge the messages. For this, you have to click on the Purge Messages button to clear all the messages from the queue.

RabbitMQ Unacked Messages Cause

The main reason that can cause RabbitMQ Unacked Messages is forgetting to ack or acknowledge the messages. If the auto_ack parameter is not turned on then you need to turn it on manually. Let’s have look at an example of a code where the auto_ack is not used. The following Python code for RabbitMQ Unacked Messages is given below:

import pika
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='Hey There!')

def callback(ch, method, properties, body):
    print(" [x] Received %r" % body)

channel.basic_consume(queue='Hey There!', on_message_callback=callback)
channel.start_consuming()
connection.close()

You can fix this issue of RabbitMQ Unacked Messages by calling basic_ack at the end of the callback function. The following code is given below.

def callback(ch, method, properties, body):
    print(" [x] Received %r" % body.decode())
    time.sleep(5)
    print(" [x] Done")
    ch.basic_ack(delivery_tag = method.delivery_tag)

You can also fix them by enabling the auto_ack while consuming the message. The following code is given below.

channel.basic_consume(queue='Hey There!',
                      auto_ack=True,
                      on_message_callback=callback)

Conclusion

In this article, you learned about RabbitMQ, its basic terminologies, and what RabbitMQ Unacked Messages are. Also, you read how RabbitMQ Unacked Messages create a serious problem in the smooth running of the application. Then, you went through the steps to remove RabbitMQ Unacked Messages. RabbitMQ is the most widely deployed message broker that helps companies develop and manage scalable applications.

It is essential to store these data streams in Data Warehouses and run Analytics on them to generate insights. Hevo Data is a No-code Data Pipeline solution that helps to transfer data from Kafka and various other sources to the desired Data Warehouse. It fully automates the process of transforming and transferring data to a destination without writing a single line of code. Sign up for Hevo’s 14-day free trial and experience seamless data migration.

FAQs