RabbitMQ Delayed Messages 101: How to Delay & Schedule Messages Made Easy

There are some scenarios in real life where messages must be sent with a delay or at a specific time, such as a smart water heater that needs to be turned on after 30 minutes, an unpaid order or sending SMS, email, and push notifications for a sale that begins at 2:00 PM.

The RabbitMQ Delayed Message AMQP protocol lacks a native delayed queue feature, if you search for “how to delay/schedule messages in RabbitMQ,” you’ll most likely come across two possible solutions. The first solution is to use the combination of the message TTL function and the dead-lettering function. And the second option is using the official RabbitMQ delayed messages exchange plugin.

This article goes into great detail about RabbitMQ Delayed Messages. Additionally, it provides a brief overview of RabbitMQ.

Table of Contents

• What is RabbitMQ?
  • Key Features of RabbitMQ
• How to Delay & Schedule RabbitMQ Delayed Messages?
  • Using TTL and DLX to Delay Message Delivery
  • RabbitMQ Delayed Message Plugin
• Benefits of Delaying & Scheduling RabbitMQ Delayed Messages
• Conclusion

What is RabbitMQ?

RabbitMQ is an open-source message broker (also known as message-oriented middleware) that was created to support the Advanced Message Queuing Protocol (AMQP). It has since been expanded with a plug-in architecture to support protocols such as Simple (or Streaming) Text Oriented Message Protocol (STOMP), Message Query Telemetry Transport (MQTT), and others.

For clustering and failover, the RabbitMQ server is written in Erlang and employs the Open Telecom Platform framework. Client libraries for interacting with the broker are available for all major programming languages, and the source code is available under the Mozilla Public License.

It is a simple messaging system that can be used on-premises or in the cloud. and which a variety of messaging protocols are supported.

Key Features of RabbitMQ

Here are some features of RabbitMQ:

• Clustering: The clustering in RabbitMQ was designed with two aims in mind. If one node fails, the event’s consumers and producers can continue to function while adding other nodes to scale messaging throughput linearly.
• Easy Routing: Messages are often routed through exchanges before arriving at queues using flexible routing. For more complicated routing, users can connect exchanges together or develop their exchange type as a plugin.
• Reliability: Persistence, delivery feedback, publisher confirmation, and high availability are among RabbitMQ’s key features that have a direct impact on performance.
• Security: Client Certificate Checking and SSL-only communication can assist secure client connections. The virtual host can regulate user access, ensuring high-level message isolation.

How to Delay & Schedule RabbitMQ Delayed Messages?

People have been looking for ways to implement delayed messaging with RabbitMQ for quite some time. To date, the accepted solution has been to use a combination of the message  — TTL and Dead Letter Exchanges  — as proposed by James Carr here.

The RabbitMQ Delay Message Plugin adds a new exchange type to RabbitMQ that allows messages routed through that exchange to be delayed if the user so desires. Let us see how the deployment and scheduling go with these 2 methods.

• Using TTL and DLX to Delay Message Delivery
• RabbitMQ Delayed Message Plugin

1) Using TTL and DLX to Delay Message Delivery

By combining these functions, we can publish a message to a queue that will expire its message after the TTL and then reroute it to the exchange with the dead-letter routing key so that it ends up in a queue.

• Step-by-step instructions:
  • Declare the pending queue
  • Set the x-dead-letter-exchange argument property to the default value “”.
  • Set the argument property x-dead-letter-routing-key to the name of the destination queue.
  • Set the x-message-ttl argument property to the number of milliseconds you want the message to be delayed.
• Sign up for the destination queue.

The following Ruby snippet, which relies on the excellent Bunny library, shows how to implement delayed messages.

#RabbitMQ Delayed Message
require 'bunny'

B = Bunny.new ENV['CLOUDAMQP_URL']
B.start

DELAYED_QUEUE='work.later'
DESTINATION_QUEUE='work.now'

def publish
  ch = B.create_channel
  # declare a queue with the DELAYED_QUEUE name for RabbitMQ Delayed Message
  ch.queue(DELAYED_QUEUE, arguments: {
    # set the dead-letter exchange to the default queue
    'x-dead-letter-exchange' => '',
    # when the message expires, set change the routing key into the destination queue name
    'x-dead-letter-routing-key' => DESTINATION_QUEUE,
    # the time in milliseconds to keep the message in the queue
    'x-message-ttl' => 3000
  })
  # publish to the default exchange with the the delayed queue name as routing key,
  # so that the message ends up in the newly declared delayed queue
  ch.default_exchange.publish 'message content', routing_key: DELAYED_QUEUE
  puts "#{Time.now}: Published the message"
  ch.close
end

def subscribe
  ch = B.create_channel
  # declare the destination queue for RabbitMQ Delayed Message
  q = ch.queue DESTINATION_QUEUE, durable: true
  q.subscribe do |delivery, headers, body|
    puts "#{Time.now}: Got the message"
  end
end

subscribe()
publish()

sleep
#RabbitMQ Delayed Message

2) RabbitMQ Delayed Message Plugin

So, let’s start with the second solution by installing the plugin, but first, let’s take a look at the following prerequisites:

• Versions 3.5.8 and later of RabbitMQ.
• Versions 18.0 and later of Erlang/OTP

Here are the steps to get started:

• Plugin Installation
• Making Use of the Exchange
• Message Delays
• Routing Flexibility
• Checking Delayed Messages

1) Plugin Installation

To install the plugin, visit our Community Plugins page and download the .ez files for your RabbitMQ installation. Copy the plugin to RabbitMQ’s plugin folder, then run the following command to enable it:

#RabbitMQ Delayed Message
rabbitmq-plugins enable rabbitmq_delayed_message_exchange

We can begin using the plugin once it has been enabled.

2) Making Use of the Exchange

To use the Delayed Message Exchange, simply declare an exchange with the “x-delayed-message” exchange type, as shown below:

// ... elided code ...
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
channel.exchangeDeclare("my-exchange", "x-delayed-message", true, false, args);
// ... more code ...

Later on, we’ll explain what the special argument “x-delayed-type” in our exchange declaration means.

3) Message Delays

To delay a message, the user must publish it with the x-delay header, which accepts an integer representing the number of milliseconds the message should be delayed by RabbitMQ. It’s worth noting that delay in this context means delaying message routing to queues or other exchanges. There is no concept of consumers in the exchange.

As a result, once the delay has passed, the plugin will attempt to route the message to the queues that match the routing rules of the exchange and the once assigned to the message. If the message cannot be routed to any queue, it will be discarded, as specified by AMQP for unroutable messages.

#RabbitMQ Delayed Message
// ... elided code ...
byte[] messageBodyBytes = "delayed payload".getBytes();
AMQP.BasicProperties.Builder props = new AMQP.BasicProperties.Builder();
headers = new HashMap<String, Object>();
headers.put("x-delay", 5000);
props.headers(headers);
channel.basicPublish("my-exchange", "", props.build(), messageBodyBytes);

The message will be delayed for five seconds in the preceding example before being routed by the plugin. That example assumes you’ve connected to RabbitMQ and obtained a channel.

4) Routing Flexibility

When we declared the exchange above, we used an x-delayed-type argument that was set to direct. That tells the exchange what kind of behavior we want it to have when routing messages, creating bindings, and so on.

5) Checking Delayed Messages

How can we tell if a message was delayed or not once we receive it on the consumer side? The x-delay message header is retained by the plugin, but the passed value is negated. So, if you send a message with a 5000-millisecond delay, the consumer will find the x-delay header set to -5000.

Benefits of Delaying & Scheduling RabbitMQ Delayed Messages

Here are some benefits of delaying & scheduling RabbitMQ delayed messages:

• High Availability and Performance: RabbitMQ Delayed Message Queue supports hierarchical delayed messages. It has high performance and can accumulate messages indefinitely for high availability. It also allows for scheduling within seconds in order to achieve highly precise mass storage.
• Product Maturity is High: RabbitMQ Delayed & Scheduled messages have been widely used in various businesses and promotional scenarios throughout the economy, withstanding numerous traffic peaks and stability tests. RabbitMQ has evolved into the industry’s most reliable scheduled message system.
• High-Tech Community Activity: RabbitMQ Delayed Message has created a complete technical ecosystem as a top-level project. Its scheduled messages naturally inherit the advantages of top-level projects, with multi-language and multi-protocol support, and feature a complete technical ecology.
• Professional Technical Support: Cloud tech focuses on cloud computing, and RabbitMQ evolves in tandem with cloud storage’s technical architecture. While providing technical facilities, their strong technical teams also provide professional technical support to RabbitMQ customers.

Conclusion

This article taught you about RabbitMQ Delayed Messages and how to delay and schedule messages using two different methods, as well as a brief introduction to RabbitMQ and its features.

There are various Data Sources that organizations leverage to capture a variety of valuable data points. But, transferring data from these sources into a Data Warehouse for a holistic analysis is a hectic task. It requires you to code and maintains complex functions that can help achieve a smooth flow of data. An Automated Data Pipeline helps in solving this issue and this is where Hevo comes into the picture. Hevo Data is a No-code Data Pipeline and has awesome 100+ pre-built Integrations that you can choose from.

visit our website to explore hevo

Hevo can help you integrate data from 100+ data sources and load them into a destination to analyze real-time data at an affordable price. It will make your life easier and Data Migration hassle-free. It is user-friendly, reliable, and secure.

SIGN UP for a 14-day free trial and see the difference!

Share your experience of learning about RabbitMQ Delayed Messages in the comments section below.