Skip to content

RabbitMQ Dead Letter Queue with Delay Pattern

The RabbitMQ Dead Letter Queue with Delay Pattern is a message processing strategy used to handle task execution with a deferred retry mechanism. This pattern leverages Time-To-Live (TTL) configurations and Dead Letter Exchanges (DLX) to move messages to a secondary queue for processing after a specified interval.^[com.galaxy.mq.rabbit.config.ReportConfig]

In the provided implementation, this pattern is used to manage asynchronous report generation timeouts. If a report is not successfully generated within a specific timeframe (e.g., 5 minutes), the message is routed to a Dead Letter Queue to trigger a failure status update, ensuring the system state remains consistent.^[com.galaxy.mq.rabbit.config.ReportConfig]

Configuration

The pattern is implemented using three primary components: a Delay Queue, a Dead Letter Exchange, and a Dead Letter Queue.^[com.galaxy.mq.rabbit.config.ReportConfig]

Delay Queue Setup

The primary queue (the delay queue) is configured with a ttl argument and deadLetterExchange/deadLetterRoutingKey arguments.^[com.galaxy.mq.rabbit.config.ReportConfig]

  • Queue Name: plt.basic.report.delay.q
  • TTL: 300,000 ms (5 minutes).
  • Dead Letter Exchange: plt.basic.report.dead.ex
  • Dead Letter Routing Key: plt.basic.report.dead.q

When a message enters this queue, RabbitMQ holds it for the duration of the TTL. Once the TTL expires, the message is automatically routed to the specified Dead Letter Exchange.^[com.galaxy.mq.rabbit.config.ReportConfig]

Dead Letter Infrastructure

The messages are routed to a standard Direct Exchange and bound to a final processing queue.^[com.galaxy.mq.rabbit.config.ReportConfig]

  • Exchange: plt.basic.report.dead.ex (Direct)
  • Queue: plt.basic.report.dead.q

Workflow Implementation

The workflow typically involves an initial processing failure or a waiting period, followed by the transition to the dead letter queue.

  1. Initial Trigger: A listener monitors the main processing queue. If a task requires delayed handling or immediate retry is not possible (or for monitoring purposes), the message is sent to the delay queue.^[com.galaxy.mq.rabbit.ReportListener]
  2. Delay Period: The message resides in plt.basic.report.delay.q for the configured TTL duration (5 minutes).^[com.galaxy.mq.rabbit.config.ReportConfig]
  3. Expiry and Routing: Upon TTL expiration, RabbitMQ routes the message to plt.basic.report.dead.ex.^[com.galaxy.mq.rabbit.config.ReportConfig]
  4. Final Handling: A listener on the dead letter queue (plt.basic.report.dead.q) consumes the message and executes the terminal logic, such as updating a database record to a "FAIL" status.^[com.galaxy.mq.rabbit.ReportListener]

Code Examples

Defining the Delayed Queue

The delay queue is configured to route expired messages to the dead letter exchange.

@Bean("reportDelayQueue")
public Queue reportDelayQueue(@Qualifier("normalAdmin") RabbitAdmin admin) {
    Queue q = QueueBuilder.durable(DELAY_QUEUE_Q)
            .quorum()
            .deadLetterExchange(DEAD_QUEUE_EX)
            .deadLetterRoutingKey(DEAD_QUEUE_Q)
            .ttl(DELAY_QUEUE_DELAY_TIME)
            .build();
    q.setAdminsThatShouldDeclare(admin);
    return q;
}
^[com.galaxy.mq.rabbit.config.ReportConfig]

Handling the Dead Letter Message

The consumer for the dead letter queue handles the final logic after the delay.

@SneakyThrows
@RabbitListener(queues = DEAD_QUEUE_Q, containerFactory = "normalContainerFactory", concurrency = "1")
public void handleFailedRecord(@Payload Message<ReportQryVo> messages, Channel channel) {
    processMessageOrRequeue(messages, channel,
            new RabbitListenerRequeueable.RequeueInfo(DEAD_QUEUE_EX, DEAD_QUEUE_Q, DEAD_QUEUE_Q, rabbitSender),
            () -> {
                ReportQryVo payload = messages.getPayload();
                reportDomainService.updateStatusFail(payload.getId());
                log.info("update message status to fail : id = " + payload.getId());
            });
}
^[com.galaxy.mq.rabbit.ReportListener]

Sources

^[com.galaxy.mq.rabbit.config.ReportConfig] ^[com.galaxy.mq.rabbit.ReportListener]