Skip to content

processMessageOrRequeue Pattern

The processMessageOrRequeue Pattern is a messaging strategy used to handle distributed tasks that depend on data collected asynchronously by multiple listeners. It ensures that a main processing step waits until prerequisite data queries are complete before proceeding, while managing potential failures through a requeue mechanism.^[001-TODO__報表下載.md]

Core Mechanism

The pattern relies on a state variable stored in a fast-access data store, such as a Redis hash. A specific key (e.g., queryDoneCount) is incremented by listeners as they finish their specific sub-tasks.^[001-TODO__報表下載.md]

The main processor checks this counter against a required threshold.^[001-TODO__報表下載.md] * If the counter is less than the threshold, the message is typically requeued for later processing. * If the counter meets or exceeds the threshold, the processor executes the final business logic (e.g., generating a CSV and uploading to GCP).^[001-TODO__報表下載.md]

Configuration

The behavior of the pattern is governed by configurable parameters to balance latency and reliability:^[001-TODO__報表下載.md]

  • Requeue Times: The maximum number of times a message is allowed to be requeued. The source indicates a default of 6 attempts.
  • Requeue Delay: The wait time before the message is available for processing again. The source indicates a default of 1 minute.

Failure Handling

If the message exhausts its maximum requeue attempts without meeting the processing conditions (e.g., queryDoneCount remains too low), the system treats the process as a failure.^[001-TODO__報表下載.md]

In such failure scenarios, the system updates the status of the relevant persistence entity (e.g., FileDownloadRecordEntity) to reflect the incomplete or failed state.^[001-TODO__報表下載.md]

Sources

^[001-TODO__報表下載.md]