Animated Architecture Patterns

Priority Queue

Urgent messages are processed ahead of routine ones.

worker pool priority=high priority=low 1st 2nd Order service urgent emails flash sale: 12/s Catalogue import nightly re-index jobs high-priority queue one queue, first in first out low-priority queue ! ! ! ! ! ! ! ! ! ! ! depth 0 · oldest – depth 1 · oldest 0 s ▲ depth and oldest age rising ▼ depth and oldest age falling depth 500 oldest 50 s oldest 3 min oldest 6 min oldest 10 min 500 jobs ahead of the email starving moving: 2.5 jobs/s or publish to one topic whose subscriptions filter on a priority attribute Reserve: worker 4 takes only low work (or pull high and low in a fixed 3 : 1 ratio) 3 Who may send urgent work? Only the order service may publish to high. If every producer marks its messages urgent, nothing is. denied low only long job worker 1 worker 2 worker 3 worker 4 ! ! ! ! ✓ sent 4 workers × 2.5 jobs/s = 10 jobs/s all 4 on high: 10 jobs/s low: 0 jobs/s high: 3 × 2.5 = 7.5 jobs/s low: 1 × 2.5 = 2.5 jobs/s high: 6 × 2.5 = 15 jobs/s low: 1 × 2.5 = 2.5 jobs/s 2 No pre-emption Worker 1 keeps its long low job; the urgent message waits for worker 2. Split long jobs into short ones. ! Password-reset email 500 jobs ahead ÷ 10 jobs/s the user waits 50 s Password-reset email high queue empty: next free worker (~0.1 s) + 0.4 s of work sent after ≈ 0.5 s Flash sale: 12 urgent orders/s arrive; the pool finishes 10/s. High-first polling: high is never empty, so low gets nothing. Scale each pool on its own queue: high grows to 6 workers, 15 jobs/s. 1 Priority inside one queue? RabbitMQ: yes, in classic and quorum queues. SQS, Service Bus, Pub/Sub, Kafka: no; use a queue, subscription or topic per level. ! ! ! ! ! ! ! password reset

Step 1 · First come, first served. The nightly import has put 500 re-index jobs on the only queue, and a password-reset email lands behind them. Four workers finish 2.5 jobs a second each, 10 in total, so the email waits 500 ÷ 10 = 50 s while the user stares at the screen. In one queue every message has the same standing, whatever it is for.

Step 2 · A queue per priority. Producers mark each message and send it to a high or a low queue (or publish to one topic whose subscriptions filter on a priority attribute). Workers always check the high queue first, so the password-reset email finds it empty, goes to the next free worker and is sent about 0.5 s after it arrived. The import carries on in the low queue.

Step 3 · Starvation, then reserve. A flash sale sends 12 urgent order confirmations a second, more than the pool's 10. With high-first polling the high queue never empties and the low queue gets nothing: its oldest message is soon 10 minutes old. Reserve capacity for it (worker 4 takes only low work, or the pool pulls high and low 3 : 1), so high gets 7.5 jobs a second and low 2.5, and scale each pool on its own queue: the high pool grows to six workers, 15 jobs a second.

Step 4 · Know the limits. RabbitMQ can order by priority inside one queue (classic and quorum queues); Amazon SQS, Azure Service Bus, Google Cloud Pub/Sub and Kafka need a queue, subscription or topic per level. Priority only decides which message is picked next: worker 1's long low-priority job keeps its worker, so split long jobs. And decide who may send urgent work: if every producer marks its messages urgent, nothing is.

The problem

Most queues hand out work in arrival order. That is fair, but it ignores what each message is for. When a nightly batch puts hundreds of routine tasks on the same queue as interactive work, a password-reset email or a "payment failed" notice waits behind all of them, and someone at a screen waits with it. More consumers shorten the wait on average, yet every urgent message still sits behind the whole backlog that arrived before it.

How it works

Give each message a priority when it is produced, and let the consumers take higher-priority work first. There are two designs:

  • One queue with broker-side priorities. The broker keeps the queue ordered by a priority field and hands out the most important message first. RabbitMQ does this. Classic queues need the x-max-priority argument when the queue is declared (1 to 255; the documentation recommends only a handful of levels, and a policy can't set it). Quorum queues always support priorities: strictly, with levels 0 to 31, since RabbitMQ 4.3; versions 4.0 to 4.2 had two levels, normal and high, delivered two high for every normal.
  • A queue, or a subscription, per priority. Producers send each message to the high or the low queue, or publish to one topic and let a subscription per priority filter on an attribute. This works on brokers that have no priority field: Amazon SQS (AWS suggests separate queues for prioritization; its fair queues balance tenants, not urgency), Azure Service Bus (the Azure pattern's example sets a custom Priority property and routes it with SQL filters to a high and a low subscription), Google Cloud Pub/Sub (a message carries attributes but no priority, and subscription filters can match an attribute) and Apache Kafka (a topic per priority).

The consumers then choose what to take next:

  • High-first polling. One pool checks the high queue and takes low work only when high is empty. It gives urgent work the shortest wait and wastes no capacity, but low work starves whenever high never empties.
  • A dedicated pool per queue. Each queue has its own consumers and its own scaling limits; the Azure sample lets the high consumer scale out to 200 instances and the low one to 40. Classes are isolated, but idle high consumers don't help with a low backlog unless you let them.
  • Weighted ratios. The pool takes, for example, three high messages for every low one. Low work always moves, at the cost of some urgent latency.

In the diagram, four workers each finish 2.5 jobs a second (10 in total, 0.4 s per job). Behind 500 routine jobs in one queue, the reset email waits 500 ÷ 10 = 50 s; with a high queue it waits only for the next free worker.

When to use it

  • Interactive work shares workers with batch work: password resets and order confirmations next to imports, re-indexing or report generation.
  • Customers or tenants buy different service levels, and premium requests need a tighter latency target.
  • Alerts and pages must not wait behind routine reports or digests.
  • Not when all work is equally urgent, or when only average throughput matters: priority adds queues, routing rules and failure modes without improving either.

Trade-offs

  • Starvation. Strict high-first service can hold low work back for as long as urgent work keeps coming. Bound it: reserve consumers for low (one of four workers in the diagram, so high gets 7.5 jobs a second and low 2.5), pull in a fixed ratio, or age messages by promoting anything that has waited past its target to the higher queue. RabbitMQ's documentation notes that classic queues serve their priority levels in cycles so lower levels are never starved, while strict quorum queues can delay them indefinitely.
  • No pre-emption. Priority only chooses the next message. A long low-priority job that has already started keeps its worker until it finishes, so split long jobs into short steps with checkpoints and the workers come back to the queue often.
  • Order across priorities is lost. A later urgent message overtakes earlier routine ones. If two messages change the same entity, keep them in the same class or make the consumer cope with any order.
  • Priority is not capacity. If urgent work alone exceeds what the consumers finish, the high queue grows too. Scale each queue's consumers on its own depth, or on the age of its oldest message: see Autoscaling and Competing Consumers.
  • More moving parts. Separate queues bring separate dead-letter queues, alarms, permissions and polling costs.

Implementation notes

  • Priority only matters when work waits. With RabbitMQ, use manual acknowledgements and a small prefetch: a consumer with a large prefetch has already taken low-priority messages before the urgent one arrives. With SQS, long polling an empty high queue holds the loop for up to the wait time before it looks at the low queue; use a short wait in a high-first loop, or separate pollers.
  • Kafka has no message priority. A consumer fetches from all of its partitions at once; to favour a high-priority topic, pause the low topic's partitions while high has a backlog and resume them afterwards, or run a separate consumer group per topic.
  • Decide who sets priority. Derive it from facts the system trusts (the message type, the customer's plan), not from a flag any producer can set. Restrict who may publish to the high queue with separate credentials or topic permissions, and watch the volume per producer: if every producer marks its messages urgent, nothing is.
  • Scale each queue on its own signal. KEDA has scalers for RabbitMQ, Amazon SQS, Azure Service Bus, Google Cloud Pub/Sub and Kafka; the Pub/Sub scaler can scale on the age of the oldest unacknowledged message, and SQS publishes a matching signal as the CloudWatch metric ApproximateAgeOfOldestMessage.
  • Dead-letter per queue. Give each priority its own dead-letter queue, so a poison urgent message can't block the high path and its alarm reaches the right team.
  • Monitor per class. Track depth and the age of the oldest message for each queue against a target per class, for example a few seconds for high and an hour for low. An average across classes hides starvation.
  • Rank work at the edge too. Queues are one place to prioritise. At the API, shed the least critical requests first when overloaded (Rate Limiting; Google's SRE book describes criticality levels that travel with each request), and give each class its own pool so one can't exhaust another (Bulkhead).