Skip to content

Queueing

Overview

In CiviCRM, queueing helps the system organize time-consuming workloads. The general process is:

  1. Create a queue.
  2. Add items to the queue.
  3. Start an agent to run the items.

The queueing system is flexible, and you have options in how to approach each part. You can:

  • Create one basic queue that stays available continuously - or create several dynamic queues for distinct workloads.
  • Define work-items with simple PHP functions or with optimized batching functions.
  • Execute work-items sequentially or concurrently.
  • Set up an agent to monitor queues from the user's web-browser or on the PHP server.

To get oriented, read the Quick Start for an overview. Then, use the other pages to explore in greater depth.

  • "Queueing: Quick Start" develops a minimal program that demonstrates key concepts.
  • "Queueing: Guide" walks through the various options and considers when to use them.
  • "Queueing: Reference" gives a technical breakdown of options for key interfaces.
  • "Queueing: Comparison" evaluates the differences between the two generations of CiviCRM queueing code (e.g. CiviCRM 4.2 vs 5.47).

Appendix: History

  • v4.2: Introduce CRM_Queue subsystem, including the CRM_Queue_Service, the single-threaded CRM_Queue_Runner, and the Sql/Memory drivers.
  • v5.28: Add SqlParallel driver.
  • v5.47: Add Civi::queue() helper and preliminary support for persistent queues (civicrm_queue, Civi\Api4\Queue, is_persistent).
  • v5.68: Add more APIv4 methods for managing queues. Add BasicHandlerTrait. Deprecate runner property in favor of agent+payload.