Queueing: Quick Start

About this document

This provides one example to demonstrate how the major queueing features fit together. For a broader perspective, see Overview and Guide.

To keep the example as short as possible, we will do all interaction on the command-line and show messages with printf().

Create a module-extension with civix. Let's call it "Queue Quick Start" (qstart):

civix generate:module qstart --enable=no
cd qstart

In the main file (qstart.php), create the queue and fill it with work-items.

/**
 * Register the new queue (during installation).
 */
function qstart_civicrm_enable(): void {
  Civi::queue('qstart-greeter', [
    'type' => 'Sql',      // Store the queue in MySQL.
    'agent' => 'server',  // Who is responsible for monitoring the queue
    'payload' => 'task',  // We will add executable tasks to this queue
    'error' => 'abort',   // If any task encounters a final failure, then stop the entire queue.
    'reset' => TRUE,      // Clear out any old work-items.
  ]);
  qstart_printf("Created queue\n");
}

/**
 * Add several tasks to the queue.
 */
function qstart_fill(): void {
  $queue =  Civi::queue('qstart-greeter');
  $names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve', 'Frank'];
  foreach ($names as $name) {
    $task = new CRM_Queue_Task('qstart_task_hello', [$name]);
    $queue->createItem($task);
  }
  qstart_printf("Added all items\n");
}

/**
 * Define the task.
 */
function qstart_task_hello(CRM_Queue_TaskContext $ctx, string $name): bool {
  qstart_printf("Hello $name!\n");
  return TRUE;
}

function qstart_printf(...$args) {
  fprintf(STDERR, ...$args);
}

Observe a few important things:

• It calls Civi::queue() to create the queue. It specifies the name (qstart-greeter) and the expected type of data (payload=>task).
• It creates an executable task object ($task = new CRM_Queue_Task(...)). This is based on a PHP function (qstart_task_hello).
• It adds this object to the queue (createItem($task)).

When we have this code in place, then we can enable and run the extension:

## Enable the new extension
cv en qstart
> Enabling extension "qstart"
> Created queue

## Fill the queue
cv ev 'qstart_fill();'
> Added all items

## Run some tasks from the queue
cv api4 Queue.run queue=qstart-greeter
> Hello Alice!
> Hello Bob!
> Hello Carol!
> Hello Dave!
> Hello Eve!
> Hello Frank!
> [
>     {
>         "loop_duration": "0.021",
>         "loop_requests": 7,
>         "item_successes": 6,
>         "item_errors": 0,
>         "queue_ready": 0,
>         "queue_blocked": 0,
>         "queue_total": 0,
>         "exit_message": "No claimable items"
>     }
> ]

In this final step, we used the Queue API to run items from the queue. The output shows that:

• The task performed some work. (Hello Alice!, Hello Bob!, and so on)
• The API returned a summary of how much work was done. (6 items were successfully executed in 0.02 seconds. It exited because there were no more items to run.)

This is a small example of queueing, but it shows the essential pieces: create the queue, add items, and run the queue. For more realistic usage, you will want to change some options. The "Queueing: Guide" explores the options in more depth.