Skip to content

hook_civicrm_queueRun

Summary

Implement a custom handler for evaluating batched data in the queue.

Definition

hook_civicrm_queueRun_{PAYLOAD}(CRM_Queue_Queue $queue, array $items, &$outcomes) { : void

Parameters

  • {PAYLOAD} - the type of data supported by this queue
  • $queue - the specific queue which received the data (subclass of CRM_Queue_Queue)
  • $items - the list of items. (type varies based on $queue; for ordinary SQL-backed queues, this is CRM_Queue_DAO_QueueItem)
  • $outcomes - alterable list of outcome (one outcome for each item). these are reported back to the agent that initiated the run.

Availability

Example

For a full example, we need to create and populate a queue. Later, when the queue runs, we will receive hook_queueRun_{PAYLOAD}.

Part 1: Create and populate a queue

$queue = Civi::queue('my-mail-queue', [
  'type' => 'SqlParallel',
  'agent' => 'server',
  'payload' => 'sendmail',
  'batch_limit' => 10,
  'error' => 'delete',
]);

$defaults = ['from' => 'Info <info@example.org>',  'subject' => 'Greetings'];

$queue->createItem(['toEmail' => 'alice@example.org', 'text' => 'Hello Alice!'] + $defaults);
$queue->createItem(['toEmail' => 'bob@example.com', 'text' => 'Hello Bob!'] + $defaults);
$queue->createItem(['toEmail' => 'carol@example.net', 'text' => 'Hello Carol!'] + $defaults);
$queue->createItem(['toEmail' => 'dave@example.info', 'text' => 'Hello Dave!'] + $defaults);

Part 2: Execute items from the queue

function example_civicrm_queueRun_sendmail(CRM_Queue_Queue $queue, array $items, array &$outcomes) {
  foreach ($items as $key => $item) {
    try {
      // Assert: $item->data['toEmail'] is an email address.
      // Assert: $item->data['subject'] is the email subject-line.
      CRM_Utils_Mail::send($item->data);

      // The send process succeeded. We're finished with the item.
      $queue->deleteItem($item);
      $outcomes[$key] = 'ok';
    }
    catch (\Exception $e) {
      // The send process failed. We'll let go of the item so it can be retried later.
      $queue->releaseItem($item);
      $outcomes[$key] = 'retry';
    }
  }
});

The hook should visit all $items. For each item, you need to do a few things:

  1. Perform some work (e.g CRM_Utils_Mail::send()).
  2. Update or delete the item (e.g. $queue->deleteItem())
  3. Report back about the outcome (e.g. $outcomes[$key] = 'ok')

For simplicity, this example treats each item as independent. However, you could perform set-oriented operations -- as long as you eventually do all three steps (perform work; update item; report outcome).

Tip: Batching requires careful error-handling

If you have batch_limit > 1, then your process will claim several items simultaneously. An error in one item can affect the batch.

For example, if the first item raises an unhandled exception, then the second item won't run. You should generally look for exceptions and decide the impact: Will you continue running other items? Will you release the other items for a future invocation?

It is not always possible to catch errors. For example, power-outages and PHP fatals are not catchable. This is why the queue-system applies a lease-time to every item (civicrm_queue_item.release_time). Eventually, when the lease expires, the queue will retry the item.

Never-the-less, implementations of hook_queueRun_{PAYLOAD} should make a best effort handle errors. This will minimize delays and unexpected results.