Token Reference

Introduction

CiviCRM tokens enable one to generate personalized emails, reminders, print documents. etc.

When working with tokens as a developer, there are two major tasks:

• Composing a message by evaluating a token expression (Hello {contact.first_name}! How do you like {address.city}?).
• Defining a token and its content.
• Listing available tokens.

For each task, there are a couple available patterns, and we'll explore them in more depth below. But first, it helps to put the functionality in context.

Examples

Token	Description
{domain.name}	Name of this domain (organization/site/deployment)
{domain.address}	Meta-token with the full-formed mailing address of this domain (organization/site/deployment))
{contact.first_name}	The contact's first_name
{contact.display_name}	The contact's display_name
{mailing.name}	The name of the mailing

For more examples of tokens and token replacement, see User Guide: Common workflows: Tokens and mail merge.

Composing messages

Civi\Token\TokenProcessor provides a flexible way to define and process tokens. It allows open-ended data for many entities, and it provides loose coupling between token-consumers, token-providers, and template-language.

TokenProcessor was phased-in between v4.7 and v5.43

TokenProcessor was introduced in v4.7 for use with Scheduled Reminders. It was gradually adopted in other use-case and became the ordinary/typical mechanism circa v5.43. For more information, see Appendix: History.

If you are writing an application controller which sends a message (e.g. for CiviMail or PDFs or scheduled reminders), the process is:

• Create an instance of Civi\Token\TokenProcessor.
• Fill the TokenProcessor with:
  • General context data.
  • Recipient context data (addRow()).
  • Message templates (addMessage()).
• Evaluate the tokens. This fires an event (civi.token.eval). Other modules will inspect the message-template(s) and recipients and produce token data.
• For each of the recipients, request the rendered output.

For example:

$tokenProcessor = new TokenProcessor(\Civi::dispatcher(), [
  // Give a unique identifier for this instance.
  'controller' => __CLASS__,

  // Enable or disable handling of Smarty notation.
  'smarty' => FALSE,

  // List any data fields that we plan to provide.
  'schema' => ['contactId'],
]);

// Define the message template.
$tokenProcessor->addMessage('body_text', 'Hello {contact.display_name}!', 'text/plain');

// Fill in data about the recipients/targets.
$tokenProcessor->addRow(['contactId' => 123]);
$tokenProcessor->addRow(['contactId' => 456]);

// Evaluate any tokens which are referenced in the message.
// In this example, `evaluate()` will:
//  - Recognize that `{contact.display_name}` needs to be available.
//  - Fetch the display names (`SELECT id, display_name FROM civicrm_contact WHERE id IN (123,456)`)
$tokenProcessor->evaluate();

// Display mail-merge data.
foreach ($tokenProcessor->getRows() as $row) {
  echo $row->render('body_text');
}

Scope of context data

The example above has a data-array for new TokenProcessor() and another data-array for addRow().

These two arrays use the same data. Top-level fields are inherited at the row-level.

Defining tokens

If a use-case builds on the newer TokenProcessor (above), then an additional API is available for defining tokens. This API resolves the limitations above.

TokenProcessor emits two events which allow you to define new tokens. Consider this example which defines {profile.viewUrl} and {profile.viewLink}:

use Symfony\Component\Config\Resource\FileResource;
use Symfony\Component\DependencyInjection\ContainerBuilder;

/**
 * Add token services to the container.
 *
 * @param \Symfony\Component\DependencyInjection\ContainerBuilder $container
 */
function example_civicrm_container(ContainerBuilder $container) {
  $container->addResource(new FileResource(__FILE__));
  $container->findDefinition('dispatcher')->addMethodCall('addListener',
    ['civi.token.list', 'example_register_tokens']
  )->setPublic(TRUE);
  $container->findDefinition('dispatcher')->addMethodCall('addListener',
    ['civi.token.eval', 'example_evaluate_tokens']
  )->setPublic(TRUE);
}

function example_register_tokens(\Civi\Token\Event\TokenRegisterEvent $e) {
  $e->entity('profile')
    ->register('viewUrl', ts('Profile View URL'))
    ->register('viewLink', ts('Profile View Link'));
}

function example_evaluate_tokens(\Civi\Token\Event\TokenValueEvent $e) {
  foreach ($e->getRows() as $row) {
    /** @var TokenRow $row */
    $row->format('text/html');
    $row->tokens('profile', 'viewUrl', 'http://example.com/profile/' . $row->context['contactId']);
    $row->tokens('profile', 'viewLink', ts("<a href='%1'>Open Profile</a>", array(
      1 => $row->tokens['profile']['viewUrl'],
    )));
  }
}

Some notes on the the above:

• $row->context['...'] returns contextual data, regardless of whether you declared it at the row level or the processor level.
• To update a row's data, use the context() and tokens() methods. To read a row's data, use the $context and $tokens properties. These interfaces support several notations, which are described in the TokenRow class.
• You have control over the loop. You can do individual data-lookups in the loop (for simplicity) – or you can also do prefetches and batched lookups (for performance).
• To avoid unnecessary computation, you can get a list of tokens which are actually required by this mailing. Call $e->getTokenProcessor()->getMessageTokens().
• In this example, we defined tokens in HTML format, and we rely on a default behavior that auto-converts between HTML and text (as needed). However, we could explicitly define HTML and plain-text variants by calling $row->format() and $row->tokens() again.
• The class \Civi\Token\AbstractTokenSubscriber provides a more structured/opinionated way to handle these events.
• For background on the event dispatcher (e.g. listeners vs subscribers), see Symfony Documentation

The main limitation of this technique is that it only works with TokenProcessor. At time of writing, this is used for Scheduled Reminders and FlexMailer.

Listing tokens

Suppose the application sends an automated "Thank You" letter for each donation. The administrator may customize this letter - choosing their own prose and mixing in different tokens. But which tokens can they use? In the administrative interface, you should provide a list of available tokens (via dropdown, autocomplete, etc).

The list of tokens depends on the use-case and available data, as in:

Use-Case	Available Data	Available Tokens
Send a donation "Thank You" letter	Contact record Contribution record	{contact.*} {contribution.*}
Send a newsletter	Contact record Mailing record	{contact.*} {mailing.*} {action.*}

This list is also influenced by extensions - an extension may define extra tokens like {contact.foo_bar} or {foo.bar}.

An application may request the list of available tokens via listTokens():

$tokenProcessor = new TokenProcessor(\Civi::dispatcher(), [
  'controller' => __CLASS__,
  'smarty' => FALSE,
  'schema' => ['contactId', 'contributionId'],
]);
$available = $tokenProcessor->listTokens();

Note that we do not provide a literal contactId or contributionId -- these values do not exist yet. (We are acting as administrators preparing a template.) However, we will provide them when it matters - and this means various tokens will be valid. The schema gives an important hint to determine available tokens.

Appendix: Token Context

When creating a TokenProcessor and adding rows/recipients, one passes in context data. The data passed in will determine which tokens are available. These are common parameters for enabling common tokens:

Parameter	Description
activityId (int)	Required for {activity.*} tokens
contactId (int)	Required for {contact.*} tokens
contributionId (int)	Required for {contribution.*} tokens
mailingId (int)	Required for {mailing.*} tokens
mailingActionTarget (array)	Required for CiviMail {action.*} tokens
mailingJobId (int)	Required for CiviMail {action.*} tokens

Additionally, there are some generic parameters. These provide extra hints to the token-providers about what's going on.

Parameter	Description
controller (string)	Give a symbolic name for this processor, such as "CiviMail" or "ScheduledReminder". Usually, this is the name of the controller class.
smarty (bool)	Enable or disable support for Smarty directives
schema (string[])	List of fields used by the TokenProcessor. By default, this is array_keys($context). However, you may add extra fields here to promise that values will be provided in the future. See Listing Tokens.

Appendix: History

Token functionality originated in CiviMail, which focuses on writing and delivering newsletters to large constituencies. In its original form, the design placed heavy weight on:

• Performance: Divide mail-composition into batches and split the batches among parallel workers. Moreover, when processing each batch, improve efficiency by minimizing the #SQL queries - i.e. fetch all records in a batch at once, and only fetch columns which are actually used.
• Security: Do not trust email authors with a fully programmable language.
• Contact Records: The main data for mail-merge came from contact records. Other data (contribution, event, participant, membership, etc) were not applicable.
• Adaptive Text/HTML: Email messages often have two renditions, text/plain and text/html. Some tokens, such as {domain.address}, may present different formatting in each medium. Other tokens, such as {action.unsubscribe}, can even present a different user-experience.

Over time, the token functionality evolved:

• Add optional support for more powerful templates with conditions and loops (Smarty). (In the case of CiviMail, this was still disabled by default as a security consideration, but in other use-cases it might be enabled by default.)
• Add a hook for custom tokens.
• Expand to other applications, such as individual mailings, print letters, receipts for contributions, and scheduled reminders.

Up through CiviCRM v4.7, the primary interfaces for working with tokens were CRM_Utils_Token (helpers for composing messages), hook_civicrm_tokens and hook_civicrm_tokenValues (helpers for custom tokens).

CiviCRM v4.7 introduced Civi\Token\TokenProcessor and civi.token.*, which provide a more flexible way to define and process tokens. These preserve the performance, batching, and security virtues of CRM_Utils_Token and also:

• Allow more contextual information -- enabling tokens for more entities (Contribution, Participant, etc).
• Loosen the coupling between token-consumers and token-providers.
• Loosen the coupling between token-content and template-language.
• Centralize the evaluation of token language.
• Distinguish token-data which is defined with (or without) HTML content.
• Further reduce unnecessary computations (custom tokens only need to be prcoessed if they are being used)

In v4.7, the TokenProcessor and civi.token.* were initially used for Scheduled Reminders (CRM-13244), and they could be optionally used for CiviMail (by installing/configuring FlexMailer). Specific use-case have changed during 5.x, with most being converted circa v5.43.

Appendix: CRM_Utils_Token

For most of its history, CiviMail used a helper class, CRM_Utils_Token, with a number of static helper functions. This class is now deprecated and should not be used.

See also

• Custom Case Token Step-by-step guide
• hook_civicrm_tokenValues
• hook_civicrm_tokens
• hook_civicrm_container
• Symfony Event Listener Documentation
• Example Case Token extension
• Activity Token extension]