Piano Cxense SDK for Flutter

Tracks page views and DMP performance events with the Piano Cxense (Insight and DMP) APIs, and loads the content recommendations of the Content API. Events are queued in a local database and sent in the background, so they survive an application restart and a period without a network.

Installation

Add the dependencies to your pubspec.yaml:

dependencies:
  piano_common: ^1.0.4
  piano_cxense: ^1.0.4

Supported Platforms

Android, iOS and macOS. The event queue is a SQLite database, opened through sqflite, so the SDK supports the platforms that package implements.

Windows and Linux work when your application provides the database itself: add sqflite_common_ffi and install its factory before Piano.init, which is what the SDK then opens its queue through.

if (Platform.isWindows || Platform.isLinux) {
  sqfliteFfiInit();
  databaseFactory = databaseFactoryFfi;
}

The web is not supported: sqflite has no web implementation.

The queue lives in the database directory of the application, so it is as durable as the application's own storage: it survives a restart, and is gone when the application is uninstalled or its data is cleared.

The Device Clock

Everything the SDK reports is timestamped with the local clock of the device, and the X-cXense-Authentication token of an authorized request signs the moment it was made. A device whose clock is far off from real time therefore has its authorized requests — the batched performance events and every cxense.dmp query — refused by the platform, and reports the events it tracks under the wrong time. Nothing is corrected for it, which is what the Android and iOS SDKs do as well.

Since a tracked event keeps the timestamp it was tracked with, an event queued while the clock was wrong is sent with that time even once the clock is right again, and one whose timestamp is more than outdatePeriod in the past is dropped by the next flush rather than sent.

Getting Started

The Cxense SDK is a module of the Piano SDK: pass a PianoCxenseConfiguration to Piano.init and reach the client it creates through piano.cxense.

import 'package:piano_common/piano_common.dart';
import 'package:piano_cxense/piano_cxense.dart';

final piano = await Piano.init(
  endpoint: PianoEndpoint.production,
  aid: '<AID>',
  configurations: [
    PianoCxenseConfiguration(
      // Signs the authorized DMP push; see "Sending Events" below.
      userName: '<USER_NAME>',
      apiKey: '<API_KEY>',
    ),
  ],
);

final cxense = piano.cxense;

Reaching piano.cxense without that configuration — or after piano.close() — throws a StateError.

Piano.init opens the event storage and starts the dispatch loop, so the events a previous launch could not send are on their way before your application tracks anything. It therefore reports what opening that storage failed with: handle it the way you handle any other failed initialization — the instance was not created, so nothing was left open, and calling Piano.init again is what tries the storage again.

Several Instances

An application that reports to more than one Piano account runs one Piano instance per account, and every instance names itself:

final news = await Piano.init(
  endpoint: PianoEndpoint.production,
  aid: '<NEWS_AID>',
  id: 'news',
  configurations: [PianoCxenseConfiguration()],
);

final sports = await Piano.init(
  endpoint: PianoEndpoint.production,
  aid: '<SPORTS_AID>',
  id: 'sports',
  configurations: [PianoCxenseConfiguration()],
);

The client of an instance names its event storage after the identifier the instance carries, so each of them queues, merges and sends its own events: one dispatch loop never sends what the other one queued, applies the consent of the other configuration to it, or closes a database the other one is still working on. An instance finds the events of its previous launch again as long as it is created with the same identifier, so an identifier is worth keeping stable across releases.

An application that names no identifier gets default, which is the single instance case — nothing to configure there.

Configuration

Every setting is read when it is used rather than snapshotted, so changing it applies to the events tracked and the requests made from then on:

final configuration = PianoCxenseConfiguration(
  // Enriches every event with the application name and version.
  autoMetaInfoTrackingEnabled: true,
  // How often the queue is flushed, at least 10 seconds.
  dispatchPeriod: const Duration(minutes: 5),
  // How long an event that could not be sent is kept, at least 10 minutes.
  outdatePeriod: const Duration(days: 7),
  // How long a queued event stays open for merging; `null` puts no limit on it,
  // `Duration.zero` queues every tracked event on its own.
  eventsMergePeriod: null,
  // The weakest connection the queue is sent over; `none` turns the check off.
  minimumNetworkStatus: PianoCxenseNetworkStatus.none,
  persistentId: '<PERSISTED_QUERY_ID>',
);

configuration.dispatchPeriod = const Duration(minutes: 1);

minimumNetworkStatus is what keeps a flush off a connection you do not want to spend: a flush that finds a weaker one sends nothing and leaves the events queued for a later flush. The statuses are ordered by how much they carry — PianoCxenseNetworkStatus.none < gprs < mobile < wifi — so wifi sends only over Wi-Fi or a wired connection, mobile also accepts a cellular one, and gprs accepts any connection at all.

The default is none, which turns the check off: no connection is below it, so the connectivity of the device is not even read. Note that a reported connection does not guarantee the API can be reached over it — a captive portal answers a request the same way a working network does — so an event a flush could not send stays queued either way.

Report what the user has consented to with PianoCxenseConsentSettings. Every consent is denied by default, and every event carries the granted ones:

configuration.consentSettings = const PianoCxenseConsentSettings(
  consentRequired: true,
  pvAllowed: true,
  segmentAllowed: true,
  adAllowed: true,
);

While consent is required but page view tracking is not allowed, nothing is sent at all: the queued events stay queued until consent allows them, or are dropped when they outdate before that.

Tracking Page Views

await cxense.send(
  PianoCxensePageViewEvent(
    userId: '<CXENSE_USER_ID>',
    siteId: '<SITE_ID>',
    location: 'https://piano.io/article',
    referrer: 'https://piano.io/',
    pageName: 'Article',
    customParameters: [PianoCxenseCustomParameter('section', 'news')],
    externalUserIds: [PianoCxenseExternalUserId('crm', 'customer-42')],
  ),
);

A page without a URL is tracked by its content id instead, which is reported as a synthetic content URL:

await cxense.send(
  PianoCxensePageViewEvent(
    userId: '<CXENSE_USER_ID>',
    siteId: '<SITE_ID>',
    contentId: 'article-42',
  ),
);

The same page tracked twice updates the queued event rather than queueing a second one, and the request that was already registered keeps its identity. See PianoCxenseConfiguration.eventsMergePeriod for how long a queued event stays open for that.

Merging stays within one user: the queued event keeps its own rnd and timestamp, so a page view of another userId is queued on its own. Pass mergeScope when your events carry an identity Cxense does not see in them — a logged in user of your own, for instance — and page views naming different scopes are kept apart the same way.

Tracking Active Time

Track a page view with an eventId to report later how long the user spent on that page:

await cxense.send(
  PianoCxensePageViewEvent(
    userId: '<CXENSE_USER_ID>',
    siteId: '<SITE_ID>',
    location: 'https://piano.io/article',
    eventId: 'article-42',
  ),
);

// When the user leaves the page.
await cxense.trackActiveTime('article-42');

The queued page view reports the time next to its own parameters, so the API attributes it to the page view request that was registered instead of counting a second page view. Pass activeTime to report a time the application measured itself; an omitted one is the time since the page view was tracked.

Nothing is reported when no page view event was tracked under that identifier, or when the one that was has outdated meanwhile. A page view that has been sent already keeps the time locally, since a sent event is not sent again — track the time before the queue is flushed, or pass withFlush: true to send it right away.

Tracking Performance Events

await cxense.send(
  PianoCxensePerformanceEvent(
    userId: '<CXENSE_USER_ID>',
    siteId: '<SITE_ID>',
    // Prefixed by your three character customer prefix.
    origin: 'abc-app',
    eventType: 'click',
    identities: [PianoCxenseUserIdentity('cxd', '<DEVICE_ID>')],
    customParameters: [PianoCxenseCustomParameter('campaign', 'sale')],
  ),
);

Pass the rnd of a page view event as prnd to link a performance event to the page view it happened on.

Sending Events

A tracked event is queued and sent by the dispatch loop, which flushes the queue ten seconds after Piano.init and every PianoCxenseConfiguration.dispatchPeriod from then on. The first flush therefore sends what a previous launch left queued, whether the application has tracked another event by then or not. Pass withFlush: true to flush as soon as the event was queued:

// Queues the event and awaits the flush it asks for.
await cxense.send(event, withFlush: true);

Page view events are reported to the Insight API. Performance events are pushed to the DMP API in a single authorized batch when the configuration carries both a userName and an apiKey, and are reported one by one with the persistentId of a persisted query otherwise.

An event is only retired once the API has accepted it, so an event that could not be sent is retried by the next flush until it is PianoCxenseConfiguration.outdatePeriod old.

A flush reads a bounded number of events, so a queue that could not be sent for a long time does not have to be read at once; the events it leaves behind are sent by the following flushes, oldest first. Asking for a flush therefore asks for the queue to be sent rather than for one event to go out now: an event tracked behind a queue that could not be sent for a while is not necessarily part of the flush withFlush makes for it, and waits for one of the following ones. A flush you asked for reports what it failed with — send(..., withFlush: true) throws it — while one the dispatch loop made logs it through the logger of the Piano instance instead, because nothing asked for it.

flushEventsQueue sends the queue without tracking an event first, which is what send(..., withFlush: true) does on top of queueing one:

// Awaits the flush and throws what it failed with.
await cxense.flushEventsQueue();

eventsQueueStatus reports what the queue holds, split into the events that were sent and the ones that were not. Each of them carries the eventId the event was tracked with, or null when it was tracked without one:

final status = await cxense.eventsQueueStatus;
debugPrint('${status.notSentEvents.length} events are still queued');

A sent event stays part of the answer until it is outdatePeriod old, because that is when it is deleted; two events that were merged into one are reported as one event.

Set PianoCxenseConfiguration.minimumNetworkStatus to keep a flush from sending over a connection you do not want to spend; see the configuration above.

Querying the DMP API

cxense.dmp reads the segments, the profiles and the external data the platform holds. Unlike a tracked event, a query is made when you ask for it and its answer is returned; every query is authorized, so the configuration has to carry both a userName and an apiKey — otherwise the query throws a PianoCxenseNotAuthorizedException before it is made.

Segments

// The segments of a site group, optionally narrowed to some users.
final lookup = await cxense.dmp.lookupSegments(
  siteGroupIds: ['<SITE_GROUP_ID>'],
  identities: [PianoCxenseUserIdentity('cxd', '<DEVICE_ID>')],
);
for (final segment in lookup.segments) {
  print('${segment.id} is a ${segment.type.name} segment');
}

// The segments one user belongs to.
final segments = await cxense.dmp.getUserSegments(
  identities: [PianoCxenseUserIdentity('cxd', '<DEVICE_ID>')],
  siteGroupIds: ['<SITE_GROUP_ID>'],
);

Both segment queries need the user's segment consent while consent is required: they throw a PianoCxenseConsentRequiredException unless PianoCxenseConsentSettings.segmentAllowed is granted. The profile and external data queries are not gated.

User profiles

final user = await cxense.dmp.getUser(
  identity: PianoCxenseUserIdentity('cxd', '<DEVICE_ID>'),
  // The interest groups to report, and the identities of the same user.
  groups: ['category'],
  identityTypes: ['cid'],
);
for (final profile in user.profiles) {
  for (final group in profile.groups) {
    print('${group.group}: ${group.weight}');
  }
}

External data

External data is your own data about a user, stored on the platform as groups of typed items:

await cxense.dmp.setUserExternalTypedData(
  PianoCxenseUserExternalTypedData(
    identity: PianoCxenseUserIdentity('cid', 'customer-42'),
    items: [
      PianoCxenseExternalTypedItem(
        'loyalty',
        const PianoCxenseTypedItem.string('gold'),
      ),
      PianoCxenseExternalTypedItem('visits', PianoCxenseTypedItem.number(17)),
    ],
  ),
);

final data = await cxense.dmp.getUserExternalTypedData(
  type: 'cid',
  id: 'customer-42',
);

await cxense.dmp.deleteUserExternalData(
  PianoCxenseUserIdentity('cid', 'customer-42'),
);

The item groups are prefixed with the identity type, so the loyalty above is stored as cid-loyalty. An item the user already has is overwritten and the rest of their profile is left alone, so one item can be changed without sending the whole profile back.

Identity mappings

Map your own identifier to the Cxense one to reach the same user under both:

await cxense.dmp.addUserExternalLink(
  cxenseId: '<CXENSE_ID>',
  identity: PianoCxenseUserIdentity('cid', 'customer-42'),
);

final identity = await cxense.dmp.getUserExternalLink(
  cxenseId: '<CXENSE_ID>',
  type: 'cid',
);

Content Recommendations

cxense.content loads what a content widget recommends and reports what the user did with it. These are the public endpoints of the API: the widget id is what authorizes them, so they need neither a userName nor an apiKey.

final items = await cxense.content.loadWidgetRecommendations(
  widgetId: '<WIDGET_ID>',
  // The page the recommendations are shown on, so they fit its content.
  context: PianoCxenseWidgetContext(
    url: 'https://piano.io/article',
    referrer: 'https://piano.io/',
    keywords: ['news'],
  ),
  // The user they are loaded for, and the categories they prefer.
  user: PianoCxenseContentUser.withUserId(
    '<CXENSE_USER_ID>',
    likes: PianoCxenseUserPreference(categories: ['sport'], boost: 1.5),
  ),
  // The `rnd` of the page view event of the page above.
  prnd: '<PAGE_VIEW_RND>',
);

for (final item in items) {
  print('${item.title} — ${item.url}');
  // Whatever else the widget template reports, keyed by field name.
  print(item.properties['description']);
}

The granted consents are reported with the request and the platform decides what it answers for them, so nothing is refused before it is made.

Report a click on a recommendation and how long the user saw it, so the platform learns which of its recommendations worked:

// Reported to the click url of the item, wherever it points.
await cxense.content.trackItemClick(item);

// Or with the click url itself.
await cxense.content.trackClick(item.clickUrl!);

await cxense.content.reportWidgetVisibilities([
  PianoCxenseImpression(clickUrl: item.clickUrl!, seconds: 3),
]);

trackItemClick throws an ArgumentError for an item carrying no click url, because there is nothing to report the click to.

Persisted Queries

A persisted query is a request the platform has been configured to accept without credentials, which is how to reach an endpoint this SDK does not wrap:

// GET https://api.cxense.com/profile/user/segment?persisted=<QUERY_ID>
final answer = await cxense.executePersistedQuery(
  url: 'profile/user/segment',
  persistentQueryId: '<QUERY_ID>',
);

// The same endpoint as a POST, with the body the query expects.
final segments = await cxense.executePersistedQuery(
  url: 'profile/user/segment',
  persistentQueryId: '<QUERY_ID>',
  data: {
    'identities': [
      {'type': 'cid', 'id': 'customer-42'},
    ],
  },
);

A relative url is resolved against https://api.cxense.com, and an absolute one is used as it is. Omitting data makes a GET and passing it makes a POST. The answer is the JSON object the endpoint returned, because a persisted query has no fixed shape to parse it into.

persistentQueryId is the query the request is run as — not PianoCxenseConfiguration.persistentId, which is the query the tracked performance events are pushed with.

Libraries

piano_cxense