Extensions

JavaScript CometD API: Extensions

The JavaScript CometD implementation has the capability to add/remove extensions.
An extension is a function that is being called to give the chance to modify the message just before it is being sent (an outgoing extension) or just after it is being received (an incoming extension).

An extension normally adds fields to the message being sent or received in the ext object defined by the Bayeux specification (see here).

An extension is not a way to add business fields to a message, but rather a way to process all messages, including the meta messages used by the Bayeux protocol.

Extensions are normally setup on both the client and the server, since fields added by the client normally needs a special processing by the server; it may be possible that an extension is only client-side or only server-side, though.
Most of the times, however, they are needed in both client and server, and when the extension does not behave as expected, it's normally because the extension on one of the two sides is missing.

The JavaScript CometD Extensions are described in the next sections, and follow the same pattern used by the portable JavaScript CometD implementation: a portable implementation of the extension with bindings for the specific JavaScript toolkit, currently Dojo and jQuery.

Writing the Extension

An extension is a JavaScript object with 4 optional methods:

outgoing(message), called just before a message is being sent
incoming(message), called just after a message is received
registered(name, cometd), called when the extension is registered
unregistered(), called when the extension is unregistered

All 4 methods are optional, or there can be only one, or maybe two, three or all of them. If they are present, they will be invoked at the proper time.

Writing an extension that logs and counts the long polls is quite easy: we need a reference to the cometd object, that has the logging methods, and we need only the outgoing extension method:

var LoggerExt = function()
{
    var _cometd;
    var _counter;

    this.registered = function(name, cometd)
    {
        // Store the cometd object reference
        _cometd = cometd;
    };

    this.outgoing(message)
    {
        if (message.channel == '/meta/connect')
        {
            // Log the long poll
            _cometd._info('bayeux connect');

            // Count the long polls
            if (!message.ext) message.ext = {};
            if (!message.ext.logger) message.ext.logger = {};
            if (!message.ext.logger.counter) message.ext.logger.counter = 0;
            message.ext.logger.counter = ++_counter;
        }
    };
};

Note that also meta messages are passed to the extension methods; you normally have to filter the messages that the extension method receives by looking at the channel or at some other message value.
Note that the message can be modified by adding fields, normally in the ext field.

Note
Be careful to not overwrite the ext field that it may have been set by other extensions: check if it's present first.
It is also a good practice to group your extension fields so that there is no clash with other extensions (in the example above the only field - counter - is "grouped" in the message.ext.logger object).

The outgoing() and incoming() methods can avoid to return something, or return the message itself (or another message object). This means that the message has been processed by the extension and can therefore be processed by other extensions, if present, or processed by the implementation (either be sent to the server - for outgoing extensions - or be notified to listeners - for incoming extensions).
If null is returned by the extension method, this means that the processing should stop: the message will not be processed by other extensions and will not be further processed (therefore it will neither be sent to the server nor be notified to listeners).

Registering the Extension

The JavaScript CometD API defines 3 methods to manage extensions:

registerExtension(name, extension), to register an extension with the given name
unregisterExtension(name), to unregister the extension previously registered with the given name
getExtension(name), to obtain a reference to the extension previously registered with the given name

Following the example above, we can register the extension like this:

cometd.registerExtension('loggerExt', new LoggerExt());

From now on, the meta connect messages will be modified to carry the counter from the example extension above.

Unregistering the extension is similar:

cometd.unregisterExtension('loggerExt');

It is not possible to register 2 extensions under the same name.

You can register more than one extension, and they will be applied following the registration order: outgoing extensions methods will be called in registration order and, by default, incoming registration methods will be called in reverse registration order. See also the reverseIncomingExtensions configuration parameter in the configuration section.
For example, if you register extA and extB, then for outgoing messages the methods called are: extA.outgoing() and then extB.outgoing(), while for incoming messages the methods called are extB.incoming() and then extA.incoming().

Extension Exception Handling

While it is normally good practice to catch exceptions within extension functions, sometimes this is tedious to code, or there is no control about the quality of the extension (e.g. it's a third party extension).
The JavaScript CometD API provides a way to define the global extension exception handler that is invoked every time an extension throws an exception (for example, calling a method on an undefined object, etc.):

cometd.onExtensionException = function(exception, extensionName, outgoing, message)
{
    // Uh-oh, something went wrong, disable this extension
    // Object "this" points to the CometD object
    this.unregisterExtension(extensionName);

    // If the message is going to the server, add the error to the message
    if (outgoing)
    {
        // Assume we have created the message structure below
        var badExtension = message.ext.badExtensions[extensionName];
        badExtension.exception = exception;
    }
}

Be very careful to use the CometD object to publish messages within the extension exception handler, or you may end up in an infinite loop (the publish message is processed by the extensions, which may fail and call again the extension exception handler).
If the extension exception handler itself throws an exception, this exception is logged at level "info" and the CometD implementation will not break.
Note that a similar mechanism exists for listeners and subscribers, see here.

The following sections will explain in detail the usage of the provided extensions.

Acknowledge

Acknowledge Extension

The acknowledged messages extension provides reliable ordered messaging to the Bayeux protocol.
This extension requires both a client-side extension and a server-side extension. The server-side extension is available in Java.

Enabling the Server-side Extension

To enable support for acknowledged messages, the extension must be added to the org.cometd.Bayeux instance during initialization:

bayeux.addExtension(new org.cometd.server.ext.AcknowledgedMessagesExtension());

The AcknowledgedMessageExtension is a per-server extension that monitors handshakes from new clients, looking for clients that also support the acknowledged message extension and then adds the AcknowledgedMessagesClientExtension to each client during the handshake.

Once added to a client, the AcknowledgedMessagesClientExtension prevents messages being delivered on any request other than a long poll request.
This prevents the possibility of out of order delivery.
The extension also maintains a list of non-acknowledged messages and intercepts the traffic on the /meta/connect channel to insert and check acknowledge IDs.

Enabling the Client-side Extension

The client side extension binding for Dojo is provided by dojox/cometd/ack.js and it is sufficient to use Dojo's dojo.require mechanism:

dojo.require("dojox.cometd.ack");

The client side extension binding for jQuery is provided by the file jquery.cometd-ack.js. This file must be included in the HTML page via the <script> tag:

<script type="text/javascript" src="AckExtension.js"></script>
<script type="text/javascript" src="jquery.cometd-ack.js"></script>

In both Dojo and jQuery extension bindings, the extension is registered on the default cometd object under the name "ack".

Furthermore, the extension may be programmatically disabled/enabled before initialization by setting the ackEnabled boolean field on the cometd object:

// Disables the ack extension during handshake
cometd.ackEnabled = false;
cometd.init(cometdURL);

Acknowledge Extension Details

In order to enable message acknowledgement, both client and server must indicate that they support message acknowledgement.
This is negotiated during handshake. On handshake, the client sends {"ext":{"ack": "true"}} to indicate that it supports message acknowledgement.
If the server also supports message acknowledgment, it likewise replies with {"ext":{"ack": "true"}}.

The extension does not insert ack IDs to every message, as this would impose a significant burden on the server for messages sent to multiple clients (which would need to be reserialized to json for each client). Instead the ack ID is inserted in the ext field of the /meta/connect messages that are associated with message delivery. Each /meta/connect request contains the ack ID of the last received ack response: "ext":{"ack": 42}. Similarly, each /meta/connect response contains an ext ack ID that uniquely identifies the batch of responses sent.

If a /meta/connect message is received with an ack ID lower that any unacknowledged messages held by the extension, then these messages are requeued prior to any more recently queued messages and the /meta/connect response sent with a new ack ID.

Demo

There is an example of acknowledged messages in the Dojo chat demo that comes bundled with the Cometd distribution.
The example can also be run by building the Cometd project.

To run the demo, follow these steps:

$ cd cometd-demo
$ mvn jetty:run

Then point your browser to http://localhost:8080/dojo-examples/chat/ and make sure to check "Enable reliable messaging".

Use two different browsers instances to begin a chat session, then briefly disconnect one browser from the network (you can do this by setting the "work offline" feature).
While one browser is disconnected, type some chat in the other browser and this will be received when the disconnected browser is reconnected to the network.

Note that if the disconnected browser is disconnected for in excess of maxInterval (default 10s), then the client will be timed out and the unacknowledged queue discarded.

Timestamp

Timestamp Extension

The timestamp extension add a timestamp to the message object, for every message sent by the client and/or server.
It is a non standard extension because it does not add the additional fields to the ext field, but to the message object itself.
This extension requires both a client-side extension and a server-side extension. The server-side extension is available in Java.

Enabling the Server-side Extension

To enable support for timestamped messages, the extension must be added to the org.cometd.Bayeux instance during initialization:

bayeux.addExtension(new org.cometd.server.ext.TimestampExtension());

Enabling the Client-side Extension

The client side extension binding for Dojo is provided by dojox/cometd/ack.js and it is sufficient to use Dojo's dojo.require mechanism:

dojo.require("dojox.cometd.timestamp");

The client side extension binding for jQuery is provided by the file jquery.cometd-timestamp.js. This file must be included in the HTML page via the <script> tag:

<script type="text/javascript" src="jquery.cometd-timestamp.js"></script>

In both Dojo and jQuery extension bindings, the extension is registered on the default cometd object under the name "timestamp".

Timesync

Timesync Extension

The timesync extension uses the messages exchanged between a client and a server to calculate the offset between the client's clock and the server's clock.
This is independent from the timestamp extension, which uses the local clock for all timestamps.
This extension requires both a client-side extension and a server-side extension. The server-side extension is available in Java.

Enabling the Server-side Extension

To enable support for acknowledged messages, the extension must be added to the org.cometd.Bayeux instance during initialization:

bayeux.addExtension(new org.cometd.server.ext.TimesyncExtension());

Enabling the Client-side Extension

The client side extension binding for Dojo is provided by dojox/cometd/timesync.js and it is sufficient to use Dojo's dojo.require mechanism:

dojo.require("dojox.cometd.timesync");

The client side extension binding for jQuery is provided by the file jquery.cometd-timesync.js. This file must be included in the HTML page via the <script> tag:

<script type="text/javascript" src="jquery.cometd-timesync.js"></script>

In both Dojo and jQuery extension bindings, the extension is registered on the default cometd object under the name "timesync".

Timesync Extension Details

The timesync extension allows the client and server to exchange time information on every handshake and connect message so that the client may calculate an approximate offset from it's own clock epoch to that of the server.
The algorithm used is very similar to the NTP algorithm.

With each handshake or connect, the extension sends timestamps within the ext field like:

{ext:{timesync:{tc:12345567890,l:23,o:4567},...},...}

where:

tc is the client timestamp in ms since 1970 of when the message was sent
l is the network lag that the client has calculated
o is the clock offset that the client has calculated

The accuracy of the offset and lag may be calculated with tc-now-l-o, which should be zero if the calculated offset and lag are perfectly accurate.

A Bayeux server that supports timesync, should respond only if the measured accuracy value is greater than accuracy target.
The response will be an ext field like:

{ext:{timesync:{tc:12345567890,ts:1234567900,p:123,a:3},...},...}

where:

tc is the client timestamp of when the message was sent
ts is the server timestamp of when the message was received
p is the poll duration in ms - ie the time the server took before sending the response
a is the measured accuracy of the calculated offset and lag sent by the client

On receipt of the response, the client is able to use current time to determine the total trip time, from which p is subtracted to determine an approximate two way network traversal time. Thus:

lag = (now-tc-p)/2
offset = ts-tc-lag

In order to smooth over any transient fluctuations, the extension keeps a sliding average of the offsets received.
By default this is over 10 messages, but this can be changed by passing a configuration object during the creation of the extension:

// Unregister the default timesync extension
cometd.unregisterExtension('timesync');

// Re-register with different configuration
cometd.registerExtension('timesync', new org.cometd.TimeSyncExtension({ maxSamples: 20 }));

The client-side timesync extension also exposes several methods to deal with the result of the time synchronization:

getNetworkLag(), to obtain the calculated network latency between client and server
getTimeOffset(), to obtain the offset between the client's clock and the server's clock in ms
getServerTime(), to obtain the server's time
setTimeout(), to schedule a function to be executed at a certain server time

Reload

Reload Extension

The reload extension allows a page to be loaded (or the same page to be reloaded) without having to re-handshake in the new (or reloaded) page, therefore resuming the existing CometD connection.
This extension requires only the client-side extension.

Enabling the Client-side Extension

The client side extension binding for Dojo is provided by dojox/cometd/reload.js and it is sufficient to use Dojo's dojo.require mechanism:

dojo.require("dojox.cometd.reload");

The client side extension binding for jQuery is provided by the file jquery.cometd-reload.js. This file must be included in the HTML page via the <script> tag, along with the jQuery cookie plugin and the reload extension implementation:

<script type="text/javascript" src="jquery.cookie.js"></script>
<script type="text/javascript" src="ReloadExtension.js"></script>
<script type="text/javascript" src="jquery.cometd-reload.js"></script>

In both Dojo and jQuery extension bindings, the extension is registered on the default cometd object under the name "reload".

Reload Extension Details

The reload extension allows a page to be reloaded and an existing CometD connection resumed by the reloaded page.

To activate the reload extension, the page must call cometd.reload() just before the page is reloaded (for example, after a link is clicked or on the page unload event).
This saves a short lived cookie with the connection details.
The new page still needs to call subscribe to the channels it is interested in to register the callbacks, since the client state has been completely reset due to the reload.

An example simple usage is:

<html>
    <head>
        <script type="text/javascript" src="dojo/dojo.js"></script>
        <script type="text/javascript">
            dojo.require("dojox.cometd");
            dojo.require("dojox.cometd.reload");

            dojox.cometd.init({ url: "/context/cometd", logLevel: "info" });
            dojox.cometd.subscribe("/some/channel", function() { ... });
            dojox.cometd.subscribe("/some/other/channel", function() { ... });
            
            // On page unload, call dojox.cometd.reload()
            dojo.addOnUnload(dojox.cometd, "reload");
        </script>
    </head>
    <body>
    ...
    </body>
</html>