CometD 2 Concepts

CometD 2 Concepts

The CometD project implements various Comet techniques to provide a scalable web messaging system, one that can run over HTTP or over other emerging web protocol such as WebSocket.

Definitions

Let's define the client as the entity that initiates a connection, and the server as the entity that accepts the connection.
The connection established is persistent - that is, it remains open until either side decides to close it.

Typical clients are browsers (after all, we are in a web environment), but may also be other applications such as Java applications, browser plugin applications (such as Silverlight), or scripts in any scripting language.

Depending on the Comet technique employed, a client may open more than one physical connection to the server, but we can assume there exists only one logical conduit between the client and the server.

The CometD project uses the Bayeux protocol to exchange information between the client and the server.
The unit of information exchanged is a Bayeux message and it is formatted in JSON.
A message contains several fields, some of which are mandated by the Bayeux protocol, and others may be added by applications.
A field is a key/value pair, and saying that a message has a foo field means that the message has a field whose key is the string "foo".

All messages that are exchanged between client and server have a channel field.

There are 3 types of channels: meta channels, service channels and broadcast channels.
Meta channels are created by the Bayeux implementation, and applications cannot create new meta channels.
Service channels and broadcast channels are created by the application; they can be created at any time, and as many as necessary.
Channels are represented by a string and are similar to a URL path, for example: /meta/handshake, /service/game or /foo/bar.
A channel that starts with /meta/ is a meta channel, a channel that starts with /service/ is a service channel, and all other channels are broadcast channels.

A message whose channel field is a meta channel is referred to as a meta message, and similarly there are service messages and broadcast messages.

The High Level View

CometD implements a web messaging system, in particular a web messaging system based on the publish/subscribe paradigm.

In a publish/subscribe messaging system publishers send messages, and those messages are characterized in classes. Subscribers express their interest in one or more classes of messages, and receive only messages that match the interest they subscribed to. Senders, in general, have no idea which recipient or how many recipients will receive the messages they publish.

In CometD, the characterization of messages in classes is given by the channel field.
The channel is a central concept in CometD: publishers publish messages to channels, and subscribers subscribe to channels to receive messages.
We will see how this is strongly reflected in the CometD APIs.

One of the features of messaging systems is their topology, and CometD implements the hub-spoke topology.
In the default configuration, this means that there is one central server (the hub) and all clients are connected to that server via conduit links (the spokes).

In CometD, the server receives messages from publishers and, if the message's channel is a broadcast channel, re-routes the messages to interested subscribers.
Meta messages and service messages are treated specially by the CometD server, and are not re-routed to any subscriber (in fact, by default it is forbidden to subscribe to meta channels, and it is a no-operation to subscribe to service channels).

For example, let's imagine that clientAB is subscribed to channels /A and /B, and clientB is subscribed to channel /B.
If a publisher publishes a message on channel /A, only clientAB will receive it. On the other hand, if a publisher publishes a message on channel /B, both clientAB and clientB will receive the message. Furthermore, if a publisher publishes a message on channel /C, neither clientAB nor clientB will receive the message, that will just end its journey on the server.
Re-routing broadcast messages is the default behavior of the server, and it does not need any application code to perform the re-routing.

Looking from high level then, you will see messages flowing back and forth among clients and server through the conduits.
A single broadcast message may arrive to the server and be re-routed to all clients; you can imagine that when it arrives on the server, the message is copied and that a copy is sent to each client (although, for efficiency reasons, this is not exactly what happens). If the sender is also subscribed to the channel it published the message to, it will receive a copy of the message back.

There are open questions, though:
What an application can do with meta messages and service messages, if their journey ends on the server ?
How can a client communicate with another, specific, client ?
Can the server be a publisher too ?

To answer these and others questions, we need to take a closer look at how CometD works. Read on.

A Lower Level View

In the following sections we will take a deeper look at how the CometD implementation works.

It should be clear by now that CometD, at its heart, is a client/server system that communicates via a protocol, the Bayeux protocol.

In the CometD implementation, the client/server communication is captured by the half-object plus protocol pattern: when a half-object on the client establishes a communication conduit with the server, its correspondent half-object is created on the server, and the two may - logically - communicate.
In CometD, a variation of this pattern is used, because there is the need to abstract the transport used to carry messages to and from the server.
The transport can be based on the HTTP protocol, but in recent CometD versions also on the WebSocket protocol (and more transports can be plugged in).

In broad terms, the "client" is composed of the client half-object (represented by the org.cometd.Cometd object in JavaScript, and by the org.cometd.bayeux.client.ClientSession class in Java) and of the client transport (represented by the org.cometd.Transport object in JavaScript and by the org.cometd.client.transport.ClientTransport in Java).
The "server" is a more complex entity represented by an instance of org.cometd.bayeux.server.BayeuxServer, and composed of server transports (represented by the org.cometd.bayeux.server.ServerTransport class), of a message processing component, of a repository of server-side half-objects (represented by org.cometd.bayeux.server.ServerSession) and of a repository of channels (represented by org.cometd.bayeux.server.ServerChannel).

Message Flow

Client half-objects delegate to a client transport the delivery of messages; the client transport is responsible to establish the conduit with the server, and to send messages on that conduit.
On the server, it is the server transport that receives the messages.
The server transport delegates the processing of the messages to the message processing component that processes incoming messages, generates a message replies, and delegates back to the server transport the delivery of the replies to the client.

In the figure above, you can see that the conduit links the client transport and the server transport, and that the ServerSession may additionally participate in sending (only sending, and not directly receiving) messages to the client (again via the server transport).

TODO: explain how broadcast message flow works, but this requires to introduce the concept of queue in server session

Bayeux Protocol

A client communicates with the server by exchanging Bayeux messages.

The Bayeux protocol requires that the first message sent by a new client is a handshake message (a message sent on /meta/handshake channel).
On the server, if the processing of the incoming handshake message is successful, then BayeuxServer creates the server-side half object instance (a ServerSession) that represents, on the server, the client that initiated the handshake.
When the processing of the handshake is completed, the server sends back a reply to the client.

The client processes the handshake message reply, and if it is successful, starts - under the covers - a heartbeat mechanism with the server, by exchanging connect messages.
The details of this heartbeat mechanism depend on the client transport used, but can be seen as the client sending a connect message and expecting a reply after some time (when using HTTP transports, the heartbeat mechanism is also known as "long-polling").
The heartbeat mechanism allows a client to detect if the server is gone (the client will not receive the connect message reply from the server), and allows the server to detect if the client is gone (the server will not receive the connect message request from the client).

Connect messages continue to flow between client and server until either side decides to disconnect by sending a disconnect message (a message sent on the /meta/disconnect channel).