diff --git a/packages/livedata/DDP.md b/packages/livedata/DDP.md new file mode 100644 index 0000000000..3ee96c6d7f --- /dev/null +++ b/packages/livedata/DDP.md @@ -0,0 +1,207 @@ +# DDP Specification + +DDP is a protocol between a client and a server that supports two operations: + + * Remote procedure calls by the client to the server + * Transferring documents and keeping them updated, from the server to the + client, based on a set of subscriptions specified by the client. + +This document specifies the version "pre1" of DDP. It's a rough description of +the protocol and not intended to be entirely definitive. + +## General Message Structure: + +DDP may use either SockJS or Websockets as a lower-level message transport. + +DDP messages are JSON objects, with some fields specified to be EJSON. Each one +has a `msg` field that specifies the message type, as well as other fields +depending on message type. + +## Establishing a DDP Connection: + +### Messages: + + * `connect` (client -> server) + - `session`: string (if trying to reconnect to an existing DDP session) + - `version`: string (the proposed protocol version) + - `support`: array of strings (protocol versions supported by the client, in order of preference) + * `connected` (server->client) + - `session`: string (an identifier for the DDP session) + * `failed` (server->client) + - `version`: string (a suggested protocol version to connect with) + +### Procedure: + +The server may send an initial message which is a JSON object lacking a `msg` +key. If so, the client should ignore it. The client does not have to wait for +this message. (This message is used to help implement hot code reload over our +SockJS transport. It is currently sent over websockets as well, but probably +should not be.) + + * The client sends a `connect` message. + * If the server can speak the `version` of the protocol specified in the + `connect` message, it sends back a `connected` message. + * Otherwise the server sends back a `failed` message with a version of DDP it + would rather speak, informed by the `connect` message's `support` field, and + closes the underlying socket. + * The client is then free to attempt to connect again speaking a different + version of DDP. The client may optimistically send more messages after the + `connect` message, assuming that the server will support the proposed + protocol version. If the server does not support that version, it must ignore + those additional messages. + +## Managing Data: + +### Messages: + + * `sub` (client -> server): + - `id`: string (an arbitrary client-determined identifier for this subscription) + - `name`: string (the name of the subscription) + - `params`: optional array of EJSON items (parameters to the subscription) + * `unsub` (client -> server): + - `id`: string (the id passed to 'sub') + * `nosub` (server -> client): + - `id`: string (the id passed to 'sub') + * `error`: optional Error (an error raised by the subscription as it concludes, or sub-not-found) + * `added` (server -> client): + - `collection`: string (collection name) + - `id`: string (document ID) + - `fields`: optional object with EJSON values + * `changed` (server -> client): + - `collection`: string (collection name) + - `id`: string (document ID) + - `fields`: optional object with EJSON values + - `cleared`: optional array of strings (field names to delete) + * `removed` (server -> client): + - `collection`: string (collection name) + - `id`: string (document ID) + * `ready` (server -> client): + - `subs`: array of strings (ids passed to 'sub' which have sent their initial batch of data) + * `addedBefore` (server -> client): + - `collection`: string (collection name) + - `id`: string (document ID) + - `fields`: optional object with EJSON values + - `before`: string or null (the document ID to add the document before, or null to add at the end) + * `movedBefore` (server -> client): + - `collection`: string + - `id`: string (the document ID) + - `before`: string or null (the document ID to move the document before, or null to move to the end) + +### Procedure: + + * The client specifies sets of information it is interested in by sending `sub` messages to the server. + + * At any time, but generally informed by the `sub` messages, the server can + send data messages to the client. Data consist of `added`, `changed`, and + `removed` messages. These messages model a local set of data the client + should keep track of. + + - An `added` message indicates a document was added to the local set. The ID + of the document is specified in the `id` field, and the fields of the + document are specified in the `fields` field. Minimongo interperets the + string id field in a special way that transforms it to the _id field of + Mongo documents. + + - A `changed` message indicates a document in the local set has new values + for some fields or has had some fields removed. The `id` field is the ID of + the document that has changed. The `fields` object, if present, indicates + fields in the document that should be replaced with new values. The + `cleared` field contains an array of fields that are no longer in the + document. + + + - A `removed` message indicates a document was removed from the local + set. The `id` field is the ID of the document. + + * If a collection is ordered, the `added` message is replaced by `addedBefore`, + which additionally contains the ID of the document after the one being added + in the `before` field. If the document is being added at the end, `before` + is set to null. For a given collection, the server should only send `added` + messages or `addedBefore` messages, not a mixture of both, and should only + send `movedBefore` messages for a collection with `addedBefore` messages. + + * The client maintains one set of data per collection. Each subscription does + not get its own datastore, but rather overlapping subscriptions cause the + server to send the union of facts about the one collection's data. For + example, if subscription A says document `x` has fields `{foo: 1, bar: 2}` + and subscription B says document `x` has fields `{foo: 1, baz:3}`, then the + client will be informed that document `x` has fields `{foo: 1, bar: 2, baz: + 3}` + + * When one or more subscriptions have finished sending their initial batch of + data, the server will send a `ready` message with their IDs. + +## Remote Procedure Calls: + +### Messages: + + * `method` (client -> server): + - `method`: string (method name) + - `params`: optional array of EJSON items (parameters to the method) + - `id`: string (an arbitrary client-determined identifier for this method call) + * `result` (server -> client): + - `id`: string (the id passed to 'method') + - `error`: optional Error (an error thrown by the method (or method-not-found) + - `result`: optional EJSON item (the return value of the method, if any) + * `updated` (server -> client): + - `methods`: array of strings (ids passed to 'method', all of whose writes + have been reflected in data messages) + +### Errors: + +Errors appear in `result` and `nosub` messages in an optional error field. An +error is an Object with the following fields: + + * `error`: number + * `reason`: optional string + * `details`: optional string + +Such an Error is used to represent errors raised by the method or subscription, +as well as an attempt to subscribe to an unknown subscription or call an unknown +method. + +Other erroneous messages sent from the client to the server can result in +receiving a top-level `msg: 'error'` message in response. These conditions +include: + + * sending messages which are not valid JSON objects + * unknown `msg` type + * other malformed client requests (not including required fields) + * sending anything other than `connect` as the first message, or sending + `connect` as a non-initial message + +The error message contains the following fields: + + * `reason`: string describing the error + * `offendingMessage`: if the original message parsed properly, it is included + here + +## Appendix: EJSON + +EJSON is a way of embedding more than the built-in JSON types in JSON. It +supports all types built into JSON as plain JSON, plus the following: + +**Dates:** + + {"$date": MILLISECONDS_SINCE_EPOCH} + +**Binary data:** + + {"$binary": BASE_64_STRING} + +Escaped things that might otherwise look like EJSON types: + + {"$escape": THING} + +For example, here is the JSON string `{$date: 10000}` stored in EJSON: + + {"$escape": {"$date": 10000}} + +Note that escaping only causes keys to be literal for one level down; you can +have further EJSON inside. + +**User-specified types:** + + {"$type": TYPENAME, "$value": VALUE} + +Implementations of EJSON should try to preserve key order where they can.