# Engine.IO Protocol This document describes the Engine.IO protocol. For a reference JavaScript implementation, take a look at [engine.io-parser](https://github.com/socketio/engine.io-parser), [engine.io-client](https://github.com/socketio/engine.io-client) and [engine.io](https://github.com/socketio/engine.io). Table of Contents: - [Revision](#revision) - [Anatomy of an Engine.IO session](#anatomy-of-an-engineio-session) - [URLs](#urls) - [Encoding](#encoding) - [Packet](#packet) - [0 open](#0-open) - [1 close](#1-close) - [2 ping](#2-ping) - [3 pong](#3-pong) - [4 message](#4-message) - [Payload](#payload) - [Transports](#transports) - [Polling](#polling) - [XHR](#xhr) - [JSONP](#jsonp) - [Server-sent events](#server-sent-events) - [WebSocket](#websocket) - [Transport fallback](#transport-fallback) - [Timeouts](#timeouts) - [Difference between v3 and v4](#difference-between-v3-and-v4) - [Difference between v2 and v3](#difference-between-v2-and-v3) ## Revision This is revision **4** of the Engine.IO protocol. The revision 2 can be found here: https://github.com/socketio/engine.io-protocol/tree/v2 The revision 3 can be found here: https://github.com/socketio/engine.io-protocol/tree/v3 ## Anatomy of an Engine.IO session 1. Transport establishes a connection to the Engine.IO URL . 2. Server responds with an `open` packet with JSON-encoded handshake data: - `sid` session id (`String`) - `pingTimeout` server configured ping timeout, used for the client to detect that the server is unresponsive (`Number`) - `pingInterval` server configured ping interval, used for the client to detect that the server is unresponsive (`Number`) 3. Client must respond to periodic `ping` packets sent by the server with `pong` packets. 4. Client and server can exchange `message` packets at will. 5. Polling transports can send a `close` packet to close the socket, since they're expected to be "opening" and "closing" all the time. Example with WebSocket connection: ``` GET /engine.io/?EIO=4&transport=websocket&sid=lv_VI97HAXpY6yYWAAAC < HTTP/1.1 101 Switching Protocols ``` WebSocket frames: ``` < 0{"sid":"lv_VI97HAXpY6yYWAAAC","pingInterval":25000,"pingTimeout":5000} => handshake > 4hello => message (not concatenated) > 4world < 2 => "ping" packet type > 3 => "pong" packet type > 1 => "close" packet type ``` Example with polling: - Request n°1 (open packet) ``` GET /engine.io/?EIO=4&transport=polling&t=N8hyd6w < HTTP/1.1 200 OK < Content-Type: text/plain; charset=UTF-8 0{"sid":"lv_VI97HAXpY6yYWAAAC","pingInterval":25000,"pingTimeout":5000} ``` Details: ``` 0 => "open" packet type {"sid":... => the handshake data ``` - Request n°2 (message in): ``` GET /engine.io/?EIO=4&transport=polling&t=N8hyd7H&sid=lv_VI97HAXpY6yYWAAAC < HTTP/1.1 200 OK < Content-Type: text/plain; charset=UTF-8 4hey ``` Details: ``` 4 => "message" packet type hey => the actual message ``` - Request n°3 (message out) ``` POST /engine.io/?EIO=4&transport=polling&t=N8hzxke&sid=lv_VI97HAXpY6yYWAAAC > Content-Type: text/plain; charset=UTF-8 4hello\x1e4world < HTTP/1.1 200 OK < Content-Type: text/plain; charset=UTF-8 ok ``` Details: ``` 4 => "message" packet type hello => the 1st message \x1e => separator 4 => "message" message type world => the 2nd message ``` ## URLs An Engine.IO url is composed as follows: ``` /engine.io/[?] ``` - The `engine.io` pathname should only be changed by higher-level frameworks whose protocol sits on top of engine's. - The query string is optional and has six reserved keys: - `transport`: indicates the transport name. Supported ones by default are `polling`, `websocket`. - `j`: if the transport is `polling` but a JSONP response is required, `j` must be set with the JSONP response index. - `sid`: if the client has been given a session id, it must be included in the querystring. - `EIO`: the version of the protocol - `t`: a hashed-timestamp used for cache-busting *FAQ:* Is the `/engine.io` portion modifiable? Provided the server is customized to intercept requests under a different path segment, yes. *FAQ:* What determines whether an option is going to be part of the path versus being encoded as part of the query string? In other words, why is the `transport` not part of the URL? It's convention that the path segments remain *only* that which allows to disambiguate whether a request should be handled by a given Engine.IO server instance or not. As it stands, it's only the Engine.IO prefix (`/engine.io`) and the resource (`default` by default). ## Encoding There's two distinct types of encodings - packet - payload ### Packet An encoded packet can be UTF-8 string or binary data. The packet encoding format for a string is as follows ``` [] ``` example: ``` 4hello ``` For binary data the packet type is not included, since only "message" packet type can include binary. #### 0 open Sent from the server when a new transport is opened (recheck) #### 1 close Request the close of this transport but does not shutdown the connection itself. #### 2 ping Sent by the server. Client should answer with a pong packet. example 1. server sends: ```2``` 2. client sends: ```3``` #### 3 pong Sent by the client to respond to ping packets. #### 4 message actual message, client and server should call their callbacks with the data. ##### example 1 1. server sends: ```4HelloWorld``` 2. client receives and calls callback ```socket.on('message', function (data) { console.log(data); });``` ##### example 2 1. client sends: ```4HelloWorld``` 2. server receives and calls callback ```socket.on('message', function (data) { console.log(data); });``` ### Payload A payload is a series of encoded packets tied together. The payload encoding format is as follows when only strings are sent and XHR2 is not supported: ``` \x1e\x1e ``` The packets are separated by the record separator ('\x1e'). More info here: https://en.wikipedia.org/wiki/C0_and_C1_control_codes#Field_separators When binary data is included in the payload, it is sent as base64 encoded strings. For the purposes of decoding, an identifier `b` is put before a packet encoding that contains binary data. A combination of any number of strings and base64 encoded strings can be sent. Here is an example of base 64 encoded messages: ``` \x1eb[...] ``` The payload is used for transports which do not support framing, as the polling protocol for example. - Example without binary: ``` [ { "type": "message", "data": "hello" }, { "type": "message", "data": "€" } ] ``` is encoded to: ``` 4hello\x1e4€ ``` - Example with binary: ``` [ { "type": "message", "data": "€" }, { "type": "message", "data": buffer <01 02 03 04> } ] ``` is encoded to: ``` 4€\x1eb4AQIDBA== with 4 => "message" packet type € \x1e => record separator b => indicates a base64 packet AQIDBA== => buffer content encoded in base64 ``` ## Transports An engine.io server must support three transports: - websocket - server-sent events (SSE) - polling - jsonp - xhr ### Polling The polling transport consists of recurring GET requests by the client to the server to get data, and POST requests with payloads from the client to the server to send data. #### XHR The server must support CORS responses. #### JSONP The server implementation must respond with valid JavaScript. The URL contains a query string parameter `j` that must be used in the response. `j` is an integer. The format of a JSONP packet. ``` `___eio[` `]("` `");` ``` To ensure that the payload gets processed correctly, it must be escaped in such a way that the response is still valid JavaScript. Passing the encoded payload through a JSON encoder is a good way to escape it. Example JSONP frame returned by the server: ``` ___eio[4]("packet data"); ``` ##### Posting data The client posts data through a hidden iframe. The data gets to the server in the URI encoded format as follows: ``` d= ``` In addition to the regular qs escaping, in order to prevent inconsistencies with `\n` handling by browsers, `\n` gets escaped as `\\n` prior to being POSTd. ### Server-sent events The client uses an EventSource object for receiving data, and an XMLHttpRequest object for sending data. ### WebSocket Encoding payloads _should not_ be used for WebSocket, as the protocol already has a lightweight framing mechanism. In order to send a payload of messages, encode packets individually and `send()` them in succession. ## Transport fallback A connection always starts with WebSocket (if supported by the client). If the connection cannot be established, the client will try to establish a SSE stream. If the connection still fails, the client will use polling as a fallback (either XHR or JSONP). ## Timeouts The client must use the `pingTimeout` and the `pingInterval` sent as part of the handshake (with the `open` packet) to determine whether the server is unresponsive. The server sends a `ping` packet. If no packet type is received within `pingTimeout`, the server considers the socket disconnected. If a `pong` packet is actually received, the server will wait `pingInterval` before sending a `ping` packet again. Since the two values are shared between the server and the client, the client will also be able to detect whether the server becomes unresponsive when it does not receive any data within `pingTimeout + pingInterval`. ## Difference between v3 and v4 - reverse ping/pong mechanism The ping packets will now be sent by the server, because the timers set in the browsers are not reliable enough. We suspect that a lot of timeout problems came from timers being delayed on the client-side. - fallback to polling instead of upgrading to WebSocket As of today, most browsers now support WebSocket: https://caniuse.com/#search=websocket Please note that this does not remove the need to use sticky-session for the polling transport. - always use base64 when encoding a payload with binary data This change allows to treat all payloads (with or without binary) the same way, without having to take in account whether the client or the current transport supports binary data or not. - use a record separator (`\x1e`) instead of counting of characters Counting characters prevented (or at least makes harder) to implement the protocol in other languages, which may not use the UTF-16 encoding. For example, `€` was encoded to `2:4€`, though `Buffer.byteLength('€') === 3`. Note: this assumes the record separator is not used in the data. The revision 4 of the protocol will be included in Socket.IO v3. ## Difference between v2 and v3 - add support for binary data v2 is included in Socket.IO v0.9, while v3 is included in Socket.IO v1/v2.