Replay Processing

To provide message replay capability, each ITC channel has a separate set of gateway instances and multicast addresses to replay requested data.  If a sequence number gap is detected, a replay request can be initiated for message recovery. The requested data for the channel(s) are transmitted through the appropriate replay gateway and multicast address.

To request a market data replay, a separate TCP connection is established to the CME Group replay gateway. A request message is made to specify the desired sequence gap and channel.  After the request is made, CME Group will transmit a Replay Response message to communicate the status of the request and then will terminate the TCP connection. A separate connection must be made for each request.

The CME Group replay gateway will authenticate the user connection and validate the replay request for a valid channel value and sequence range. CME Group has specified a maximum sequence range of 2,000 messages for any given request and a maximum of 15 requests per second. Only data for the current trading session is guaranteed for message replay availability. Client applications should cache sequence state and historical data to mitigate the amount of data recovered

To mitigate the total message rate from replay requests initiated across multiple channels, the CME Group replay gateway instances throttle message replay.  The replay channel IGMP join should be left after the replay is finished. CME Group recommends that developers externalize configurable parameters to manage the number of concurrent requests and delay between requests to be able to tailor the application to specific bandwidth configurations.

Replay Request

When initiating a replay request, the MDP client application initiates a TCP connection to the CME Group MDP Gateway.  After establishing a socket connection, a replay request is sent to the CME MDP Gateway and a response message is sent with information indicating the success or failure of the request.

To obtain a username and password for the ITC replay, contact Global Account Management (GAM).

The request and response message formats are detailed below:

Request

Example:

User=CME{SOH}Password=***{SOH}RequestType=REPLAY{SOH}Begin=1{SOH}End=100{SOH}Channel=1{SOH}

Replay Messages

With each request to replay missed or lost messages, a system message is sent preceding the message replay to alert applications of the dataset. System messages must be used to process replay requests efficiently.

Since system messages are sent artificially within a replay channel, all system messages have a sequence number of ‘0’.   Any message received on a replay channel with a sequence number of ‘0’ can be assumed to be a system message.  System messages are transmitted compressed following the standard compression.

System messages have a header and a message body formatted as follows:

Replay Batching

The CME Group  MDP replay mechanism leverages a batching algorithm to process and respond to replay requests more efficiently.  Each group of requests within a channel is analyzed for sequence number overlap to combine similar requests into one replay stream.  A reasonable gap is used to group requests that may be close to overlapping.

In the following example, five users for the same channel submit replay requests:

The request from the table above would be grouped into following replay batches:

Before each batch of messages, a system message would be sent detailing the messages to be sent as part of the batch.  For example:

Batch 1

 Batch 2

Each system message contains the RequestBegin and RequestEnd fields along with the Begin and End sequence ranges to provide detection of unavailable data.  For example, if a request is made for sequence numbers 9,000 to 11,000 for a given channel, but the lowest available sequence number is 10,000, the requested data from 9,000 to 10,000 is unavailable.  To eliminate the potential for an unrecoverable scenario, the system message would indicate that the range of 9,000 to 10,000 was requested, but unavailable for processing.

In the unlikely event that a request is made for a sequence range that is too high and not available, the Begin and End values will be 0.  For example, if a request is made for a range outside of the available range of data (i.e., 9,000 to 9,500), a system message will be sent with a Begin and End value of 0.  No additional messages will be sent.