...
To mitigate the total message rate from replay requests initiated across multiple channels, the CME Group replay gateway instances throttle message replay. The replay channel IGMP join should be left after the replay is finished. CME Group recommends that developers externalize configurable parameters to manage the number of concurrent requests and delay between requests to be able to tailor the application to specific bandwidth configurations.
Info |
---|
Customers connected to a replay channel will also receive replay requests from other customers that will impact bandwidth utilization. |
Replay Request
When initiating a replay request, the MDP client application initiates a TCP connection to the CME Group MDP Gateway. After establishing a socket connection, a replay request is sent to the CME MDP Gateway and a response message is sent with information indicating the success or failure of the request.
...
User=CME{SOH}Password=***{SOH}RequestType=REPLAY{SOH}Begin=1{SOH}End=100{SOH}Channel=1{SOH}
Field | Value | Description | Section |
---|---|---|---|
User | String | CME Group-provided user name for each firm | Header |
SOH | |||
Timestamp | String | Time stamp of when request was processed | Header |
SOH | |||
Request Type | String | ‘REPLAY’ for sequence number recovery | Header |
SOH | |||
Result | String | Indicated result of request Result Code Meaning:
| Body |
SOH | |||
Channel | String | Channel of sequence number range to be replayed | Body |
SOH |
Replay Messages
With each request to replay missed or lost messages, a system message is sent preceding the message replay to alert applications of the dataset. System messages must be used to process replay requests efficiently.
...
System messages have a header and a message body formatted as follows:
Field | Value | Description | Section |
---|---|---|---|
Type | String |
| Header |
SOH | |||
Channel | String | The channel to be replayed (i.e. ‘1’) | Body |
SOH | |||
Request Begin | String | Beginning number of sequence requested | Body |
SOH | |||
Request End | String | Ending number of sequence requested | Body |
SOH | |||
Begin | String | Beginning sequence number being sent | Body |
SOH | |||
End | String | Ending sequence number being sent | Body |
SOH | |||
Timestamp | String | Timestamp of proposed request | Body |
SOH | |||
Channel | String | 1 | Body - Heartbeat |
SOH | |||
Channel | String | 1 | Body - Announcement |
SOH | |||
Detail | String | Details of announcement | Body - Announcement |
SOH |
Replay Batching
The CME Group MDP replay mechanism leverages a batching algorithm to process and respond to replay requests more efficiently. Each group of requests within a channel is analyzed for sequence number overlap to combine similar requests into one replay stream. A reasonable gap is used to group requests that may be close to overlapping.
In the following example, five users for the same channel submit replay requests:
Request | Channel | Sequence Range | CME Response Timestamp |
---|---|---|---|
A | 1 | 1,000 - 2,000 | 12345 |
B | 1 | 1,500 – 3,000 | 12367 |
C | 1 | 3,100 – 5,000 | 12378 |
D | 1 | 10,000 - 11,100 | 12389 |
E | 1 | 10, 250 - 10,270 | 12391 |
Info |
---|
Batch 1 - Request A/B/C, insert a reasonable gap between 3,000 and 3,100 Batch 2 - Request D/E |
The request from the table above would be grouped into following replay batches:
Batch | Channel | Sequence Range |
---|---|---|
1 | 1 | 1, 000 – 5,000 |
2 | 1 | 10,000 – 11,100 |
Before each batch of messages, a system message would be sent detailing the messages to be sent as part of the batch. For example:
Batch 1
Field | Value | Description |
---|---|---|
Type | String | “Replay” – Used to alert a replay |
SOH | ||
Channel | String | 1 |
SOH | ||
RequestBegin | String | 1000 |
SOH | ||
RequestEnd | String | 5000 |
SOH | ||
Begin | String | 1000 |
SOH | ||
End | String | 5000 |
SOH | ||
Timestamp | String | 12345 |
SOH |
Batch 2
Field | Value | Description |
---|---|---|
Type | String | “Replay” – Used to alert a replay |
SOH | ||
Channel | String | 1 |
SOH | ||
RequestBegin | String | 10000 |
SOH | ||
RequestEnd | String | 11000 |
SOH | ||
Begin | String | 10000 |
SOH | ||
End | String | 11100 |
SOH | ||
Timestamp | String | 12345 |
SOH |
Each system message contains the RequestBegin and RequestEnd fields along with the Begin and End sequence ranges to provide detection of unavailable data. For example, if a request is made for sequence numbers 9,000 to 11,000 for a given channel, but the lowest available sequence number is 10,000, the requested data from 9,000 to 10,000 is unavailable. To eliminate the potential for an unrecoverable scenario, the system message would indicate that the range of 9,000 to 10,000 was requested, but unavailable for processing.
Field | Value | Description |
---|---|---|
Type | String | “Replay” – Used to alert a replay |
SOH | ||
Channel | String | 1 |
SOH | ||
RequestBegin | String | 9000 |
SOH | ||
RequestEnd | String | 11000 |
SOH | ||
Begin | String | 10000 |
SOH | ||
End | String | 11100 |
SOH | ||
Timestamp | String | 12345 |
SOH |
In the unlikely event that a request is made for a sequence range that is too high and not available, the Begin and End values will be 0. For example, if a request is made for a range outside of the available range of data (i.e., 9,000 to 9,500), a system message will be sent with a Begin and End value of 0. No additional messages will be sent.
Field | Value | Description |
---|---|---|
Type | String | “Replay” – Used to alert a replay |
SOH | ||
Channel | String | 1 |
SOH | ||
RequestBegin | String | 9000 |
SOH | ||
RequestEnd | String | 9500 |
SOH | ||
Begin | String | 0 |
SOH | ||
End | String | 0 |
SOH | ||
Timestamp | String | 12345 |
SOH |