Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

To mitigate the total message rate from replay requests initiated across multiple channels, the CME Group replay gateway instances throttle message replay.  The replay channel IGMP join should be left after the replay is finished. CME Group recommends that developers externalize configurable parameters to manage the number of concurrent requests and delay between requests to be able to tailor the application to specific bandwidth configurations.

Info

Customers connected to a replay channel will also receive replay requests from other customers that will impact bandwidth utilization.

Replay Request

When initiating a replay request, the MDP client application initiates a TCP connection to the CME Group MDP Gateway.  After establishing a socket connection, a replay request is sent to the CME MDP Gateway and a response message is sent with information indicating the success or failure of the request.

...

User=CME{SOH}Password=***{SOH}RequestType=REPLAY{SOH}Begin=1{SOH}End=100{SOH}Channel=1{SOH}

Field

Value

Description

Section

User

String

CME Group-provided user name for each firm

Header

SOH

Timestamp

String

Time stamp of when request was processed

Header

SOH

Request Type

String

‘REPLAY’ for sequence number recovery

Header

SOH

Result

String

Indicated result of request

Result Code Meaning:

  • 0001 Success

  • 0002 System unable to accept requests

  • 0003 Number of configurable requests exceeded

  • 0010 Malformed request

  • 0011 Missing fields from request

  • 0020 Missing request Type

  • 0021 Missing user Identifier

  • 0022 Missing password

  • 0023 Unknown Request Type

  • 0024 Invalid Userid/password combination

  • 0110 Missing channel identifier

  • 0111 Unknown channel identifier

  • 0112 Missing begin field

  • 0113 Malformed begin field

  • 0114 Missing End field

  • 0115 Malformed End field

  • 0116 Invalid range

  • 0200 Request incorrectly formatted

  • 9999 System error

Body

SOH 

Channel

String

Channel of sequence number range to be replayed

Body

SOH

Replay Messages

With each request to replay missed or lost messages, a system message is sent preceding the message replay to alert applications of the dataset. System messages must be used to process replay requests efficiently.

...

System messages have a header and a message body formatted as follows:

Field

Value

Description

Section

Type

String

  • "Heartbeat” – Used in lieu of message replay

  • “Replay” – Used to alert a replay

  • “Announcement” – Used to alert replay events

Header

SOH

Channel               

String

The channel to be replayed (i.e. ‘1’)

Body

SOH

Request Begin

String

Beginning number of sequence requested

Body

SOH

Request End

String

Ending number of sequence requested

Body

SOH

Begin

String

Beginning sequence number being sent

Body

SOH

End

String

Ending sequence number being sent

Body

SOH

Timestamp

String

Timestamp of proposed request

Body

SOH

Channel

String

1

Body - Heartbeat

SOH

Channel               

String

1

Body - Announcement

SOH

Detail

String

Details of announcement

Body - Announcement

SOH

Replay Batching

The CME Group  MDP replay mechanism leverages a batching algorithm to process and respond to replay requests more efficiently.  Each group of requests within a channel is analyzed for sequence number overlap to combine similar requests into one replay stream.  A reasonable gap is used to group requests that may be close to overlapping.

In the following example, five users for the same channel submit replay requests:

Request

Channel

Sequence Range

CME Response Timestamp

A

1

1,000 - 2,000

12345

B

1

1,500 – 3,000

12367

C

1

3,100 – 5,000

12378

D

1

10,000 - 11,100

12389

E

1

10, 250 - 10,270

12391

Info

Batch 1 - Request A/B/C, insert a reasonable gap between 3,000 and 3,100

Batch 2 - Request D/E

The request from the table above would be grouped into following replay batches:

Batch

Channel

Sequence Range

1

1

1, 000 – 5,000

2

1

10,000 – 11,100

Before each batch of messages, a system message would be sent detailing the messages to be sent as part of the batch.  For example:

Batch 1

Field

Value

Description

Type                      

String

“Replay” – Used to alert a replay

SOH

Channel               

String

1

SOH

RequestBegin

String

1000

SOH

RequestEnd

String

5000

SOH

Begin

String

1000

SOH

End

String

5000

SOH

Timestamp

String

12345

SOH  

 Batch 2

Field

Value

Description

Type                      

String

“Replay” – Used to alert a replay

SOH

Channel               

String

1

SOH

RequestBegin

String

10000

SOH

RequestEnd

String

11000

SOH

Begin

String

10000

SOH

End

String

11100

SOH

Timestamp

String

12345

SOH

Each system message contains the RequestBegin and RequestEnd fields along with the Begin and End sequence ranges to provide detection of unavailable data.  For example, if a request is made for sequence numbers 9,000 to 11,000 for a given channel, but the lowest available sequence number is 10,000, the requested data from 9,000 to 10,000 is unavailable.  To eliminate the potential for an unrecoverable scenario, the system message would indicate that the range of 9,000 to 10,000 was requested, but unavailable for processing.

Field

Value

Description

Type                      

String

“Replay” – Used to alert a replay

SOH

Channel               

String

1

SOH

RequestBegin

String

9000

SOH

RequestEnd

String

11000

SOH

Begin

String

10000

SOH

End

String

11100

SOH

Timestamp

String

12345

SOH

In the unlikely event that a request is made for a sequence range that is too high and not available, the Begin and End values will be 0.  For example, if a request is made for a range outside of the available range of data (i.e., 9,000 to 9,500), a system message will be sent with a Begin and End value of 0.  No additional messages will be sent.

Field

Value

Description

Type                      

String

“Replay” – Used to alert a replay

SOH

Channel               

String

1

SOH

RequestBegin

String

9000

SOH

RequestEnd

String

9500

SOH

Begin

String

0

SOH

End

String

0

SOH

Timestamp

String

12345

SOH