This page describes the market data recovery process for Market by Price (MBP) and Market by Order Full Depth (MBOFD) order books. This method:

• should be used for large-scale data recovery (e.g. major outage or late join)

• synchronizes client order books with the current state maintained by CME Group

The Market Recovery Feed:

• loops the Market Data Snapshot Full Refresh (tag 35-MsgType=W) message.

  • at the beginning of the week, a Snapshot message is generated each time book/trade activity occurs on an instrument

    • no Snapshot message is sent for an instrument with no book activity

  • more than one instrument can be updated in a packet

  • instruments expired during the trading session will stay on the market recovery feed until the end of the day. 

• supports primary (A) and backup (B) feeds

  • Feed A - disseminates market data Snapshot messages for all books having activity since beginning of the week

  • Feed B - acts as a backup in the event that Feed A becomes inoperable

This topic includes: 

Market Recovery Feed Messaging

This section describes the messaging and templates for MBP and MBOFD Market Recovery feeds.

MBP Recovery Feed Messaging

The MBP Market Recovery feed:

• loops Snapshot messages

• implements template SnapshotFullRefresh for recovery processing

Template SnapshotFullRefresh contains:

• Top of Book MBP Quotes – Bids and Offers

• Top of Book Implied Quotes – Bids and Offers

• Last Trade

• Opening Price

• Session High and Low Trade Prices

• Session High Bid and Session Low Offer

• Fixing Price

• Settlement Price 

• Cleared Trade Volume

• Open Interest

• Electronic Volume

• Threshold Limits

MBOFD Recovery Feed Messaging

The MBOFD Market Recovery feed:

• loops Snapshot messages

• implements template SnapshotFullRefresh and template SnapshotFullRefreshOrderBook for recovery processing

In addition to template SnapshotFullRefresh, template SnapshotFullRefreshOrderBook contains recovery data in the following order:

• MBP information

• market state

• statistics

• MBOFD book information

The MBOFD Market Recovery feed also supports:

• recovery chunk tag 37709-NoChunks and tag 37710- CurrentChunk

  • allows client systems to determine the number of packets required to recover an MBOFD book.

  • tag 37709-NoChunks - total number of packets that constitute a single instrument order book

  • tag 37710-CurrentChunk - current chunk in the sequence

The example below shows two different instruments updated in an MBOFD Market Recovery feed.

Market Recovery Price Interleaving (MBOFD)

To optimize recovery time for the highest priority orders, the Snapshot message alternates bid and ask orders from highest to lowest priority (tag 37707-MDOrderPriority).

Market Recovery Processing (MBP & MBOFD)

When packets are missed on both UDP Incremental feeds, packet loss is indicated by a gap in message packet sequence numbers.

• the tag 369-LastMsgSeqNumProcessed value on the Snapshot message corresponds to the packet sequence number on the Incremental feed

• tag 60-TransactTime communicates the start transaction time of the last event for the instrument

When this large-scale market recovery method is used, client systems must subscribe to the Instrument Definition feed to determine if any new instruments have been defined. A client application can determine if recovery is necessary using the tag 779-LastUpdateTime timestamp on the Market Recovery feed to detect if instrument data has changed.

Market Recovery Processing Workflow

This section describes the process to follow for large-scale recovery concurrently processing the Incremental feed and Snapshot messages from the Market Recovery feed. Once a book is recovered, client systems can resume normal processing for that instrument even if other books are still being recovered. Client systems will not recover any missed statistics on the Market Recovery feed.

Recovery processing steps are as follows:

1. Identify channel(s) in which the client system is out of sync.

2. Listen to the Incremental feed and queue real-time data for the affected channel(s), and optionally begin MBP natural refresh.

3. Listen to the Market Recovery feed for the affected channel(s).

4. For a given instrument, compare the Market Recovery Snapshot message tag 369-LastMsgSeqNumProcessed to the Incremental feed Market Data Incremental Refresh (35=X) message packet sequence number.

   • If the SecurityID appears in both the Incremental feed and Market Recovery feed updates during the comparison, then compare the Market Recovery feed tag 60-TransactTime to the Incremental feed tag 60-TransactTime. The instrument with the unequal 60-TransactTime must be recovered via the next market recovery cycle or optional concurrent natural refresh processing.

   • Drop all cached Incremental feed updates with a packet sequence number < 369-LastMsgSeqNumProcessed.

5. Once all instruments are recovered via market recovery or natural refresh, start normal processing and disconnect from the market recovery feed. 

Market Recovery Feed Processing Examples

The following section provides examples for Market Recovery feed processing.  

Example 1 – Standard Recovery

The following example shows a standard recovery scenario using the following workflow:

1. Identify the channel that is out of sync.

2. Connect to the Incremental feed and begin queuing data.

3. Connect to the Market Recovery feed. In this example, the first Market Recovery Snapshot message sequence number = 4.

4. Compare the Market Recovery Snapshot message tag 369-LastMsgSeqNumProcessed to the Incremental feed packet sequence number. In this example, the Incremental feed sequence number compared is 104.

   • If the instrument (tag 48-SecurityID) appears in both the Incremental and Market Recovery updates during the comparison, then compare the Market Recovery feed tag 60-TransactTime to the Incremental feed tag 60-TransactTime.

   • In this example, the tag 60-TransactTime comparison is not required due to different instruments between the feeds.

   • For Market Recovery Snapshot message processing, drop all cached Incremental feed updates with a packet sequence number < tag 369-LastMsgSeqNumProcessed from the Market Recovery feed.

   • Begin normal processing for recovered instrument 13205.

   • Recover subsequent instruments via the Market Recovery feed (8087, etc).

   • Additionally, as tag 369-LastMsgSeqNumProcessed increases in subsequent Market Recovery feed updates, continue to drop cached market data where the packet sequence number < tag 369-LastMsgSeqNumProcessed from the Market Recovery feed.

5. Disconnect from the Market Recovery feed once processing is complete for all instruments.

Example 2 - Same Transaction Time for Instrument

In the following example, the instrument (tag 48-SecurityID=15212) appears in both the Incremental and Market Recovery feed updates when comparing tag 369-LastMsgSeqNumProcessed. Therefore, client systems must compare the Market Recovery feed tag 60-TransactTime to the Incremental feed tag 60-TransactTime. In this example the tag 60-TransactTime values match and the instrument can be recovered. 

Example 3 - Different Transaction Time for Instrument

In the following example, the instrument (tag 48-SecurityID=13205) appears in both the Incremental and Market Recovery feed updates when comparing tag 369-LastMsgSeqNumProcessed, however the tag 60-TransactTime values do not match. Therefore, client systems must process the next Market Recovery feed iteration for the instrument.

Example 4 - New Market Recovery Snapshot Message Added

In the following example there is activity for an instrument (tag 48-SecurityID) that adds a new Market Recovery Snapshot message to the Market Recovery feed. Therefore, to ensure all new instruments are recovered, client systems must process an additional Market Recovery iteration starting from the Snapshot message with a packet sequence number = 1. Client systems can determine if a new Snapshot message has been added by comparing tag 911-TotNumReports with the previous iteration. The order of each Snapshot message iteration is not guaranteed, therefore the newly created Snapshot message may be placed mid-loop in the next iteration.