With With CME Smart Stream on Google Cloud Platform (GCP), clients clients can access real-time CME Group market data feeds through native Google Cloud Platform services. CME Smart Stream on Google Cloud Platform (GCP) supports JavaScript Object Notation (JSON) formatted futures and options data, allowing customers to use an easy, low-cost, and highly scalable format to drive their business. Data available in this universal format includes top of book information, second market level depth (i.e., second best bid and offer), trade execution, and product statistics including settlement, total daily volume, and open interest.
This page provides an overview of CME Smart Stream on GCP JSON technical impacts. CME Smart Stream on GCP leverages Google Cloud Pub/Sub technology for market data distribution. For complete details regarding Google Cloud Pub/Sub, refer to Google Cloud Pub/Sub documentation.
| Info |
|---|
CME Smart Stream on GCP also offers Simple Binary Encoded (SBE) data. For more information, refer to the CME Smart Stream on GCP SBE impact documentation. |
Contents
| Table of Contents |
|---|
Testing and Certification
...
The New Release test environment allows product and new functionality testing prior to release in production. Use the New Release environment to perform:
New product testing
Development testing on new functionality
| Info |
|---|
The New Release test environment has lower data rates than production. |
Summary of Impacts
The following market data is published in JSON format via the Google Cloud Pub/Sub:
Top of Book (TOB)
Level 2 Order Book (LVL 2)
Market Statistics (OI, Settlement and Traded Volume)
Time and Sales (Trade Information)
Market Data Offerings via Google Cloud
JSON CME Smart Stream on GCP supports CME Globex sourced products from the following Designated Contract Markets (DCM):
CME
CBOT
COMEX
NYMEX
GME
Google Cloud Pub/Sub Processing Overview
This section provides an overview of CME MDP processing for Google Cloud Pub/Sub. For complete details regarding Google Cloud Pub/Sub technology, refer to Google's online Pub/Sub documentation. For Pub/Sub libraries, refer to Google’s Pub/Sub Client Libraries. Common GCP SDK Pub/Sub processing concepts include the following:
Pulling data from a subscription in Python synchronously
andand asynchronously
| Info |
|---|
Terminology
|
CME MDP on Google Cloud Pub/Sub Message Flow
The following diagram outlines the Publisher (CME Group) relationship with the Subscriber (client system) for for Google Cloud Pub/Sub technology.
| Gliffy | ||||||
|---|---|---|---|---|---|---|
|
CME Group creates a topic in the Cloud Pub/Sub service and sends messages to the topic. To process CME MDP on Google Cloud Pub/Sub, client systems must subscribe to a topic to receive data. The Google Cloud Pub/Sub message will contain CME Group messages.
Message store ensures that published messages are retained on behalf of subscriptions.
Cloud Pub/Sub forwards messages from a topic to each individual subscription. Each subscription receives messages
...
by pulling them from the service. For CME MDP on Cloud Pub/Sub there are four types of topics for a given product group that client systems can subscribe to:
Top of Book Topic - Provides a 1-deep Market by Price (MBP) book for an instrument.
2-Level Book Topic - Provides a 2-deep Market by Price (MBP) book for an instrument.
Market Stats Topic - Includes Open Interest, Settlement and Traded Volume.
Trade Information Topic - Provides trade updates for an instrument.The subscriber receives pending messages from its subscription and acknowledges each one to the Cloud Pub/Sub service. A message contains JSON messages.
When a message is acknowledged by the subscriber it is removed from the subscription's message queue.
Message Structure
The following section outlines the CME MDP messaging within within a Google Cloud Pub/Sub Message structure.
CME MDP Included Data
The Google Cloud Pub/Sub data field provides the full CME full CME JSON message. The following diagram outlines CME data within a Google Cloud Pub/Sub message:
| Gliffy | ||||
|---|---|---|---|---|
|
CME MDP Included Attributes
In addition to the full CME Group JSON message, Smart Stream on GCP provides the following quick access data in the Google Cloud Pub/Sub attributes field:
SendingTime - Sending time from CME before the message is sent to GCP.
| Info |
|---|
Client systems may receive Saturday test data upon connection. Test data can be filtered using the SendingTime attribute. |
Message Ordering
Google Cloud Pub/Sub does not guarantee message order. Therefore, the client system must ensure the correct order for processing messages via the SendingTime attribute.
...
CME Smart Stream JSON on Google Cloud Platform (GCP) has data per product. Project names have the following methodology:
Project Name | Type | Environment |
|---|---|---|
cmegroup-marketdata-js-nr | Futures | New Release |
cmegroup-marketdata-js-opt-nr | Options | New Release |
cmegroup-marketdata-js | Futures | Production |
cmegroup-marketdata-js-opt | Options | Production |
Topic names have the following methodology:
Name | Valid Values | Description |
|---|---|---|
EnvType | PROD = Production NR = New Release CERT = Certification | Environment |
ServiceName | SSCL = Smart Stream Cloud Link | Cloud service provider |
Provider | GCP = Google Cloud Products | Name of the cloud provider |
ContentType | MD = Market Data | Type of content |
Content Timing | RT = Real Time | Time data is sent |
ContentOwner | CMEG = CME Group Owned Market Data DME = Gulf Mercantile Exchange | Reference to exchange |
Format | JSON = JavaScript Object Notation | Feed Type |
Version | v01000 | The version of the message format for a topic |
Topic Type | TOB = Top of Book LVL2 = Top Two Book Levels TRD = Time and Sales (Trade Information) STAT = Statistic Information | Defines the type of topic |
SecurityExchange | Security Exchange: XCBT = Chicago Board of Trade | Security Exchange for JSON Topics |
Product Group | Examples: CL = Crude Oil Futures | Product Group. JSON topics only Product and instrument referential data |
for CME Smart Stream on |
GCP can be obtained via CME Reference Data API. |
Topic Name Examples
The following are topic name examples:
Production JSON 30-Year U.S. Treasury Bond Futures Top of Book Information: PROD.SSCL.GCP.MD.RT.CMEG.JSON.v01000.TOB.XCBT.ZB
New Release JSON Crude Oil Futures Trade Information: NR.SSCL.GCP.MD.RT.CMEG.JSON.v01000.TRD.XNYM.CL
Additional Product and Instrument Information
Product and instrument referential data for CME Smart Stream on GCP GCP can be obtained via via CME Reference Data API.
| Info |
|---|
The attribute 'id' maps to the Reference Data API instrument field globexSecurityId. |
Message Specification
The following section outlines the full message specification for Smart Stream JSON messages.
Trade Message
This message is generated for the Trade topic.
...
Key | Attribute | Type | Enumeration | Description | |
|---|---|---|---|---|---|
| 1 | header | messageType | String | TS | Denotes trade summary message. |
| 2 | header | sentTime | String | ex "2020-05-11T11:22:12.208530531" | CME MDP gateway sends the message (UTC). Format: "yyyy-MM-dd'T'HH:mm:ss.n" |
| 3 | header | version | String | 1.0 | API version |
| 4 | payload | lastUpdateTime | String | ex "2020-05-11T11:22:12.208530531" | Last update processing time (UTC). Format: "yyyy-MM-dd'T'HH:mm:ss.n" |
| 5 | payload.tradeSummary | aggressorSide | short | 0 = No aggressor 1 = Buy 2 = Sell 255 = Null Value | Indicates which side is aggressor of the trade. If there is a zero value present, then there is no aggressor. Trades without aggressors occur:
|
| 6 | payload.tradeSummary | mdTradeEntryId | uInt32 | Common Trade ID that links each trade execution. | |
| 7 | payload.tradeSummary | tradePrice | PRICE | Trade price. | |
| 8 | payload.tradeSummary | tradeQty | int32 | Total traded quantity. | |
| 9 | payload.tradeSummary | tradeOrderCount | int32 | Identifies the total number of non-implied orders per instrument that participated in a match event. | |
| 10 | payload.tradeSummary | tradeUpdateAction | String | NEW CHANGE DELETE | Trade market data update action. |
| 11 | payload.tradeSummary.orderQty | orderId | int32 | Unique ID assigned by CME Globex to identify orders. | |
| 12 | payload.tradeSummary.orderQty | lastOrdQty | int32 | Quantity of trade | |
| 13 | payload.Instrument | definitionSource | String | E = exchange defined U = user defined | Identifies user-defined instruments. If the tag is not present, the instrument is not user-defined. |
| 14 | payload.Instrument | exchangeMic | String | XCBT = Chicago Board of Trade | Exchange used to identify a security. |
| 15 | payload.Instrument | id | int32 | Unique instrument ID as qualified by the exchange per market segment. The unique instrument ID value will not be reused until the next trade date following an instrument expiration or deletion. | |
| 16 | payload.Instrument | marketSegmentId | int32 | Identifies the market segment. Populated for all CME Globex instruments. | |
| 17 | payload.Instrument | periodCode | String | ex "202106" | This field provides the calendar month reflected in the instrument symbol. Format YYYYMM (e.g., 201912) For futures spreads, this field contains the first leg's calendar month reflected in the instrument symbol. |
| 18 | payload.Instrument | productCode | String | String field that indicates the underlying asset code (Product Code). Example: SR1 (SOFR), ES (E-Minis). | |
| 19 | payload.Instrument | productType | String | FUT = Future or Future Spread | Product Type |
| 20 | payload.Instrument | productGroup | String | Product Group Code. | |
| 21 | payload.Instrument | symbol | String | e.g. ‘6EN2 P1220’, 6AZ4 | Instrument Name or Symbol. |
| 22 | payload.Instrument | PutCallIndicator | INT | 1 = Call 0 = Put | Indicates whether an option instrument is a put or call. |
| 23 | payload.Instrument | Strike | Price | Strike Price for an option instrument. | |
| 24 | payload.Instrument | UnderlyingSymbol | String | Underlying Instrument Symbol (Contract Name) * this value will be the same as that contained in Leg Instrument's Security Definition Tag 55-Symbol. |
Book Message
This message is generated for top of book and level 2 book topics.
...
Key | Attribute | Type | Enumeration | Description | |
|---|---|---|---|---|---|
| 1 | header | messageType | String | OBTS = top of book LVL2 = level 2 book depth | Denotes trade book message |
| 2 | header | sentTime | String | ex "2020-05-11T11:22:12.208530531" | Time CME MDP gateway sends the message (UTC). Format: "yyyy-MM-dd'T'HH:mm:ss.n" |
| 3 | header | version | String | API version | |
| 4 | payload | tradingStatus | String | 2=Trading Halt 4=Close 15=New Price Indication 17=Ready To Trade (start of session) 18=Not Available For Trading 20=Unknown or Invalid 21=Pre-Open 24=Pre-Cross 25=Cross 26=Post Close 103=No Change | Identifies the trading status applicable to the instrument or product group. |
| 5 | payload.Instrument | definitionSource | String | E = exchange-defined U = user-defined | Identifies user-defined instruments. If the tag is not present, the instrument is not user-defined. |
| 6 | payload.Instrument | exchangeMic | String | XCBT = Chicago Board of Trade | Exchange used to identify a security. |
| 7 | payload.Instrument | marketSegmentId | int32 | Identifies the market segment. Populated for all CME Globex instruments. | |
| 8 | payload.instrument | periodCode | String | ex "202106" | The calendar month reflected in the instrument symbol. For futures spreads, this field contains the first leg's calendar month reflected in the instrument symbol. |
| 9 | payload.instrument | productCode | String | An exchange-specific code assigned to a group of related securities, which are concurrently affected by market events. | |
| 10 | payload.instrument | productGroup | String | Product group | |
| 11 | payload.instrument | productType | String | FUT = Future or Future Spread | Product type |
| 12 | payload.instrument | symbol | String | Instrument Name or Symbol. | |
| 13 | payload.askLevel | lastUpdateTime | String | ex "2020-05-11T11:22:12.208530531" | Last update time for ask price Format: "yyyy-MM-dd'T'HH:mm:ss.n" |
| 14 | payload.askLevel | orderCnt | int32 | Aggregate number of orders at the given price level. | |
| 15 | payload.askLevel | price | long | Price of the MD Entry. | |
| 16 | payload.askLevel | qty | int32 | Order quantity. | |
| 17 | payload.bidLevel | lastUpdateTime | String | ex "2020-05-11T11:22:12.208530531" | Last update time for bid price. Format: "yyyy-MM-dd'T'HH:mm:ss.n" |
| 18 | payload.bidLevel | orderCnt | int32 | Aggregate number of orders at the given price level. | |
| 19 | payload.bidLevel | price | long | Price of the MD Entry. | |
| 20 | payload.bidLevel | qty | int32 | Order quantity | |
| 21 | payload.Instrument | PutCallIndicator | INT | 1 = Call 0 = Put | Indicates whether an option instrument is a put or call. |
| 22 | payload.Instrument | Strike | Price | Strike Price for an option instrument. | |
| 23 | payload.Instrument | UnderlyingSymbol | String | Underlying Instrument Symbol (Contract Name) * this value will be the same as that contained in Leg Instrument's Security Definition Tag 55-Symbol. |
Statistic Message
This message is generated for statistic topics.
See CME Group Settlements for more information.
...
Key | Attribute | Type | Enumeration | Description | |
|---|---|---|---|---|---|
| 1 | header | messageType | String | TSTAT | Denotes statistic message. |
| 2 | header | sentTime | String | ex "2020-05-11T11:22:12.208530531" | Time CME MDP gateway sends the message (UTC). Format: "yyyy-MM-dd'T'HH:mm:ss.n" |
| 3 | header | version | String | API version | |
| 4 | payload | lastUpdateTime | long | ex "2020-05-11T11:22:12.208530531" | Last update processing time (UTC). Format: "yyyy-MM-dd'T'HH:mm:ss.n" |
| 5 | payload.tradeStatistics | settlementPrice | PRICENULL | Settlement price. | |
| 6 | payload.tradeStatistics | settlementPriceDate | String | ex "2020-05-13" | Date of trade session corresponding to a statistic entry. Format: "YYYY-MM-DD" |
| 7 | payload.tradeStatistics | settlementPriceTimestamp | String | ex "2020-05-11T11:22:12.208530531" | Time of trade session corresponding to a statistic entry. Format: "yyyy-MM-dd'T'HH:mm:ss.n" |
| 8 | payload.tradeStatistics | settlementFinal | boolean | Final settlement price | |
| 9 | payload.tradeStatistics | settlementActual | boolean | Actual settlement price | |
| 10 | payload.tradeStatistics | settlementRounded | boolean | Rounded settlement price | |
| 11 | payload.tradeStatistics | openInterest | int32 | The total open interest for the market at the close of the prior trading session. | |
| 12 | payload.tradeStatistics | openInterestDate | String | ex "2020-05-13" | Open interest trade date Format: "YYYY-MM-DD" |
| 13 | payload.tradeStatistics | openInterestTimestamp | String | ex "2020-05-11T11:22:12.208530531" | Open interest update time. |
| 14 | payload.tradeStatistics | clearedVolume | int32 | Cleared volume quantity. | |
| 15 | payload.tradeStatistics | clearedVolumeDate | String | Cleared volume date. | |
| 16 | payload.tradeStatistics | clearedVolumeTimestamp | String | Cleared volume time. | |
| 17 | payload.Instrument | definitionSource | String | E = exchange-defined U = user-defined | Identifies user-defined instruments. If the tag is not present, the instrument is not user-defined. |
| 18 | payload.Instrument | exchangeMic | String | XCBT = Chicago Board of Trade | Exchange used to identify a security. |
| 19 | payload.Instrument | id | int | Unique instrument ID as qualified by the exchange per market segment. The unique instrument ID value will not be reused until the next trade date following an instrument expiration or deletion. | |
| 20 | payload.Instrument | marketSegmentId | int | Identifies the market segment. Populated for all CME Globex instruments. | |
| 21 | payload.Instrument | periodCode | String | ex "202106" | The calendar month reflected in the instrument symbol. Format YYYYMM (e.g., 201912) For futures spreads, this field contains the first leg's calendar month reflected in the instrument symbol. |
| 22 | payload.Instrument | productCode | String | String field that indicates the underlying asset code (Product Code). Example: SR1 (SOFR), ES (E-Minis). | |
| 23 | payload.Instrument | productType | String | FUT = Future or Future Spread | Product Type. |
| 24 | payload.Instrument | productGroup | String | Product Group Code. | |
| 25 | payload.Instrument | symbol | String | e.g. ‘6EN2 P1220’, 6AZ4 | Instrument Name or Symbol. |
| 26 | payload.Instrument | PutCallIndicator | INT | 1 = Call 0 = Put | Indicates whether an option instrument is a put or call. |
| 27 | payload.Instrument | Strike | Price | Strike Price for an option instrument. | |
| 28 | payload.Instrument | UnderlyingSymbol | String | Underlying Instrument Symbol (Contract Name) * this value will be the same as that contained in Leg Instrument's Security Definition Tag 55-Symbol. |
Contact Information
For technical development support, contact Certification Support for Electronic Trading (CSET).
...