Packet Capture Dataset

Owned by Space Sync for Confluence
yesterday at 11:05 AM
8 min read

Market Data Packet Captures (PCAP) are the complete historical raw data feeds on all channels for both futures and options from the channels' MBO release. The Settlements and Valuations channel is a new channel available in PCAP format.

Dates Available

Packet Capture data is available on four exchanges from January 2017 to present and all files are in FIX format. For the availability of each Channel please reference the FAQs below.

Please note that CME Term SOFR will not be available on Channel 312 after January 8th.



By Exchange



Exchange

Open Date

Close Date

Exchange

Open Date

Close Date

CME (Electronic)

5/13/2017

Present

CBOT (Electronic)

4/1/2017

Present

NYMEX (Electronic)

1/7/2017

Present

COMEX (Electronic)

1/7/2017

Present





Message Specifications

 

Field contains the following data:

  • Message Length: 2 bytes

  • Binary Packet header: 12 bytes

    • MsgSeqNum: 4 bytes

    • SendingTime: 8 bytes

  • Message Header: 10 bytes

    • MsgSize: 2 bytes (repeats here)

  • SBE Message Payload

    • SBE Header: 8 bytes

      • Block Length: 2 bytes

      • Template ID: 2 bytes

      • Schema ID: 2 bytes

      • Version: 2 bytes

  • SBE Message: MsgSize - 22 bytes

[Top]

Security Definition Files



Tag Usage by Security Type

Tag

FIX Name

Future Outright

NYMEX Decay/Variable Future

Future Spread

Option Outright

Option Spread

UDS Combo

UDS Covered

Tag

FIX Name

Future Outright

NYMEX Decay/Variable Future

Future Spread

Option Outright

Option Spread

UDS Combo

UDS Covered

Application Sequence Control

5799

MatchEventIndicator

X

X

X

X

X

X

X

911

TotNumReports

c

c

c

c

c

c

c

980

SecurityUpdateAction

X

X

X

X

X

X

X

779

LastUpdateTime

X

X

X

X

X

X

X

1180

ApplID

X

X

X

X

X

X

X

Instrument Data

1300

MarketSegmentID

X

X

X

X

X

X

X

55

Symbol

X

X

X

X

X

X

X

48

SecurityID

X

X

X

X

X

X

X

22

SecurityIDSource

X

X

X

X

X

X

X

200

MaturityMonthYear

X

X

X

X

X

X

X

1151

SecurityGroup

X

X

X

X

X

X

X

6937

Asset

X

X

X

X

X

X

X

167

SecurityType

FUT

FUT

FUT

OOF

OOF

OOF

MLEG

762

SecuritySubType

–

–

X

–

X

X

X

461

CFI Code

X

X

X

X

X

X

X

201

PutOrCall

–

–

–

X

–

–

–

462

UnderlyingProduct

X

X

X

X

–

–

–

207

SecurityExchange

X

X

X

X

X

X

X

1682

MDSecurityTradingStatus

c

c

c

c

c

c

c

202

StrikePrice

–

–

–

X

–

–

–

947

StrikeCurrency

–

–

–

X

–

–

–

15

Currency

X

X

X

X

X

X

X

120

SettlCurrency



 –

 -

c

–

–

–

9850

MinCabPrice

–

–

–

c

–

–

–

9779

UserDefinedInstrument

–

–

–

–

–

Y

Y

Underlying (options only)

711

NoUnderlyings

-

-

-

X

-

-

-

→311

UnderlyingSymbol

-

-

-

X

-

-

-

→309

UnderlyingSecurityID

-

-

-

X

-

-

-

→305

UnderlyingSecurityIDSource

-

-

-

X

-

-

-

Leg Group (spreads only)

555

NoLegs

-

-

X

-

X

X

X

→602

LegSecurityID

-

-

X

-

X

X

X

→624

LegSide

-

-

X

-

X

X

X

→623

LegRatioQty

-

-

X

-

X

X

X

→566

LegPrice

-

-

-

-

-

-

X

→1017

LegOptionDelta

-

-

-

-

-

-

X

Trading Rules

1141

NoMdFeedTypes

X

X

X

X

X

X

X

→1022

MDFeedType

X

X

X

X

X

X

X

→264

MarketDepth

X

X

X

X

X

X

X

864

NoEvents

2

2

2

2

2

2

2

→865

EventType

X

X

X

X

X

X

X

→1145

EventTime

X

X

X

X

X

X

X

1142

MatchAlgorithm

X

X

X

X

X

X

X

1234

NoLotTypeRules

X

X

X

X

X

X

X

→1093

LotType

X

X

X

X

X

X

X

→1231

MinLotSize

X

X

X

X

X

X

X

562

MinTradeVol

X

X

X

X

X

X

X

1140

MaxTradeVol

X

X

X

X

X

X

X

969

MinPriceIncrement

X

X

X

c

X

X

X

1146

MinPriceIncrementAmount

X

X

-

X

-

-

-

9787

DisplayFactor

X

X

X

X

X

X

X

5770

PriceRatio

(Implied Intercommodity Ratio Spreads only)

–

–

c

–

–

–

–

6350

TickRule

–

–

–

c

c

–

–

37702

MainFraction

c

–

–

c

–

–

–

37703

SubFraction

c

–

–

c

–

–

–

9800

PriceDisplayFormat

c

_

 – 

c

–

–

–

Instrument Attribute Group

870

NoInstAttrib

X

X

X

X

X

X

X

→871=24

InstAttribType

X

X

X

X

X

X

X

→872

InstAttribValue

X

X

X

X

X

X

X

Contract Lot Size/Measure/Unit

1435

ContractMultiplierUnit

-

X

-

-

-

-

-

1439

FlowScheduleType

-

X

-

-

-

-

-

231

ContractMultiplier

-

X

-

-

-

-

-

996

UnitOfMeasure

X

X

X

X

X

X

X

1147

UnitOfMeasureQty

X

X

-

X

-

-

-

5818

DecayQty

-

c

-

-

-

-

-

5819

DecayStartDate

-

c

-

-

-

-

-

5849

OriginalContractSize

-

c

-

-

-

-

-

Statistics and Limits

1150

TradingReferencePrice

c

c

c

c

c

c

c

731

SettlPriceType

c

c

c

c

c

c

c

5796

TradingReferenceDate

c

c

c

c

c

c

c

5792

OpenInterestQty

c

c

c

c

c

c

c

5791

ClearedVolume

c

c

c

c

c

c

c

1149

HighLimitPrice

c

c

c

c

c

c

c

1148

LowLimitPrice

c

c

c

c

c

c

c

1143

MaxPriceVariation

c

c

c

–

–

–

–

[Top]



Sample Files

In order to obtain a sample PCAP file you must set up an s3 account and then contact CME Group's Data Sales team @CMEDataSales@cmegroup.com.  You can find instruction on how to entitle your s3 account to DataMine data here. 


[Top]

Layout Guides

View layout guides and message specifications for the following market data formats.

CME MDP 3.0 (Current Production Format)

The CME Group Market Data Platform (MDP) disseminates event-based bid, ask, trade, and statistical data for CME Group markets and also provides recovery and support services for market data processing. Click below to access layout guide.

FAQ

What format is the file delivered in?

PCAP files are SBE raw data grouped by channel. The file contains channel number, length of message and actual payload.  For example: 312 (channel), 96 (length of message), <actual payload>.

How do I best understand the timestamp(s)?

All messages will have FIX tag 60 (within payload), this represents the Globex started processing time at nanosecond granularity with microsecond accuracy.

What is the precision level for timestamping in PCAP?

PCAP is delivered in nanosecond granularity with microsecond accuracy.

Does PCAP come from the same source as Top-of-Book and Time & Sales data?

No, our PCAP files are produced from CME Globex Market Data Platform FIX/FAST feeds, while our Top-of-Book and Time & Sales data come from an internal post-trade processing system. The granularity of the timestamps for the data offerings is different due to the sources.

What is the Settlements and Valuations Channel?

With the launch of Settlements and Valuations, CME Group introduced theoretical valuations. Theoretical valuations is new type of data produced by CME Group for futures and options products without volume, open interest, or underlying pricing required for Clearing products. For more information, see here.

How far back do you maintain MBO records?

Channel

Channel Description

Start Date

End Date

Channel

Channel Description

Start Date

End Date

310

CME Globex Equity Futures

5/20/2017

Present

311

CME Globex Equity Options

5/20/2017

Present

312

CME Globex Interest Rate Futures

5/20/2017

Present

313

CME Globex Interest Rate Options

5/20/2017

Present

314

CME Globex FX Futures

5/13/2017

Present

315

CME Globex FX Options

5/13/2017

Present

316

CME Globex Commodity Futures

5/13/2017

Present

317

CME Globex Commodity Options

5/13/2017

Present

318

CME Globex Equity Futures- excludes E-mini S&P 500

5/13/2017

Present

319

CME Globex Equity Options- excludes E-mini S&P 500

5/20/2017

Present

320

CME Globex FX Futures II

5/13/2017

Present

321

CME Globex FX Options II

5/13/2017

Present

340

CBOT Globex Commodity Futures

4/1/2017

Present

341

CBOT Globex Commodity Options

4/1/2017

Present

342

CBOT Globex Equity Index Futures

4/22/2017

Present

343

CBOT Globex Equity Index Options

4/22/2017

Present

344

CBOT Globex Interest Rate Futures

4/22/2017

Present

345

CBOT Globex Interest Rate Options

4/22/2017

Present

346

CBOT Globex Commodity Futures II

4/1/2017

Present

360

COMEX Globex Futures

1/7/2017

Present

361

COMEX Globex Options

1/7/2017

Present

380

NYMEX Globex Emissions Futures

1/7/2017

Present

381

NYMEX Globex Emissions Options

1/7/2017

Present

382

NYMEX Globex Crude & Crude Refined Futures

1/21/2017

Present

383

NYMEX Globex Crude & Crude Refined Options

1/7/2017

Present

384

NYMEX Globex Metals, Softs, & Alternative Market Futures

1/7/2017

Present

385

NYMEX Globex Metals, Softs, & Alternative Market Options

1/7/2017

Present

386

NYMEX Globex Nat Gas & other Non-Crude Energy Futures

1/7/2017

Present

387

NYMEX Globex Nat Gas & other Non-Crude Energy Options

1/7/2017

Present

What product is a channel on?

Please reference the CME Globex Product Reference Sheet to see the MDP channel for each product.  Please note the channel is listed in column AC.

How many levels of depth are available? Has this depth changed over time?

PCAP has the Full depth of book. This has not changed over time.

Are spreads included in the PCAP files?

 Exchange-listed spread products are included for both futures and options spreads.

Do you have sample files available?

In order to obtain a sample PCAP file you must set up an s3 account and then contact CME Group's Data Sales team CMEDataSales@CMEGroup.com. You can find instruction on how to entitle your s3 account to DataMine data here. 

Are there any anomolies with this data?

For Channel 381, NYMEX Globex Emissions Options, there is a data gap from 6/27/2021 - 7/12/2021

When are these files delivered?

CME Group’s process to distribute PCAP files begins at 5:00 p.m. Central Time (CT) and is typically complete by midnight.

How many files do I receive per day for a PCAP subscription?

You will receive 2 files per day for each channel you are subscribed to. The first will have the -e in the file name. This is data that occurred from 4:30 PM CT to Midnight CT. The second file will have the -f in the file name and will be from Midnight CT to 4:00 PM CT. On Sundays, you will receive all data from the start of the feed. The earliest heartbeat messages that are captured have a timestamp of 5:30 AM CT.

If I purchase daily updates of PCAP data, will I get historical data as well?

No. When an order is placed for daily updates of PCAP data, the first file included will be generated for the start date of the subscription. However, files remain accessible for 30 days after purchase, enabling the customer to reference previous day’s data.



[Top]



How was your Client Systems Wiki Experience? Submit Feedback

Copyright © 2024 CME Group Inc. All rights reserved.