/
CME DataMine Post-Purchase Information

CME DataMine Post-Purchase Information

Delivery Options

There are three different ways to access historical data purchases:

1) RESTful API data download service. A CME Group Login and API ID are required to access the data download API.

2) SFTP File Transfer. A CME Group Login and API ID are required to access the data download API.

3) Automatic S3 transfer.  User must have an AWS S3 bucket in order to perform this transfer.

To discuss other options for data delivery, please contact CMEDataSales@cmegroup.com.

API Delivery Set Up

  1. Log in to the CME Customer Center with your CME Group Login.

See additional information about setting up CME Group Log in

2. From the navigation bar, click your CME Group Login, then click My Profile.

3. Click API Management.

4. Register for an API ID, and select the Type "Basic Auth" for basic authentication.

Data Download API

View information about accessing our RESTful API service.

List

You can query a list of entitled files using this API call, with optional criteria to limit the number of results returned. The maximum number of result from a list is 1000. If you have more results than the limit, you must use the paging links that are returned in the "paging" section as part of the response.

 

Name

Description

Required

Type

Name

Description

Required

Type

dataset

Dataset the user is querying.

 

Example: bbo, block, eod, md, tick (see reference section for remaps)

No

String

exchangecode

Exchange Code the data is requested for.

 

Example: xcme, xnym, xcbt, xcec, cmed, etc. (see reference section for remaps)

No

String

foiindicator

Future Option Indicator.

 

Possible values:

  • fut – Futures

  • opt – Options

  • idx – Indices

No

String

venue

The venue the product is traded in.

 

Possible values: 

  • rth - Regular trading hours

  • eth - Extended trading hours

  • otc - Clearport

No

String

productcode

CME Product Code

 

Example: cl, es, gs

No

String

yyyymmdd

CME trade date. Accepted format is yyyymmdd

 

Example: 20150131

No

String

limit

Amount of results per request. Maximum 1000 rows will be returned.

 

A paging information will return with offset information.

No

int

offset

Pagination mechanism

 

Use the offset returned by previous query for previous or next page.

No

int

firstfid

Alternative pagination mechanism

 

Use the firstfid returned in the paging section to query previous page

No

String

lastfid

Use the lastfid returned in the paging section to query next page

No

String

 

 

curl --user {{UNO_API_KEY}}:{{UNO_API_PASSWORD}} 'https://{{URL}}/cme/api/v1/list?{{criteria1}}={{val1}}&{{criteria2}}={{val2}}&{{criteriaN}}={{valN}}'

 

Curl Users:

curl --user API_JOHNSMITH:12345

'https://datamine.cmegroup.com/cme/api/v1/list?dataset=eod&exchangecode=xcec&yyyymmdd=20150130'

 

WGET Users:

wget --user API_JOHNSMITH -O- -q --password 12345 'https://datamine.cmegroup.com/cme/api/v1/list'

Response

 

Name

Description

Type

Name

Description

Type

Dataset

Dataset of the file

String

exchangecode

Exchange Code for the file

String

url

Fully qualified download URL

String

expiration

expiration date of file access

String

productcode

CME Product Code

String

yyyymmdd

CME trade date

String

checksum

MD5 Checksum for data

String

size

File size in KB

int

fid

Unique identifier for file

String

orderid

Order number

String

 

 

 

Name

Description

Type

Name

Description

Type

previous

Pre-populated link that will bring you to the previous page of result

String

next

Pre-populated link that will bring you to the next page of result

String

 

 

 

{

  "files": [

    {

      "dataset": "eod",

      "yyyymmdd": "20150130",

      "url": "http://localhost:8080/cme/api/v1/download?fid=20150130-eod_xcec_ali_fut_0-eth_p",

      "fid": "20150130-eod_xcec_ali_fut_0-eth_p",

      "orderid": "29",

      "exchangecode": "xcec",

      "productcode": "ali",

      "checksum": "c609f0a1091c693e759d46f1b8756f54",

      "size": 642,

      "expiration": "2016-06-10"

    },

    {

      "dataset": "eod",

      "yyyymmdd": "20150130",

      "url": "http://localhost:8080/cme/api/v1/download?fid=20150130-eod_xcec_ali_fut_0-eth_f",

      "fid": "20150130-eod_xcec_ali_fut_0-eth_f",

      "orderid": "31",

      "exchangecode": "xcec",

      "productcode": "ali",

      "checksum": "c609f0a1091c693e759d46f1b8756f54",

      "size": 642,

      "expiration": "2016-06-12"

    }

  ],

  "paging": {

    "previous": "http://datamine.cmegroup.com/cme/api/v1/list?limit=2&offset=0",

    "next": "http://datamine.cmegroup.com/cme/api/v1/list?limit=2&offset=3"

  }

}

**Please note that paging information is at the end of the response**

Download

You can download a file that you have access to using this api call.

 

Name

Description

Required

Type

fid

File Id, available from the list api.

The fid format is: yyyymmdd-dataset_exch_symbol_foi_spread-venue

Example:

20160107-bbo_xcme_sp_fut_0-rth

Yes

String

 

 

 

curl -J -O --user {{UNO_API_KEY}}:{{UNO_API_PASSWORD}} 'https://{{URL}}/cme/api/v1/download?fid={{val}}'

 

Curl Users:

curl -J -O --user API_JOHNSMITH:12345

'https://datamine.cmegroup.com/cme/api/v1/download?fid=20150130-eod_xcec_ali_fut_0-eth_p'

 

You can use “-o” option to name your own file:

curl --user API_JOHNSMITH:12345

'https://datamine.cmegroup.com/cme/api/v1/download?fid=20150130-eod_xcec_ali_fut_0-eth_p' -o my_eod_file.gz

 

WGET Users:

wget --content-disposition --user API_TEST_DM_PROD --password Test12345 'https://datamine.cmegroup.com/cme/api/v1/download?fid=20160920-EOD_xcbt_c_fut_0-eth_p'

 

Reference

Dataset

API value

Dataset

API value

Top-of-Book

BBO

Block Trades

BLOCK

End-of-day

EOD

Market Depth (FIX)

MD

Market Depth (RLC)

RLC

RLC-SecDef

RLCSECDEF

SecDef

SECDEF

Time & Sales

TICK

 

Exchange

API value

Exchange

API value

COMEX

XCEC

CBOT

XCBT

CME

XCME

NYMEX

XNYM

CMED

CMED

MGCB

MGCB

NYUM

NYUM

XKLS

XKLS

Web Browser API Calls

Simply put this list call in a browser, and hit enter: https://datamine.cmegroup.com/cme/api/v1/list?limit=1

Enter your API username and password, hit enter to see the listing. You should see something similar to this:

{

  "files": [

    {

      "dataset": "MD",

      "yyyymmdd": "20161003",

      "url": "https://datamine.cmegroup.com/cme/api/v1/download?fid=20161003-MD_xnym_zz6_fut_1-eth",

      "fid": "20161003-MD_xnym_zz6_fut_1-eth",

      "orderid": "2793",

      "exchangecode": "xnym",

      "productcode": "zz6",

      "checksum": "87ba2eada6a38c093307d579ad97026e",

      "size": 299,

      "expiration": "2017-07-05"

    }

  ],

  "paging": {

    "previous": "",

    "next": "https://datamine.cmegroup.com/cme/api/v1/list?limit=1&lastFid=20161003-MD_xnym_zz6_fut_1-eth&page=1"

 }

}

 

Clicking on the url for each file will download the file.

 

Sample API Calls

For Listing (can return multiple days, but you cannot specify a range)

For Normal Download (single day of single product)

For Batch Download (single day / all products)

USING CURL

To list all your files 1000 files at a time

curl --user API_ID:Password 'https://datamine.cmegroup.com/cme/api/v1/list'

 

This will provide you a list of file description and a URL to download each file

EX:

 {

      "dataset": "EOD",

      "yyyymmdd": "20150403",

      "url": "https://datamine.cmegroup.com/cme/api/v1/download?fid=20150403-EOD_xcbt_c_fut_0-eth_p",

      "fid": "20150403-EOD_xcbt_c_fut_0-eth_p",

      "s3url": "cmegroup-main-us-east-1-datamine-prod/endofday/30405102/tbcx/fut/eth/c/xcbt-eodp-c-fut-20150403.csv.gz",

      "orderid": "2371",

      "exchangecode": "xcbt",

      "productcode": "c",

      "checksum": "4f8510b7c7f7540c73a01c7a5b19bcab",

      "size": 769,

      "expiration": "2016-12-15"

    }

 

The '' url '' is what you will use to download a file.

You can add parameters to your list command to target more precisely the files you are looking for.

For example:

https://datamine.cmegroup.com/cme/api/v1/list?yyyymmdd=20161114&dataset=EOD

 

If you have more than 1000 files, you can page down using the '' next '' tag in the paging object sent at the end of the response.

EX:

"paging": {

    "previous": "",

    "next": "https://datamine.cmegroup.com/cme/api/v1/list?limit=1000&lastFid=20150403-EOD_xcbt_c_fut_0-eth_p&page=1"

  }

 

The curl command to page down is:

curl --user UNO_ID:Password 'https://datamine.cmegroup.com/cme/api/v1/list?limit=1000&lastFid=20150403-EO_xcbt_c_fut_0-eth_p&page=1'

 

From a list object, you take the '' url '' tag to download a file:

Ex:

 {

      "dataset": "EOD",

      "yyyymmdd": "20150403",

      "url": "https://datamine.cmegroup.com/cme/api/v1/download?fid=20150403-EOD_xcbt_c_fut_0-eth_p",

      "fid": "20150403-EOD_xcbt_c_fut_0-eth_p",

      "s3url": "cmegroup-main-us-east-1-datamine-prod/endofday/30405102/tbcx/fut/eth/c/xcbt-eodp-c-fut-20150403.csv.gz",

      "orderid": "2371",

      "exchangecode": "xcbt",

      "productcode": "c",

      "checksum": "4f8510b7c7f7540c73a01c7a5b19bcab",

      "size": 769,

      "expiration": "2016-12-15"

    }

 

The curl command to download a file is:

curl -O -J --user UNO_ID:Password 'https://datamine.cmegroup.com/cme/api/v1/download?fid=20150403-EOD_xcbt_c_fut_0-eth_p'

 

You can also use a simple web browser to access your data.

If you enter the following URL in the navigation bar, you will receive a list of your first 1000 files:

https://datamine.cmegroup.com/cme/api/v1/list

Once this url is pasted in your navigation bar, you will be asked for a username and password. Enter your UNO_ID and Password to receive your file list.

You can add parameters to you list command to more precisely target the files you are looking for.

For example:

https://datamine.cmegroup.com/cme/api/v1/list?yyyymmdd=20161114&dataset=EOD

 

If you paste the URL tag for a list command in the navigation bar for the web browser, you will download the specific file.

https://datamine.cmegroup.com/cme/api/v1/download?fid=20150403-EOD_xcbt_c_fut_0-eth_p

 

 

Security Definition Files

 

Tag

FIX Name

Future

Outright

NYMEX Decay/Variable Future

Future Spread

Option

Outright

Option Spread

UDS Combo

UDS Covered

Application Sequence Control

5799

MatchEventIndicator

X

X

X

X

X

X

X

911

TotNumReports

c

c

c

c

c

c

c

980

SecurityUpdateAction

X

X

X

X

X

X

X

779

LastUpdateTime

X

X

X

X

X

X

X

1180

ApplID

X

X

X

X

X

X

X

Instrument Data

1300

MarketSegmentID

X

X

X

X

X

X

X

55

Symbol

X

X

X

X

X

X

X

48

SecurityID

X

X

X

X

X

X

X

22

SecurityIDSource

X

X

X

X

X

X

X

200

MaturityMonthYear

X

X

X

X

X

X

X

1151

SecurityGroup

X

X

X

X

X

X

X

6937

Asset

X

X

X

X

X

X

X

167

SecurityType

FUT

FUT

FUT

OOF

OOF

OOF

MLEG

762

SecuritySubType

–

–

X

–

X

X

X

461

CFI Code

X

X

X

X

X

X

X

201

PutOrCall

–

–

–

X

–

–

–

462

UnderlyingProduct

X

X

X

X

–

–

–

207

SecurityExchange

X

X

X

X

X

X

X

1682

MDSecurityTradingStatus

c

c

c

c

c



How was your Client Systems Wiki Experience? Submit Feedback

Copyright © 2024 CME Group Inc. All rights reserved.