Versions Compared

Old Version 4

changes.mady.by.user CME Group

Saved on

compared with

New Version Current

changes.mady.by.user CME Group

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

iLink 3 uses the FIX Performance (FIXP) protocol to establish and manage bi-directional sessions. Per the FIXP protocol, a FIX session is defined as a bi-directional stream of ordered messages between two parties within a continuous sequence number series. With iLink 3, the Market Segment Gateways (MSGW) will support weekly client FIX sessions and also support mid-week initialization.

...

Contents

Table of Contents

Universally Unique Identifier (UUID)

Each FIXP session established at the start of the week, intra-day, or at the beginning of each trading day is represented with a unique UUID value set by the customer as a 64-bit value. CME Group recommends using the system timestamp, which represents the number of microseconds since epoch (Jan 1, 1970) as the timestamp. Business messages received from the exchange are recoverable for that sequence number + UUID combination.

UUID Rules

The following rules apply to the UUID:

  • UUID must be sent in the Negotiate message and that ID is used for the lifetime of the connection.

  • Customer must reset sequence numbers back to 1 for each direction (from customer to CME and from CME to customer) every time they negotiate with new UUID.
  • UUID in the Negotiate message is greater than the UUID used in the last successful Negotiate/Establish message.
    • This will prevent usage of duplicate UUID's intraweek, which can affect subsequent retransmission of those messages.
    • For this reason it is strongly recommended to use current timestamp value (number of microseconds since epoch) as UUID.
  • RequestTimestamp must be current timestamp in nanoseconds.
  • Customers will be able to use the same UUID across all segments at the same time.
  • One UUID can be used by more than one customer.
  • A UUID previously used on segment ‘X’ can be used to negotiate on segment ‘Y’ if never used on ‘Y’ by the given customer and higher than the most recent UUID used by that customer on ‘Y’.

Re-Setting UUID

In the event of customer application failure, it may be necessary to reset the UUID intra-session on the same trading day to force synchronization of sequence numbers to and from the exchange.

...

  • Do not Terminate the FIXP session and Re-Negotiate with a new UUID as a normal response to a Not Applied message.

  • Re-Negotiate with a new UUID should be used only to recover from a disaster situation that cannot be recovered via the use of Sequence message.

  • Re-Negotiating with a new UUID will mean recovering messages sent by the exchange in the previous FIXP session with the previous UUID.
  • Re-Negotiating with a new UUID resets the inbound and outbound sequence numbers back to "1" so sending a higher than expected sequence number to the exchange in the Establish message will result in successful Establishment followed by a Not Applied message, and the new NextSeqNo in the Establish message will be adopted as the new expected sequence number.

Weekly Reset

CME will internally reset its inbound and outbound sequence numbers to “1” at the start of each week. The customer should also reset their sequence numbers in both directions prior to negotiating and establishing a FIX session on Sunday.     

...

  • If the customer does not reset the outbound sequence number to the exchange to “1” at the start of each week, they will receive a Terminate message informing them of this error in the Code field.
  • If a new UUID is assigned to subsequent FIXP sessions after the first, the sequence stream is reset to '1' in both directions (from customer to exchange and exchange to customer).

Session Security

The customer will authenticate an iLink session by digitally signing FIXP Negotiate and Establish messages using a hashed message authentication code (HMAC). In general, the customer interaction with CME will be as follows:

...

The details of of calculating the digital signature are described below.

Steps to Sign the Negotiate and Establish Messages

  1. Create the canonical Negotiate or Establish message.
  2. Create Signature using Secret Key provided by CME and Canonical Message.
  3. Populate HMAC Signature plus Access Key ID fields of the Negotiate and Establish messages.
Info

Only the field value—not the field name—must be used for the calculation of HMAC signature.

Example: where SessionID=<ABC>, use only 'ABC'.

Step 1 - Create the Canonical Message

These field values must be assembled in this order with the values concatenated into a single string delimited by the new line character (i.e. ‘\n’).

...

  • RequestTimestamp (A timestamp older than 5 seconds will be rejected as stale.) 

  • UUID
  • SessionID
  • FirmID
  • TradingSystemName
  • TradingSystemVersion
  • TradingSystemVendor
  • NextSeqNo
  • KeepAliveInterval

An Example Canonical message for Establish: 

“RequestTimestamp” + "\n" + “UUID” + "\n" + “SessionID+ "\n" + FirmID+ "\n" + “TradingSystemName” + "\n" + “TradingSystemVersion” + "\n" + “TradingSystemvendor” + "\n" + “NextSeqNo” + "\n" + “KeepAliveInterval”

...

Panel

public String prepareCanonicalMessage For Establish(){

long requestTimestamp = 1563720650008L;

long uUID = 1563720660068L

String session = "ABC";

String firmID = "007";

String tradingSystemName = " ";

String tradingVersionId = " ";

String tradingSystemVendorId = " ";

long nextSeqNo = 1;

long keepAliveInterval = 30;


StringBuilder canonicalMsg = new StringBuilder();

canonicalMsg.append(requestTimestamp).append(New_Line);

canonicalMsg.append(uUID).append(New_Line);

canonicalMsg.append(session).append(New_Line);

canonicalMsg.append(firmID).append(New_Line);

canonicalMsg.append(tradingSystemName).append(New_Line);

canonicalMsg.append(tradingSystemVersionId).append(New_Line);

canonicalMsg.append(tradingSystemVendorId).append(New_Line);

canonicalMsg.append(nextSeqNo).append(New_Line);

canonicalMsg.append(keepAliveInterval);

return canonicalMsg.toString();

}

Step 2 - Create Signature Using the Secret Key and Canonical FIX Message 

The signature is a hash (digest) of the canonical message created in Step 1 using the Secret Key provided by CME. 

The Secret Key downloaded from Request Center is Base64 URL Encoded. Customers must decode the secret key first.

Example of Creating Signature Using HMAC SHA256 in Java 8+

The HMAC signature is populated in a fixed length string field (i.e. String32Req) since SHA-256 signature is always 32 bytes.

...

Panel

public byte[] calculateHMAC(final String canonicalRequest, final String userKey) {
    byte[] hash = null;
 
    try
    {
      // Init HMAC instance
      Mac sha256HMAC;
      sha256HMAC = Mac.getInstance("HmacSHA256");
 
      // Initialize HMAC instance with the key
      // Decode the key first, since it is base64url encoded
 
      byte[] decodedUserKey = Base64.getUrlDecoder().decode(userKey);
      SecretKeySpec secretKey = new SecretKeySpec(decodedUserKey, "HmacSHA256");
      sha256HMAC.init(secretKey);
 
 
      // Calculate HMAC
      hash = sha256HMAC.doFinal(canonicalRequest.getBytes("UTF-8"));
 
    } catch (NoSuchAlgorithmException | InvalidKeyException | IllegalStateException | UnsupportedEncodingException e)
    {
      e.printStackTrace();
    }
 
    return hash;
  }
 
 
}

Step 3 - Populate HMAC Signature Plus Access Key ID fields of the Negotiate and Establish Messages

  • tag 39004-AccessKeyID - contains the AccessKeyID 
  • tag 39005-HMACSignature - contains the HMAC signature

FIXP Session Messages

FIXP session messages are summarized as follows:

...

Gliffy
size700
nameFIXP Session State i
pagePin9

Initialization and Binding

The customer uses the Negotiate and Establish messages to authenticate themselves with the Market Segment Gateway. In iLink 3, there are three different types of Initialization and Binding operations.

Beginning of Week Initialization and Binding

Beginning of Week initialization and Binding will involve the very first messages (Negotiate and Establish) that the customer sends for the week. The customer must reset their inbound and outbound sequence numbers to “1” prior to initialization at the start of the week as well as pick a globally unique UUID for each FIXP session.

...

  • If a fill is generated for the GTC or GTD order in the current week before Start of Week Initialization, then the exchange processes the Execution Report and increments its previous sequence number for the previous UUID (which is the default UUID of 0) accordingly.
  • When the customer sends Negotiate and Establish messages to Initialize at the Start of the Week for the current week, they will receive an Establishment Acknowledgment message with:
    • PreviousSeqNo = “1”.
    • PreviousUUID="0". This way the client knows that a message was published using the default CME assigned UUID of 0 since iLink 3 requires the exchange also to reset its sequence numbers to “1” and UUID to "0" prior to Start of Week Initialization.
  • As a result, the client application sends a Retransmit Request for the PreviousUUID and PreviousSeqNo and receives the undelivered Execution Report on the GTC order.
  • The sequence gap is now filled and normal processing continues.

Mid-Week Initialization and Binding

Mid-Week Initialization and Binding will involve Negotiating and Establishing a new FIXP session once the previous one has ended gracefully following Termination. The customer must populate NextSeqNo with the correct value from the previous sequence stream associated with the previous UUID and reattempt the Initialization, or reset sequence number to 1 with the new UUID.

If the customer's inbound sequence number is reset to “1” when Initializing a new FIXP session with the same UUID as the previous one and the customer sends an Establish message (Negotiate does not have NextSeqNo), then the Market Segment Gateway responds with a Terminate message. The Terminate message will have a code indicating an error condition regarding usage of lower than expected sequence numbers. The customer must populate NextSeqNo with the correct value from the previous sequence stream associated with the previous UUID, or populate NextSeqNo with the correct value from the previous sequence stream associated with the previous UUID and reattempt the Initialization.

GTC and GTD Orders - Mid-Week

Good Till Cancel (GTC) or Good Till Date (GTD) orders sent by a customer from the previous trading day continue to be eligible for execution after Termination.

  • If a fill is generated for the GTC or GTD order in the current trading day before Initialization then the exchange processes the Execution Report and increments its previous sequence number for the previous UUID, which is the last known UUID from the previous session.
  • When the customer sends Negotiate and Establish messages to Initialize then they will receive an Establishment Acknowledgment message with:
    • PreviousSeqNo as “last known value+1”. This is higher than the client’s last known expected inbound sequence number, which is "last known value" for the PreviousUUID.
    • PreviousUUID="last known value".
  • As a result, the client application sends a Retransmit Request for the PreviousUUID and PreviousSeqNo and receives the undelivered Execution Reports on the GTC order.

  • The sequence gap is now filled and normal processing continues.

Intra-Session Binding

Intra-session binding is applicable when the FIXP session loses connectivity with the exchange for any reason such as Termination due to a protocol violation, failure, etc. In such an event:

...

  • While the customer is not established, Execution Reports, etc. are processed by the exchange.
  • As a result, the customer may receive an Establishment Acknowledgment message with PreviousSeqNo higher than expected (i.e., the Market Segment Gateway processed those Execution Reports and incremented its outbound sequence number accordingly) for the previously used UUID.
  • The client application submits a Retransmit Request message for those messages persisted while the customer was not established.

Session Negotiation

A FIXP session connection is initiated by the customer using a Negotiation message. The UUID must be sent in the Negotiation message and that ID is used for the lifetime of the FIXP session. Negotiation is provided as a mechanism used for the customer to declare what ID they will be using. There is no concept of resetting a FIXP session. Instead of starting over a FIXP session, a new FIXP session is negotiated with a new UUID.

...

  • When Negotiate message from customer is accepted by CME then CME will return a Negotiation Response.
  • When Negotiate message from customer is rejected by CME then CME will return a Negotiation Reject.
  • If no response is forthcoming from the exchange after two times the recommended KeepAliveInterval has lapsed then Negotiation can be retried or out-of-band inquiry may be made to determine application readiness.

Negotiate Message Accepted

This diagram shows an example of Negotiation Accepted.

Gliffy
nameNegotiationandNegotiationResponse
pagePin2

Negotiation Rejected

This diagram shows an example of negotiation Rejected.

Gliffy
nameNegotiationReject
pagePin1

Session Establishment

After negotiation is complete, the customer must send an Establish message to start assigning sequence numbers. Once established, the customer can proceed to submit orders and quotes. The established state is concurrent with the lifetime of a TCP connection. A customer may re-establish a previous FIXP session after reconnecting without any further negotiation (Intra-Session Initialization). Thus, Establish binds the FIXP session to the new transport instance.

...

  • UUID
  • RequestTimestamp
  • KeepAliveInterval
  • NextSeqNo

Establishment Accepted

This diagram shows an example of Establishment Accepted.

...

  • The customer should evaluate NextSeqNo, PreviousSeqNo and PreviousUUID to determine whether they have missed any messages after re-establishment of a recoverable flow. If so, the customer can then immediately send a Retransmit Request.

Establishment Rejected

This diagram shows an example of Establishment Rejected.

...

  • When the Establish message from the customer is rejected by CME, CME will return an Establishment Reject.

Transferring 

Sequence 

Not all FIXP messages are assigned a sequence number, but instead communicate the sequence number of the next business message or the last business message.

...

  • UUID
  • NextSeqNo
  • KeepAliveIntervalLapsed
  • FTI

Heartbeat

This diagram shows an example of Sequence used as a heartbeat.

Gliffy
nameHeartBeat
pagePin3

Warning

This diagram shows an example of Sequence used as a Warning due to KeepAliveInterval lapsing prior to Terminate.

Gliffy
nameWarning
pagePin2

Retransmit Request

The Retransmit Request message is sent to request a retransmission of missed messages when the customer detects a gap in the incoming sequence number series from the exchange. In iLink 3, the Retransmit Request is used to request all messages subsequent to a particular message by using the following fields:

...

  • UUID

  • LastUUID

  • RequestTimestamp

  • FromSeqNo

  • MsgCount

Retransmission

Retransmission message is sent in response to a valid Retransmit Request from customer.

...

  • tag 9765=1; replayed messages
  • tag 9765=0; original messages 

Retransmit Reject

When Retransmit Request message from customer is rejected by CME then CME will return a Retransmit Reject.

Retransmit Request

This diagram shows an example of Retransmission.

Gliffy
nameRetransmission
pagePin4

NotApplied

Since the message flow from customer to exchange is idempotent, when the exchange recognizes a sequence number gap, it will send a Not Applied message immediately and will not accept any subsequent message having a higher than expected sequence number unless the customer sends a Sequence message, which will instruct the exchange to ignore the gap and start with NextSeqNo as the valid sequence number.

...

After receiving Not Applied from CME, the customer can choose to replay business messages with those missing sequence numbers or send a Sequence message with updated next expected sequence number.

Message Sequence Numbers

The FIXP protocol recommends maintaining separate sequence numbers for incoming and outgoing messages between the customer and the exchange. This ensures that all messages to and from the exchange are in the correct order however in iLink 3 only the business messages sent from the exchange are recoverable by the customer. 

...

If the customer has designed their local architecture to employ the use of a single server process that maintains multiple FIXP sessions to CME and supports multiple applications, the software should be designed to allow a session to reset and/or recover sequence numbers without affecting other actively trading connections on the server or the server process itself.

Sequence Number Higher than Expected

If the Sequence Number on a business message sent to the exchange is higher than expected then a Not Applied message will be sent to the customer to let them know that the exchange has seen a gap in sequence numbers and that those sequence numbers have not been applied by the exchange. The customer has the flexibility to take remedial action if required and resend the missing business messages or send a Sequence message indicating that the gap should be ignored. If neither of these actions are performed then a subsequent Not Applied message will continue to be sent by the exchange if still more business messages are received with higher than expected sequence numbers.

...

While Retransmission is in progress then the customer should accept real-time messages from the Market Segment Gateway with a higher sequence number after recognizing a gap. However, the customer should queue messages for in-sequence processing after Retransmission has concluded.

Sequence Number Lower than Expected

If the incoming Sequence message has a sequence number that is lower than expected then it indicates a serious error. In this case, a Terminate message is sent and the FIXP connection will be closed. The Code in the Terminate message will contain the code describing the reason why the Terminate message is being sent.

...

Certain recovery situations will cause the exchange and customer sequence numbers to be out of sync. If a customer was terminated due to an invalid sequence number, then the recourse available to them is to Negotiate and Establish with a new UUID.

Previous UUID and Previous Sequence Number

The Establishment Acknowledgment message sent to the customer from the exchange will also contain these two pieces of information – PreviousUUID and PreviousSeqNo. These can be leveraged by the customer to determine if they missed a message published by the exchange for them when they were not connected or if they simply missed a message due to other unforeseen circumstances.

...

UUID=0 is reserved for CME usage as the default UUID to be assigned to the customer who have not yet connected for the first time.

Sequence Gap Fill

This diagram shows an example of Sequence Gap Fill.

Gliffy
nameSequenceGapFill1
pagePin1

Unbinding 

Termination

Terminate message is used to signal that the sender is going to disconnect the TCP socket connection. This unbinds the transport from the session, but it does not end a logical session. The Terminate message initiates and confirms the termination of a FIXP session.

...

When the client system sends a Terminate message, the Market Segment Gateway sends a corresponding Terminate confirmation message and terminates the FIXP session. After sending a Terminate message, the customer should not send any additional messages, since this will be considered to be a protocol violation.

Terminate - Client Initiated

This diagram shows an example of client-initiated Terminate.

...

The following integer fields will be checked for these boundary conditions:

  • UUID

  • RequestTimestamp

  • Code

Terminate - CME Initiated

This diagram shows an example of CME-initiated Terminate.

Gliffy
nameTermination CME Initiated
pagePin1

Fault Tolerance

A customer choosing to use fault tolerance must coordinate their application processes to establish separate and independent content streams to the Market Segment Gateways via TCP/IP socket connections. This group of redundant client processes (also commonly referred to as the fault tolerant group) operates together to provide fault tolerant functionality. In a typical deployment scenario, multiple redundant processes are spawned from the same executable file and each of those processes runs on a separate machine. Running redundant processes in the same machine is not recommended. If a machine fails, all the processes running on it fail simultaneously.

The client application must Negotiate and Establish with the designated primary Market Segment Gateway and then Establish with the backup Market Segment Gateway. A backup member must be ready to activate in the same data state as the former primary member being replaced. For example, inbound and outbound sequence numbers for a UUID must be maintained in a consistent state during fail-over between both processes.

...

Info

For active-active fault tolerance,

...

Negotiation is not allowed on the backup connection. 

The customer must use the same UUID for both primary and backup FIXP sessions (this is the only exception to the general rule of thumb that the UUID be globally unique) and then the sequence streams would be contiguous after failure.

...

In the event of a failure in an active-active fault tolerance scenario, the customer will receive a Sequence message from the backup or newly promoted primary Market Segment Gateway with the NextSeqNo containing a valid value (from where the old primary left off) along with the FaultToleranceIndicator=1 (primary) and this will serve as the trigger to let the customer know that a failure occurred and the transition to backup taking over as new primary was successful.

Customers Choosing Not to Implement Fault Tolerance

The exchange does not mandate that the customer use the fault tolerance feature, however CME strongly recommends using this functionality. A customer that does not implement fault tolerance will not be able to dynamically recover from process or network failures.

For customers that do not wish to implement fault tolerance, their application should just Negotiate and Establish with the designated primary Market Segment Gateway and not Establish with the backup Market Segment Gateway. However they are not precluded from doing so should they choose to Establish with the backup Market Segment Gateway at a later time after first Negotiating and Establishing with the primary gateway.

Failover Processing

Mission-critical client applications must continue to function properly despite sudden difficulties such as process termination, hardware failure and network disconnects. Fault tolerance in a network environment is characterized by rapid recovery from such failures. One method of providing fault-tolerance is through a mechanism called fail-over; the end goal is to minimize service interruption caused by error conditions listed above.

...

The Market Segment Gateway initiates a controlled fail-over when it detects either process or network failure that impacts its ability to service the customer.

Customer Primary Process Failure

If the exchange does not receive a message from the primary client process within a defined interval (two times the KeepAliveInterval sent by client in the Establish message), then the exchange designates the primary client process as failed and initiates fail-over. The primary client application and backup client applications will be terminated and disconnected from the exchange and the backup client application will be expected to communicate with the designated primary Market Segment Gateway. The backup client application is notified of fault tolerance status change by the Code in the Terminate message (i.e. ErrorCodes = 26 - DisconnectFromPrimary: Backup session will be terminated as well).

  • If the primary client process fails without closing the TCP connection, then it takes two times the KeepAliveInterval for the exchange to detect the primary process failure. If the customer would like to avoid the time delay in this process, then they should ensure the TCP connection is closed whenever their application fails.
  • Before the fail-over, the backup client application was receiving NextSeqNo set to zero. During and after the fail-over process, the backup client application is responsible for ensuring that its UUID, inbound sequence number to the exchange and outbound sequence number from the exchange are synchronized with the primary application that just failed. This is critical since the newly elected primary member must know exactly where the failed member left off.
  • If the NextSeqNo number of the Sequence message sent by the new primary client application is lower than that of the original client application for the same UUID, the exchange will terminate the client application as per FIXP protocol.
  • If the NextSeqNo number of the Sequence message sent by the new primary client application is lower than that of the original client application for a different UUID, the exchange will accept that and will also inform the customer of the perceived gap by sending a NotApplied message; if the NextSeqNo=1 in this case for a new UUID then that is acceptable.

Customer Backup Process Failure

If the exchange does not receive any message from a backup client application within a defined interval (which is two times the KeepAliveInterval in the Establish message sent by the customer), then the backup client application will be disconnected and the status of the primary client application connectivity remain intact.

...

If a backup Market Segment Gateway fails, it will be restored to working status by market operations. For active-active fault tolerance, the backup client application must again re-establish the same session (same UUID) connection to this restored backup Market Segment Gateway to maintain a backup.  The status of the primary client application connectivity remains intact.

Order Entry Service Gateway 

For the pre-registered administrative information, the Party Details Definition and Party Details List functionality will be handled by the Order Entry Service Gateway (OESGW).

...