Contents

Sequence 

Not all FIXP messages are assigned a sequence number, but instead communicate the sequence number of the next business message or the last business message.

• All business messages will be assigned sequence numbers and FIXP messages refer to these sequence numbers.

• This is unlike the official FIXP specification in which business messages have implicit sequence numbers.

• Sequence message is used in both Recoverable (CME to customer) and Idempotent (customer to CME) flows.

• Sequence numbering supports ordered delivery and recovery of messages.

The Sequence message serves four purposes in an iLink FIXP session including:

• de facto heartbeat message

• test request ping message

• failover trigger message (from primary to backup)

• sequence reset to let the exchange know what sequence number to expect next from the client system

This will let the exchange sync up the NextSeqNo with its own inbound sequence counter and identify a gap if any. Likewise the customer can infer what sequence number to expect next from the exchange and can sync up the value of NextSeqNo with their own inbound sequence counter and identify a gap if any.

The Sequence message can also be sent in response to a NotApplied message as a gap fill if it contains a higher than expected NextSeqNo value and not lower than expected.

The Sequence message is also sent at regular intervals similar to the heartbeat message to ensure that the connection between the customer and exchange is in a normal state. The KeepAliveInterval is specified while Binding in the Establish message. If there is other messaging activity in progress then the Sequence message need not be sent unless a KeepAliveInterval lapses, in which case it will function as a test request ping message.

• It is recommended to set the KeepAliveInterval at 5-60 seconds.

• The exchange will send the Sequence message interleaved with Retransmission responses to a Retransmit Request as well.

In the absence of other messaging activity, if a Sequence message is not received after 2 times the KeepAliveInterval has lapsed, then this is considered a protocol violation and will lead to the FIXP session being terminated.

A sequence message is used in these circumstances:

• From customer - to reset sequence number in response to Not Applied message sent by CME when CME detects a sequence gap from customer.

• From CME - as a heartbeat message to be sent when a KeepAliveInterval interval from CME lapses and no other message is sent to customer.

• From customer - as a heartbeat message to be sent when a KeepAliveInterval interval from customer lapses and no other message is sent to CME.

• From CME - when a KeepAliveInterval of the customer lapses without having received any message from them then send message with KeepAliveIntervalLapsed=1 as a warning before initiating disconnect of socket connection.

• From customer - when a KeepAliveInterval of CME lapses without having received any message from CME then send message with KeepAliveIntervalLapsed=1 as a warning before initiating disconnect of socket connection.

The following integer fields will be checked for these boundary conditions:

• UUID

• NextSeqNo

• KeepAliveIntervalLapsed

• FTI

Heartbeat

This diagram shows an example of Sequence used as a heartbeat.

Warning

This diagram shows an example of Sequence used as a Warning due to KeepAliveInterval lapsing prior to Terminate.

Retransmit Request

The Retransmit Request message is sent to request a retransmission of missed messages when the customer detects a gap in the incoming sequence number series from the exchange. In iLink, the Retransmit Request is used to request all messages subsequent to a particular message by using the following fields:

• UUID: the UUID associated with the sequence numbers being requested.

• FromSeqNo: message sequence number of first message in range to be resent.

• Count: count of messages being requested (if defined as zero then this means infinity or latest sequence number).

The exchange itself will not send a Retransmit Request to recover missing messages, instead a Not Applied message is sent to indicate that the exchange has seen a gap in sequence numbers from the customer and it is the customer who can choose how best to respond to this:

• Send a Sequence message with the appropriate value in NextSeqNo where the customer wants to start sending messages from.

• Retransmitting the missed messages.

• Ignoring the NotApplied message and keep trying to moving ahead.

  • This not advisable since the exchange will keep sending a Not Applied message for the same sequence gap.

• Terminating the FIXP session and Re-Establishing with a new UUID.

  • This is not advisable and should be used only in extreme failure cases to recover from lower than expected sequence number error condition since the sequence streams get reset back to '1' both ways between customer and exchange.

Since the customer is receiving a recoverable flow from CME, they can use Retransmit Request to recover missed messages. When the customer sends a Retransmit Request message, the exchange resends all reject messages and/or execution responses using the same message sequence numbers as the original transmissions along with Sequence messages if any. FIXP messages are not replayed, only business messages.

• The Gateway resends messages for the current UUID or all previously used UUIDs for that week only. Messages from previous weeks are not resent.

• If the customer sends a Retransmit Request message to the Gateway—even if the Gateway has already sent a Not Applied message—then that request will be fulfilled by the exchange.

• The customer should accept messages with a higher sequence number from Market Segment Gateway after recognizing a gap. However, their application should queue messages for in-sequence processing after a requested retransmission is received and completed from the Gateway.

• Retransmission implies that subsequent messages are sequenced with a lower sequence number in response to a Retransmit Request.

• Retransmission responses from the exchange to customer may be inter-leaved with other real-time messages based on the PossRetransFlag (Tag 9765) such that replayed messages will be 9765=1 and original messages will be 9765=0.

• It is the customer's responsibility to determine whether the current batch fills the original gap. If not, then they should send another Retransmit Request for the remainder. Requests and responses should proceed iteratively until all desired messages have been retransmitted. This can be done by combining NextSeqNo+Count from all Retransmission messages.

• If the exchange is unable to retransmit requested messages, it responds with a Retransmit Reject message.

• If a customer Retransmit Request exceeds the maximum resend limit (2,500 messages) then the exchange sends a Retransmit Reject message. The customer should initiate separate Retransmit Requests for each batch of 2500 messages.

• Only one Retransmit Request is allowed in-flight at a time per FIXP session. Another Retransmit Request must not be sent until the previous request has concluded otherwise it will be rejected.

The following integer fields will be checked for these boundary conditions:

• UUID

• LastUUID

• RequestTimestamp

• FromSeqNo

• MsgCount

Retransmission

Retransmission message is sent in response to a valid Retransmit Request from customer.

When RetransmitRequest message from customer is accepted by CME, then CME will return a Retransmission followed by replaying business messages.

• If no messages are replayed, the Retransmission message will be followed by a Sequence message confirming the next sequence number to be sent for the current UUID.

While messages are being replayed, they may be interleaved with other real-time messages; tag 9756-PossRetransFlag on business messages provides the means to distinguish replayed messages from real-time messages:

• tag 9765=1; replayed messages

• tag 9765=0; original messages 

Retransmit Reject

When Retransmit Request message from customer is rejected by CME then CME will return a Retransmit Reject.

Retransmit Request

This diagram shows an example of Retransmission.

NotApplied

Since the message flow from customer to exchange is idempotent, when the exchange recognizes a sequence number gap, it will send a Not Applied message immediately and will not accept any subsequent message having a higher than expected sequence number unless the customer sends a Sequence message, which will instruct the exchange to ignore the gap and start with NextSeqNo as the valid sequence number.

The customer on an idempotent flow uses the Not Applied message to discover which of its requests have not been fulfilled and must decide whether to resend the transactions with new sequence numbers, send different transactions, or send a Sequence message to gap fill.

Since the message flow from exchange to customer is recoverable, this allows the Not Applied message to be re-synchronized if necessary. Thus, the customer can determine with certainty which requests have been accepted.

After receiving Not Applied from CME, the customer can choose to replay business messages with those missing sequence numbers or send a Sequence message with updated next expected sequence number.

Message Sequence Numbers

The FIXP protocol recommends maintaining separate sequence numbers for incoming and outgoing messages between the customer and the exchange. This ensures that all messages to and from the exchange are in the correct order however in iLink only the business messages sent from the exchange are recoverable by the customer. 

To guarantee message delivery, both the customer and exchange must maintain inbound and outbound sequence numbers. The customer's responsibility for maintaining inbound and outbound sequence numbers includes:

• Reset the inbound and outbound sequence numbers to “1” prior to Negotiating at the start of each week and subsequently at the start of a FIXP session with a new UUID.

• Increment inbound sequence number by one for each message received from the exchange for a particular UUID.

• Increment outbound sequence number by one for each message sent to the exchange for a particular UUID.

• Issue a RetransmitRequest when a sequence gap is detected from the exchange for a particular UUID.

  • Retransmit responses will be interleaved with other real-time original message transmissions if any.

• If the exchange detects a sequence gap from customer, then a Not Applied message will be sent and the customer can take action as needed.

  • This can include sending a Sequence message instructing the exchange to ignore the gap and proceed ahead with the NextSeqNo (recommended behavior).

  • This can include trying to replay the messages missed by the exchange (although this is not recommended behavior).

• Maintain sequence numbers between multiple FIXP sessions (a FIXP session occurs each time a customer Establishes session with iLink) on the basis of assigning a unique UUID for each connection.

• If customer chooses to use fault tolerance then maintain sequence numbers for a particular UUID among the primary and backup processes during fail-over scenario.

• The Sequence, Establish, Establishment Acknowledgment and Retransmission messages are sequence forming since they contain the NextSeqNo, which indicates the sequence number of subsequent business messages.

  • FIXP messages themselves are not assigned a sequence number; they only carry a reference as to the next expected sequence number of the next business message to be sent.

  • All business messages will explicitly be assigned a sequence number.

• Since there is no heart beat message, the Sequence message itself serves as the de-facto heart beat and at a minimum it must be sent after the lapse of each KeepAliveInterval if no other messages are being sent.

• Since there is no test request message, the Sequence message also acts as a pinging message with KeepAliveIntervalLapsed=1 to inform the other side that no message has been received from it.

CME Globex preserves all outbound sequence numbers for recovering missed business messages sent from the exchange to the customer during a failover scenario. Business messages sent from the customer to the exchange are not recoverable by the exchange.

If the customer has designed their local architecture to employ the use of a single server process that maintains multiple FIXP sessions to CME and supports multiple applications, the software should be designed to allow a session to reset and/or recover sequence numbers without affecting other actively trading connections on the server or the server process itself.

Sequence Number Higher than Expected

If the Sequence Number on a business message sent to the exchange is higher than expected then a Not Applied message will be sent to the customer to let them know that the exchange has seen a gap in sequence numbers and that those sequence numbers have not been applied by the exchange. The customer has the flexibility to take remedial action if required and resend the missing business messages or send a Sequence message indicating that the gap should be ignored. If neither of these actions are performed then a subsequent Not Applied message will continue to be sent by the exchange if still more business messages are received with higher than expected sequence numbers.

If the Sequence Number on a business message received from the exchange is higher than expected then a Retransmission Request message should be sent by the customer to recover the missing messages. This can be done for the current UUID as well as for previously used UUIDs.

Retransmit Requests will be limited to requesting 2500 messages at a time. Retransmission responses will be interleaved with other original real-time messages and these should be queued up for later processing since these should not trigger additional Retransmission Requests.

While Retransmission is in progress then the customer should accept real-time messages from the Market Segment Gateway with a higher sequence number after recognizing a gap. However, the customer should queue messages for in-sequence processing after Retransmission has concluded.

Sequence Number Lower than Expected

If the incoming Sequence message has a sequence number that is lower than expected then it indicates a serious error. In this case, a Terminate message is sent and the FIXP connection will be closed. The Code in the Terminate message will contain the code describing the reason why the Terminate message is being sent.

If the incoming Establish message has a sequence number that is lower than expected with the same UUID that was in use previously then it indicates a serious error. In this case, a Terminate message is sent and the FIXP connection will be closed. The Code in the Terminate message will contain the code describing the reason why the Terminate message is being sent.

If the incoming Establish message has a sequence number that is lower than expected with a new UUID then this will be permissible since the use of a new UUID indicates that the sequence numbers being sent to the exchange and received from the exchange should be reset back to 1. This will be an acceptable scenario for an intra-day Negotiate and Establish when the new UUID is different from the one used to Negotiate and Establish the previous FIXP session. Using a new UUID is also possible when Negotiating a new FIXP session at the start of each trading day.

Certain recovery situations will cause the exchange and customer sequence numbers to be out of sync. If a customer was terminated due to an invalid sequence number, then the recourse available to them is to Negotiate and Establish with a new UUID.

Previous UUID and Previous Sequence Number

The Establishment Acknowledgment message sent to the customer from the exchange will also contain these two pieces of information – PreviousUUID and PreviousSeqNo. These can be leveraged by the customer to determine if they missed a message published by the exchange for them when they were not connected or if they simply missed a message due to other unforeseen circumstances.

If the customer detects a gap between the current UUID and current sequence number versus previous UUID and previous sequence number they can promptly issue a Retransmission Request to recover the same.

UUID=0 is reserved for CME usage as the default UUID to be assigned to the customer who have not yet connected for the first time.

Sequence Gap Fill

This diagram shows an example of Sequence Gap Fill.