TCP Handshake:

The Transmission Control Protocol (TCP) handshake is the foundation of reliable communication over the internet. This blog breaks down the handshake process, its variants, security considerations, and real-world implications.

Understanding TCP Handshake: A Deep Dive

• SYN – Initiates connection.

• ACK – Acknowledges received data.

• FIN – Requests connection termination.

• RST – Abruptly resets the connection.

• URG – Marks urgent data.

• PSH – Pushes data immediately to the application.

• Every TCP packet has a Sequence Number (SEQ) and an Acknowledgment Number (ACK).

• SEQ represents the byte number in the data stream.

• ACK confirms receipt by sending the next expected byte number (previous SEQ + data length).

EQ = 1000 ACK = 0

Client -----------------> Server

SEQ = 5000 ACK = 1001

Client <----------------- Server

2. Sequence and Acknowledgment Numbers

1. SYN (Synchronize) – The client sends a TCP segment with the SYN flag set. It includes:

   • Initial Sequence Number (ISN) – A randomly chosen 32-bit number.

   • Window Size – The client’s advertised TCP window.

   • Maximum Segment Size (MSS) – Defines the maximum payload per segment.

2. SYN-ACK (Synchronize-Acknowledge) – The server responds with:

   • SYN flag set – Server’s own ISN.

   • ACK flag set – Acknowledges client’s ISN by sending ISN + 1.

3. ACK (Acknowledgment) – The client acknowledges the server’s ISN by sending ISN + 1, completing the handshake.

Once this process is complete, data transmission begins.

Client Server

|-------SYN------->|

|<----SYN-ACK-----|

|-------ACK------->|

Connection Established

1. The Three-Way Handshake

TCP establishes a connection using a three-step process between a client and a server:

Conclusion

Understanding the TCP handshake is crucial for optimizing network performance, diagnosing connectivity issues, and mitigating security threats. By leveraging modern enhancements like TCP Fast Open and Window Scaling, networks can improve efficiency and reduce latency.

10. Performance Optimizations

• Enable Window Scaling – Crucial for high-bandwidth, long-distance networks.

• Use TCP Fast Open – Reduces latency for repeat connections.

• Adjust MSS and MTU – Prevents packet fragmentation.

• Enable SACK – Improves efficiency in packet loss scenarios.

9. Real-World TCP Handshake Scenarios

• High Latency Networks – Delayed ACKs can slow initial data transfer.

• TLS Over TCP – Adds additional round-trips for encryption setup.

• Load Balancers and Proxies – May terminate and establish backend connections separately.

8. Security Considerations

• SYN Flood Attacks – Attackers send multiple SYNs without completing the handshake, exhausting server resources.

  • Mitigations: SYN Cookies, rate limiting.

• RST Injection – Attackers forge RST packets to terminate connections.

• Session Hijacking – Predictable ISNs can allow attackers to hijack connections.

| State | Description |

|-----------------|-----------------------------------------|

| LISTEN | Waiting for connections |

| SYN-SENT | Client sent SYN, waiting for reply |

| SYN-RECEIVED | Server received SYN, sent SYN-ACK |

| ESTABLISHED | Connection is active |

| FIN-WAIT-1 | Waiting for termination request |

| FIN-WAIT-2 | Awaiting final termination |

| TIME-WAIT | Ensuring no duplicate packets |

| CLOSED | Connection fully terminated |

7. TCP Connection States

1. FIN (Finish) – One side initiates termination.

2. ACK – The other side acknowledges.

3. FIN – The second side sends its own FIN.

4. ACK – Final acknowledgment, closing the connection.

Client Server

|-------FIN-------->|

|<------ACK--------|

|<------FIN---------|

|-------ACK-------->|

Connection Closed

6. Connection Termination (Four-Way Handshake)

• Simultaneous Open – Both sides send SYN at the same time, leading to a SYN-SYN-ACK-ACK exchange.

• TCP Fast Open (TFO) – Reduces latency by sending data in the SYN phase.

• Half-Open Connections – Occurs when one side completes the handshake while the other does not respond (often exploited in SYN flood attacks).

5. Handshake Variants

• MSS (Maximum Segment Size) – Limits payload size to prevent fragmentation.

• Window Scaling – Increases TCP window beyond 64 KB for high-bandwidth networks.

• Selective Acknowledgment (SACK) – Allows acknowledgment of individual packets in case of loss.

• Timestamps – Helps in Round Trip Time (RTT) calculation and sequence number protection.

4. TCP Options in Handshake

3. TCP Flags in the Handshake

| Option | Description |

|--------------|------------------- --------|

| MSS | Maximum Segment Size |

| WS | Window Scaling |

| SACK | Selective Acknowledgment|

| TS | Timestamps |