Task Statement 3.2: Monitor and analyze network traffic to troubleshoot and optimize connectivity patterns.

📘AWS Certified Advanced Networking – Specialty

1. What is a Packet Size Mismatch?

A packet size mismatch happens when:

A device sends a network packet that is too large for the next network segment
The packet cannot be forwarded
And it gets dropped or blocked

This leads to symptoms like:

Websites not loading fully
SSH/RDP connections hanging
APIs timing out
Partial data transfer failures
Intermittent connectivity issues

2. Key Concept: MTU (Maximum Transmission Unit)

MTU definition

MTU is the largest size of a network packet (in bytes) that can be transmitted in one frame without fragmentation.

Common MTU values in AWS:

1500 bytes (Standard MTU) → Most VPC networking
9001 bytes (Jumbo Frames) → Supported in some AWS services like:
- EC2 instances (enhanced networking)
- Direct Connect (with configuration)
- Some internal high-performance networks

3. Why Packet Size Issues Happen in AWS VPC

Packet size problems usually occur when traffic moves across different network domains with different MTU limits.

Common causes:

1. Mixed MTU environments

Example:

EC2 instance uses 9001 MTU
VPN tunnel supports only 1500 MTU
➡ Large packets fail when entering VPN

2. VPN and Direct Connect encapsulation overhead

When traffic goes through:

IPsec VPN
Transit Gateway
GRE tunnels

Extra headers are added → reducing usable MTU.

So even if MTU is 1500, effective payload becomes smaller.

3. Path MTU Discovery (PMTUD) failure

AWS relies on ICMP “Fragmentation Needed” messages.

If ICMP is blocked:

Sender does NOT learn the correct MTU
Keeps sending large packets
Packets get dropped silently

4. Security group / NACL blocking ICMP

If ICMP is blocked:

MTU discovery breaks
Connectivity issues appear only for large payloads

5. Cross-VPC or Transit Gateway routing

Different VPCs or attachments may have:

Different MTU settings
Different encapsulation layers

4. How Packet Loss Appears in Real AWS Environments

You may see:

Application-level symptoms:

API requests fail when payload size increases
File uploads stop midway
Database replication fails intermittently

Network-level symptoms:

Ping works for small packets but fails for large ones:
- ping -s 56 works
- ping -s 1472 fails

Cloud symptoms:

CloudWatch logs show timeouts
VPC Flow Logs show ACCEPT but no response return

5. AWS Services Commonly Involved

Packet size mismatches often involve:

Amazon VPC
EC2 instances (ENI interfaces)
NAT Gateway
AWS VPN (Site-to-Site VPN)
AWS Direct Connect
AWS Transit Gateway
Elastic Load Balancer (ALB/NLB)

6. How to Troubleshoot Packet Size Issues (Exam Focus)

This is the most important part for your exam.

Step 1: Identify symptoms

Look for:

Timeouts on large requests only
Partial connectivity
Intermittent application failure

Step 2: Check MTU configuration

On EC2 instances:

Linux: ip link show
Windows: netsh interface ipv4 show subinterfaces

Check if:

MTU is 1500 or 9001
Mismatch exists between endpoints

Step 3: Use ping with packet size testing

Test progressively:

Small packet test:
- Works = basic connectivity OK
Large packet test:
- Fails = MTU issue likely

Use DF (Don’t Fragment) flag:

Ensures packet is not fragmented
Helps detect exact MTU limit

Step 4: Check ICMP blocking

Verify:

Security Groups allow ICMP
Network ACLs allow ICMP type 3 (Fragmentation Needed)

If blocked:

Path MTU Discovery will fail

Step 5: Inspect VPN / Transit Gateway / Direct Connect

Look for:

Encapsulation overhead
MTU reduction requirements
Tunnel configuration mismatch

Step 6: Review VPC Flow Logs

Check:

ACCEPT vs REJECT
Traffic size patterns
Repeated retransmissions

Step 7: Validate NAT Gateway behavior

NAT does not change MTU but:

Encapsulation + return path issues may drop large packets

7. Common Fixes (Very Important for Exam)

Fix 1: Reduce MTU on EC2 instances

Set MTU to:

1500 (safe standard)
Or lower (e.g., 1400) for VPN environments

Fix 2: Enable ICMP for PMTUD

Allow:

ICMP Type 3 (Destination Unreachable)
ICMP fragmentation-needed messages

Fix 3: Adjust VPN MTU/MSS settings

For Site-to-Site VPN:

Enable TCP MSS clamping
Reduce MSS to avoid fragmentation

Fix 4: Use correct MTU for Direct Connect

Ensure both ends support jumbo frames if using 9001 MTU
Otherwise standardize to 1500

Fix 5: Align MTU across all VPC components

Ensure consistency across:

EC2 ENIs
Transit Gateway attachments
VPN tunnels
On-prem routers

8. Exam Tips (High Priority)

You should remember:

1. Most common root cause:

👉 MTU mismatch across network boundaries

2. Most common hidden issue:

👉 ICMP blocked → breaks Path MTU Discovery

3. Most tested AWS concept:

👉 VPN encapsulation reduces effective MTU

4. Best troubleshooting order:

Identify symptom
Test packet size (ping with DF)
Check ICMP rules
Check MTU settings
Check VPN/TGW/Direct Connect configuration

9. Simple Summary (Exam Memory Version)

Packet size mismatch = packet too large for network path
Root cause = MTU difference between systems
AWS issue often happens with VPN, Direct Connect, TGW
ICMP must be allowed for MTU discovery
Fix = reduce MTU, allow ICMP, or adjust MSS