Overlay networks

Task Statement 1.5: Design a routing strategy and connectivity architecture between
on-premises networks and the AWS Cloud.

📘AWS Certified Advanced Networking – Specialty


1. What is an Overlay Network?

An overlay network is a logical network that is built on top of an existing physical network.

  • The underlay network = the physical infrastructure (routers, switches, physical links, ISP network, AWS backbone)
  • The overlay network = a virtual network created using encapsulation and tunneling

👉 Overlay networks allow you to create a virtual, isolated network without changing the physical network.


2. How Overlay Networks Work

Overlay networks use encapsulation:

  • Data is wrapped inside another packet before being sent over the underlay network
  • This wrapping allows the creation of virtual paths between endpoints

Key Mechanism:

  1. Original packet is created (Layer 3 or Layer 2 traffic)
  2. The packet is encapsulated inside another protocol
  3. The encapsulated packet is sent across the underlay network
  4. At the destination, the packet is decapsulated and delivered

3. Common Overlay Technologies (Exam Important)

You must know the most common overlay technologies used in networking and AWS.

3.1 GRE (Generic Routing Encapsulation)

  • Used to encapsulate different network protocols
  • Supports multicast and non-IP traffic
  • Does not provide encryption by itself

📌 Used with IPsec for secure tunnels.


3.2 IPsec (Internet Protocol Security)

  • Provides encryption, authentication, and integrity
  • Used to secure VPN connections
  • Often used with GRE or as standalone tunnels

👉 In AWS:

  • Used in Site-to-Site VPN
  • Encrypts traffic between on-premises and AWS

3.3 VXLAN (Virtual Extensible LAN)

  • Extends Layer 2 networks over Layer 3
  • Uses UDP encapsulation
  • Supports large-scale environments (up to 16 million segments)

📌 Important for:

  • Data center virtualization
  • Extending VLANs across networks

3.4 MPLS (Multiprotocol Label Switching)

  • Uses labels instead of IP routing for forwarding
  • Often used by service providers
  • Can be used as an underlay for overlay networks

4. Why Overlay Networks Are Used

Overlay networks are used to solve several problems:

4.1 Network Isolation

  • Create separate virtual networks without physical changes
  • Useful for multi-tenant environments

4.2 Extending Networks

  • Extend on-premises networks into AWS
  • Allows seamless communication between environments

4.3 Overcoming Underlay Limitations

  • Physical networks may not support certain features (e.g., Layer 2 extension)
  • Overlay networks provide flexibility

4.4 Scalability

  • Easily create and manage many virtual networks
  • No need to reconfigure physical infrastructure

5. Overlay Networks in AWS

Overlay networks are widely used in AWS networking design.

5.1 AWS Site-to-Site VPN

  • Uses IPsec tunnels
  • Creates a secure overlay connection over the internet
  • Connects:
    • On-premises network
    • AWS Virtual Private Cloud (VPC)

5.2 AWS Direct Connect (with VPN over DX)

  • Provides a private physical connection
  • Overlay (IPsec) can still be used on top for encryption

5.3 AWS Transit Gateway (TGW)

  • Acts as a central hub for multiple networks
  • Can support overlay designs using VPN attachments

5.4 SD-WAN Integration

  • SD-WAN solutions often use overlay tunnels (IPsec, GRE)
  • Used to connect branch offices to AWS

6. Overlay vs Underlay (Exam Comparison)

FeatureUnderlay NetworkOverlay Network
TypePhysical networkLogical/virtual network
ComponentsRouters, switches, linksTunnels, encapsulation
FunctionProvides connectivityProvides abstraction and segmentation
ControlPhysical controlLogical control
ExampleAWS backbone, ISP networkIPsec VPN, VXLAN

7. Key Concepts to Remember for the Exam

Encapsulation & Decapsulation

  • Core mechanism of overlay networks
  • Adds headers for tunneling

Tunnel Types

  • GRE tunnel
  • IPsec tunnel
  • VXLAN tunnel

Control Plane vs Data Plane

  • Overlay networks mainly operate at the data plane level
  • Control plane manages tunnel creation and routing

MTU Considerations

  • Encapsulation adds extra headers
  • This reduces the effective MTU
  • Important for:
    • Avoiding packet fragmentation
    • Performance optimization

Routing in Overlay Networks

  • Can use:
    • Static routing
    • Dynamic routing (BGP)
  • Overlay networks often rely on BGP for scalability

8. Advantages of Overlay Networks

  • Network flexibility
  • Easier network segmentation
  • Better scalability
  • Independent of physical infrastructure
  • Easier cloud integration

9. Disadvantages / Challenges

  • Added complexity
  • Encapsulation overhead
  • Performance impact due to extra processing
  • Troubleshooting can be more difficult
  • MTU issues and fragmentation

10. Exam Scenarios You May See

You should be able to identify when overlay networks are the correct solution:

  • Extending on-premises network into AWS using VPN
  • Connecting multiple VPCs using Transit Gateway
  • Using SD-WAN for hybrid connectivity
  • Implementing secure encrypted tunnels (IPsec)
  • Creating virtual Layer 2 networks over Layer 3 infrastructure

11. Key Takeaway (Very Important for Exam)

Overlay networks allow you to:

✔ Build virtual networks on top of existing infrastructure
✔ Extend on-premises networks into AWS securely
✔ Use tunneling protocols like IPsec, GRE, and VXLAN
✔ Enable scalable and flexible hybrid networking architectures

Buy Me a Coffee