Flow logs and traffic mirroring in architectures to provide visibility

Task Statement 1.4: Define logging and monitoring requirements across AWS and hybrid networks.

📘AWS Certified Advanced Networking – Specialty


1. Introduction

In modern cloud networks, visibility into network traffic is critical for:

  • Monitoring network activity
  • Troubleshooting connectivity issues
  • Detecting security threats
  • Meeting compliance and auditing requirements

AWS provides two important features to monitor network traffic:

  1. Amazon VPC Flow Logs
  2. Amazon VPC Traffic Mirroring

Although both provide visibility into network traffic, they serve different purposes:

FeaturePurpose
VPC Flow LogsCaptures metadata about IP traffic
Traffic MirroringCaptures full packet data for deep inspection

Understanding how they work, when to use them, and their limitations is important for the AWS Advanced Networking Specialty exam.


2. Amazon VPC Flow Logs

2.1 What Are VPC Flow Logs?

VPC Flow Logs capture information about IP network traffic going to and from network interfaces within a VPC.

They record metadata about traffic, such as:

  • Source IP address
  • Destination IP address
  • Source port
  • Destination port
  • Protocol
  • Number of packets
  • Number of bytes
  • Accept or reject status

Flow Logs do not capture packet payloads.

They help administrators analyze traffic patterns and troubleshoot network connectivity.


2.2 Where Flow Logs Can Be Enabled

Flow Logs can be enabled at three levels:

1. VPC Level

Captures traffic for all network interfaces in the VPC.

Use when:

  • Monitoring entire VPC traffic
  • Security auditing
  • Compliance monitoring

2. Subnet Level

Captures traffic for all network interfaces inside a subnet.

Useful when:

  • Monitoring a specific application subnet
  • Investigating issues in a network tier

3. Elastic Network Interface (ENI) Level

Captures traffic for a specific instance or resource.

Useful for:

  • Debugging a single EC2 instance
  • Investigating suspicious activity

2.3 Where Flow Logs Are Stored

Flow logs can be sent to:

1. Amazon CloudWatch Logs

Benefits:

  • Real-time monitoring
  • Log searching
  • Integration with alarms and metrics

Common use cases:

  • Security monitoring
  • Real-time alerting

2. Amazon S3

Benefits:

  • Long-term storage
  • Log analysis with big data tools
  • Lower cost for large data

Often used with:

  • Amazon Athena
  • Amazon QuickSight

2.4 Flow Log Record Fields

Each flow log entry contains multiple fields.

Important fields include:

FieldDescription
srcaddrSource IP address
dstaddrDestination IP address
srcportSource port
dstportDestination port
protocolProtocol number
packetsNumber of packets
bytesNumber of bytes
actionACCEPT or REJECT
log-statusStatus of log

These logs help identify:

  • Blocked traffic
  • Unexpected connections
  • Unusual traffic patterns

2.5 Traffic That Flow Logs Capture

Flow Logs capture traffic related to:

  • EC2 instances
  • Load balancers
  • NAT gateways
  • Network interfaces

They record:

  • Accepted traffic
  • Rejected traffic

This allows administrators to identify security group or network ACL issues.


2.6 Flow Logs Limitations

For the exam, it is important to know the limitations.

Flow Logs do NOT capture:

  • Packet payload
  • DNS requests directly
  • DHCP traffic
  • Metadata traffic
  • Traffic to/from Amazon DNS
  • Some AWS internal services

They also:

  • Are not real-time packet capture
  • Provide aggregated records

2.7 Use Cases in IT Environments

Security Monitoring

Security teams analyze logs to detect:

  • Unauthorized connections
  • Suspicious port scanning
  • Unusual traffic patterns

Troubleshooting Connectivity

If an application server cannot connect to a database:

Flow logs can show:

  • Traffic rejected by security groups
  • Network ACL blocking

Compliance and Auditing

Organizations must often keep logs for compliance.

Flow Logs allow:

  • Traffic audit trails
  • Security investigations

2.8 Flow Logs and Hybrid Networks

Flow logs can help monitor traffic between:

  • On-premises networks
  • AWS environments

For example:

Traffic flowing through:

  • AWS Site-to-Site VPN
  • AWS Direct Connect

can be analyzed using flow logs.

This helps monitor hybrid connectivity.


3. Amazon VPC Traffic Mirroring

3.1 What Is Traffic Mirroring?

Traffic Mirroring allows copying network traffic from an EC2 instance and sending it to monitoring tools.

It captures full packet data, not just metadata.

This allows:

  • Deep packet inspection
  • Intrusion detection
  • Advanced monitoring

Traffic Mirroring works by duplicating packets from a network interface.


3.2 Components of Traffic Mirroring

Traffic mirroring uses several components.

1. Mirror Source

The source of network traffic.

Usually:

  • Elastic Network Interface (ENI)

Example:

  • EC2 instance network interface

2. Mirror Target

The destination where mirrored traffic is sent.

This could be:

  • Network monitoring appliance
  • Security tool
  • Packet analysis system

Targets may include:

  • EC2 instances running monitoring software
  • Network load balancers

3. Mirror Filter

Defines which traffic should be mirrored.

Filters are based on:

  • Protocol
  • Port
  • Source IP
  • Destination IP

This prevents unnecessary traffic duplication.


4. Mirror Session

Connects the source to the target using a filter.

The session defines:

  • Source
  • Target
  • Filter
  • Priority
  • Packet length

3.3 Traffic Mirroring Architecture

Typical architecture includes:

  1. Application EC2 instances
  2. Traffic mirroring enabled on ENIs
  3. Traffic sent to monitoring tools

Security tools may include:

  • Intrusion detection systems
  • Packet analyzers
  • Threat detection systems

3.4 Traffic Mirroring vs Flow Logs

Understanding this difference is very important for the exam.

FeatureVPC Flow LogsTraffic Mirroring
Data capturedTraffic metadataFull packets
Payload visibilityNoYes
Performance impactLowHigher
StorageCloudWatch/S3Monitoring tools
Use caseMonitoring and troubleshootingDeep security inspection

3.5 Traffic Mirroring Limitations

Traffic mirroring has several limitations.

Instance Support

Only supported on:

  • Nitro-based EC2 instances

Performance Impact

Mirroring duplicates traffic, which may increase:

  • Network bandwidth usage
  • Monitoring system load

Cost Considerations

Costs include:

  • Data transfer
  • Monitoring infrastructure
  • Storage for packet captures

3.6 Security Use Cases

Traffic mirroring is widely used for advanced security monitoring.

Examples include:

Intrusion Detection Systems (IDS)

Security tools analyze mirrored packets to detect:

  • Malware communication
  • Suspicious behavior

Packet Analysis

Network engineers capture packets to investigate:

  • Application errors
  • Protocol issues
  • Unexpected network behavior

Threat Detection

Security platforms analyze mirrored traffic for:

  • Data exfiltration
  • Unauthorized connections

4. Combining Flow Logs and Traffic Mirroring

Many architectures use both services together.

ServiceRole
Flow LogsBroad traffic visibility
Traffic MirroringDeep packet inspection

For example:

  • Flow Logs identify suspicious connections
  • Traffic Mirroring analyzes packet content

This provides layered network visibility.


5. Integration with AWS Monitoring Services

Flow Logs integrate with monitoring services such as:

Amazon CloudWatch

Used for:

  • Monitoring logs
  • Creating alarms
  • Tracking metrics

AWS CloudTrail

Tracks API activity, while flow logs track network traffic.

Together they provide complete visibility.


6. Best Practices for the Exam

Remember these key points.

Enable Flow Logs for Critical Networks

Always enable flow logs for:

  • Production VPCs
  • Sensitive workloads

Use S3 for Long-Term Storage

Store logs in:

  • Amazon S3 for auditing
  • Analytics tools

Use Traffic Mirroring for Security Inspection

Use mirroring when you need:

  • Full packet analysis
  • Intrusion detection

Filter Mirrored Traffic

Always apply filters to:

  • Reduce monitoring load
  • Lower costs

7. Key Exam Tips

Important exam facts:

  • Flow Logs capture metadata only
  • Traffic Mirroring captures full packets
  • Flow logs support VPC, subnet, and ENI levels
  • Traffic mirroring works on ENI level
  • Flow logs are used for troubleshooting and monitoring
  • Traffic mirroring is used for deep security analysis

Short Summary

FeatureFlow LogsTraffic Mirroring
DataTraffic metadataFull packets
UseMonitoring & troubleshootingSecurity inspection
Capture levelVPC, subnet, ENIENI
StorageCloudWatch / S3Monitoring tools
Buy Me a Coffee