Log ingestion

1.1 System and Network Architecture Concepts

CompTIA CySA+ (CS0-003)


Security analysts depend heavily on logs to detect threats, investigate incidents, and monitor activity across systems and networks. For logs to be useful, they must be collected correctly, time-aligned, and stored with the right level of detail.
This section explains log ingestion, time synchronization, and logging levels—all required knowledge for the CySA+ exam.


1. Log Ingestion

What is Log Ingestion?

Log ingestion is the process of collecting, receiving, and transporting logs from different systems and delivering them to a central platform, such as a:

  • SIEM (Security Information and Event Management)
  • Log management system
  • Threat detection platform
  • Centralized syslog server

The goal is to have all important logs in one place so that analysts can correlate events, detect attacks, and respond quickly.


Why Log Ingestion Matters in Cybersecurity

Security data comes from many sources, such as:

  • Firewalls
  • Endpoint security agents
  • Servers
  • Cloud platforms
  • Authentication systems
  • Network devices
  • Applications

Attackers often try to hide their activity by spreading actions across multiple devices.
Log ingestion allows analysts to see the full picture by combining logs from everywhere.


Common Methods of Log Ingestion

1. Agent-Based Log Ingestion

A small software agent is installed on each endpoint or server.
The agent collects logs, formats them, and sends them to the SIEM.

Advantages:

  • Works even if network traffic is encrypted
  • Can collect detailed OS-level and application logs
  • Supports real-time forwarding

Use case in IT:
An organization installs Splunk or Elastic agents on Linux and Windows servers to send security event logs to their SIEM.


2. Agentless Log Ingestion

No software agent is installed. Instead, logs are collected using built-in protocols like:

  • Syslog (UDP/TCP/514)
  • Windows Event Forwarding (WEF)
  • SNMP traps
  • API-based log pulling (common in cloud environments)

Advantages:

  • Easier to maintain
  • Less overhead on devices

Example in IT:
Network switches send syslog messages directly to a SIEM without installing any agent.


3. Batch Log Ingestion

Logs are uploaded at scheduled intervals. This is common with:

  • Cloud platforms exporting logs hourly
  • Applications generating log files, which SIEM ingests periodically

Disadvantages:
Not real-time. Detection can be delayed.


Log Ingestion Challenges You Must Know for the Exam

CySA+ expects you to understand issues that occur when logs are improperly collected:

a. Log overload

Too many logs create “noise” and slow down analysis.

b. Missing log sources

Critical logs not being collected results in blind spots.

c. Improper parsing

If logs aren’t normalized or parsed correctly, detection rules may fail.

d. Duplicate logs

Can inflate storage and cause false correlations.

e. Network delays or dropped logs

Especially with UDP-based syslog, which is not guaranteed delivery.


2. Time Synchronization

Time synchronization ensures every device uses the same clock.
This is one of the most important foundational requirements in cybersecurity logging.


Why Time Synchronization Matters

Security logs are useful only if the timestamps are correct.
If different systems show different times, analysts cannot create an accurate timeline of events.

For example, an attack may appear to happen out of order, making investigation extremely difficult.


NTP – Network Time Protocol

The most common protocol for time synchronization is:

  • NTP (Network Time Protocol) – standard time-sync protocol for servers, network devices, SIEM, etc.

NTP ensures accurate time using a hierarchical model called stratum levels:

  • Stratum 0 – Reference clock
  • Stratum 1 – Systems connected directly to reference clocks
  • Stratum 2, 3, etc. – Lower levels receiving time from the level above

Importance in IT Security

1. Accurate Log Correlation

SIEM uses timestamps to:

  • Join related events
  • Detect anomalies
  • Identify attack chains

Without synchronized time, correlation rules fail.

2. Incident Response

Investigators reconstruct the timeline of events.
Incorrect time = wrong conclusions.

3. Compliance Requirements

Many frameworks (PCI-DSS, HIPAA, SOC 2) require time synchronization.


Common Time Synchronization Problems

CySA+ expects you to recognize these:

a. Time drift

When a device slowly falls out of sync if NTP is not configured properly.

b. Wrong time zone

Device logs may use the wrong local time, causing mismatches.

c. Unsynchronized cloud services

Cloud logs may use UTC while on-prem devices use local time.

d. Tampered timestamps

Attackers may alter system time to hide activity.


3. Logging Levels

Logging levels define how much detail a system includes in logs.
Different systems use different names, but the concept is the same.

Higher levels = more detail.
Lower levels = fewer details.


Common Logging Levels (Generalized)

1. Emergency / Critical / Alert

Indicates system failure or extremely severe issues.

Example in IT:
A critical alert from a firewall showing its service crashed.


2. Error

Indicates a problem but not a full outage.

Example:
Failed login attempts due to authentication issues.


3. Warning

Signals something unusual that could become a problem.

Example:
A server is running low on disk space.


4. Info (Informational)

Normal system operations.

Examples:

  • User login
  • System boot
  • Network connection established

5. Debug

Provides additional information useful when troubleshooting.


6. Trace / Verbose

Very detailed, often showing step-by-step internal processes.


Selecting the Right Logging Level

The CySA+ exam expects you to understand trade-offs:

High-Level Logging (Error, Warning, Critical)

  • Uses less storage
  • Easier to analyze
  • But may miss important details

Verbose Logging (Debug, Trace)

  • Detailed, useful for deep investigations
  • Uses more storage
  • High performance impact

Logging Levels and SIEM Usage

A SIEM depends on consistent and correct logging levels.
Important considerations:

  • Too much debug logging causes unnecessary noise and costs.
  • Too little logging prevents detection of attacks.
  • Critical security logs should always be collected (authentication logs, firewall logs, etc.).

Putting It All Together (Exam Perspective)

A successful security architecture requires:

1. Proper Log Ingestion

  • Agent-based, agentless, or batch
  • Logs normalized and parsed
  • Correct sources included
  • Avoiding duplicates and missing logs

2. Accurate Time Synchronization

  • Use NTP
  • Ensure consistent time zones
  • Required for correlation and investigations

3. Appropriate Logging Levels

  • Balance between detail and resource usage
  • Ensure essential visibility for detection
  • Maintain compliance

Together, these components make logs useful, consistent, and actionable—exactly what the CySA+ exam wants you to understand.


Buy Me a Coffee