3.2 Searching and Extracting Data from Files (Weight: 3)

📘Linux Essentials (LPI 010-160)

---

In Linux, commands often produce output that appears on the terminal screen. However, in real IT environments, administrators rarely use command output alone. Instead, they redirect output to files, read input from files, or send the output of one command directly to another command.

This is done using Input/Output (I/O) redirection and pipes. These features allow users to process large amounts of data efficiently and combine multiple commands together.

Understanding pipes and redirection is very important for the Linux Essentials (LPI 010-160) exam because they are commonly used when searching, filtering, and extracting information from files.

---

1. Standard Streams in Linux

Every Linux command works with three default data streams.

Stream	Number	Description
Standard Input	0	Input provided to a command
Standard Output	1	Normal output produced by a command
Standard Error	2	Error messages produced by a command

1. Standard Input (stdin)

Standard input is the data a command receives.

Example:

cat

If you type text after running this command, the command reads the input from the keyboard.

Example interaction:

cat
Hello
Hello

The keyboard is the default stdin source.

---

2. Standard Output (stdout)

Standard output is the normal result produced by a command.

Example:

ls

Output:

file1.txt
file2.txt
file3.txt

The terminal screen is the default destination for stdout.

---

3. Standard Error (stderr)

Standard error displays error messages.

Example:

ls missingfile

Output:

ls: cannot access 'missingfile': No such file or directory

This message is sent through stderr, not stdout.

Separating stdout and stderr allows administrators to store results while keeping errors separate.

---

2. What is I/O Redirection?

I/O redirection changes where input comes from or where output goes.

Instead of displaying output on the screen, you can:

• Save output to a file
• Read input from a file
• Separate error messages
• Append output to existing files

Redirection operators control this behavior.

---

3. Output Redirection (>)

The > operator sends standard output to a file.

Example:

ls > files.txt

What happens:

1. ls lists files
2. Instead of displaying on screen
3. Output is saved to files.txt

Contents of files.txt:

file1.txt
file2.txt
file3.txt

Important behavior:

If the file already exists, it will be overwritten.

---

4. Appending Output (>>)

The >> operator adds output to the end of a file instead of replacing it.

Example:

date >> log.txt

If the file already contains data:

System started

After running the command:

System started
Thu Mar 7 10:30:00 UTC 2026

This is commonly used in system logging and monitoring tasks.

---

5. Redirecting Standard Input (<)

The < operator sends file contents as input to a command.

Example:

wc < users.txt

wc counts words, lines, and characters.

Instead of typing text manually, the command reads from users.txt.

Example file:

admin
developer
operator

The command processes the file as input.

---

6. Redirecting Standard Error (2>)

Error messages can be redirected separately.

Operator:

2>

Example:

ls missingfile 2> error.log

Result:

• Error message is saved in error.log
• Nothing appears on screen

Contents of error.log:

ls: cannot access 'missingfile': No such file or directory

This is useful when administrators run scripts and want to review errors later.

---

7. Redirecting Both Output and Errors

Sometimes both output and errors must be stored together.

Example:

command > output.txt 2>&1

Explanation:

• > redirects standard output
• 2>&1 redirects stderr to the same location as stdout

Both outputs go into output.txt.

---

8. Discarding Output with /dev/null

Sometimes output is not needed.

Linux provides a special file:

/dev/null

Anything redirected to /dev/null is discarded.

Example:

command > /dev/null

Suppress errors:

command 2> /dev/null

Suppress everything:

command > /dev/null 2>&1

This is useful when running automated scripts where unnecessary output should be hidden.

---

9. What is a Pipe ( | )?

A pipe connects two commands.

The output of the first command becomes the input of the second command.

Symbol:

|

Structure:

command1 | command2

Instead of saving output to a file, the data flows directly between commands.

---

10. Basic Pipe Example

Example:

ls | less

Process:

1. ls lists files
2. Output is passed to less
3. less allows scrolling through the results

This is helpful when there are many files.

---

11. Pipes with Search Commands

Pipes are commonly used with data filtering tools.

Example:

ls /etc | grep conf

Steps:

1. ls /etc lists files
2. grep conf searches for the word conf

Output only shows matching files.

---

12. Pipes with Sorting

Example:

cat users.txt | sort

Process:

1. cat prints file content
2. sort arranges lines alphabetically

Result:

admin
developer
operator

---

13. Pipes with Counting

Example:

ls | wc -l

Steps:

1. ls lists files
2. wc -l counts lines

Output:

25

This shows how many files are in the directory.

This is frequently used by administrators to analyze system data quickly.

---

14. Using Multiple Pipes

Commands can be chained together.

Example:

cat access.log | grep ERROR | wc -l

Process:

1. cat access.log reads a log file
2. grep ERROR finds error entries
3. wc -l counts them

Output shows the number of errors in the log file.

This method is very common in log analysis and system monitoring.

---

15. Pipe vs Redirection

Feature	Pipe	Redirection
Symbol	`	`
Purpose	Send output to another command	Send output to a file
Data flow	Command → Command	Command → File
Speed	Faster (no file creation)	Creates or modifies files

---

16. Practical IT Usage Examples

Log File Analysis

Find login failures:

cat auth.log | grep "failed"

---

Count Running Processes

ps aux | wc -l

---

Filter Configuration Files

cat /etc/passwd | grep bash

Shows users with Bash shell access.

---

Search Installed Packages

rpm -qa | grep httpd

Used on RPM-based Linux systems.

---

17. Best Practices for Pipes and Redirection

1. Use pipes to combine commands

This avoids creating unnecessary temporary files.

2. Redirect errors when running scripts

Example:

script.sh 2> error.log

3. Use append (>>) for logs

This prevents overwriting important records.

4. Use /dev/null to silence unwanted output

Useful for background automation tasks.

---

18. Key Exam Points to Remember

For the Linux Essentials exam, you should know:

• The three standard streams (stdin, stdout, stderr)
• Output redirection using >
• Appending output using >>
• Input redirection using <
• Redirecting errors using 2>
• Redirecting both output and errors
• The purpose of /dev/null
• Pipe operator |
• How pipes combine multiple commands
• Using pipes with commands like:
  • grep
  • sort
  • wc
  • less
  • cat
  • ls

---

✅ Summary

Pipes and I/O redirection are essential Linux command-line features that allow users to control how data flows between commands and files.

• Redirection sends input or output to files.
• Pipes send output directly to another command.
• These tools allow administrators to build powerful command chains for searching, filtering, analyzing, and managing system data.

They are heavily used in system administration, log analysis, automation scripts, and monitoring tasks, making them a core skill required for the Linux Essentials (LPI 010-160) certification.