Five Pipelines Every Textsmith Should Know

Ishe Chinyoka

- 3 minutes read

Table of Contents

One of the most beautiful ideas in Unix is that programs don’t have to do everything.

Instead, each program performs one small task, handing its output to the next through a pipe (|). The result is often clearer, easier to understand, and surprisingly powerful.

Here are five practical pipelines I find myself returning to again and again.

1. Find the Most Common Error Messages

Suppose you have a large log file and want to know which errors occur most frequently.

grep ERROR server.log | sort | uniq -c | sort -nr

Let’s read it from left to right:

  • grep finds the error lines.
  • sort groups identical messages together.
  • uniq -c counts each group.
  • sort -nr shows the highest counts first.

No special reporting software required.

2. Count the Number of Markdown Files

Need a quick inventory of your documentation?

find . -name "*.md" | wc -l

This searches the current directory and all subdirectories for Markdown files before counting the results.

Sometimes a single number tells you everything you need to know.

3. Remove Blank Lines from a File

Configuration files, generated reports, or copied text often contain unnecessary blank lines.

grep -v '^$' notes.txt > cleaned.txt

Here we meet our first redirection.

Instead of displaying the cleaned text on the screen, > writes it to a new file called cleaned.txt, leaving the original untouched.

Sometimes the simplest filter makes a document much easier to read.

4. Find Your Ten Largest Files

Storage mysteriously fills up. This pipeline quickly identifies the biggest files in the current directory tree.

find . -type f -exec du -h {} + | sort -hr | head

This command:

  • finds every file,
  • measures its size,
  • sorts from largest to smallest,
  • displays only the first ten results.

It’s often the quickest way to answer the question, “What’s eating my disk space?”

5. Extract a Column from a CSV File

Suppose you have a CSV file listing employees.

Name,Department,Location
Alice,Sales,Harare
Brian,IT,Bulawayo
Carol,Finance,Mutare

To display only the department names:

cut -d',' -f2 staff.csv

Output:

Department
Sales
IT
Finance

Small tools make short work of structured text.

The Secret Isn’t the Commands

You don’t need to memorize hundreds of Unix utilities.

Instead, learn a handful of dependable tools:

  • grep to search.
  • find to locate files.
  • sort to organize.
  • uniq to count.
  • cut to extract fields.
  • head and tail to preview data.
  • wc to count.
  • awk when simple filters grow into computations.

The real magic happens when you combine them.

A pipe lets one command become the input to another.

Redirection lets you save the result instead of merely displaying it.

Together they form a language for solving problems one small step at a time.

That is why, fifty years later, the Unix philosophy still feels remarkably modern.

Small tools.

Simple ideas.

Endless possibilities.