awk — Pattern scanning and processing language for structured text across all 5 shells
Equivalents in every shell
awk '{print $1}' fileawk '{print $1}' fileawk '{print $1}' fileGet-Content file | ForEach-Object { ($_ -split '\s+')[0] }No native awk; pipelines plus `-split` cover most one-liners.
for /f "tokens=1" %a in (file) do @echo %aUse `%%a` instead of `%a` inside a batch file.
Worked examples
Print the first whitespace-delimited column
awk '{print $1}' file.txtGet-Content file.txt | ForEach-Object { ($_ -split '\s+')[0] }for /f "tokens=1" %a in (file.txt) do @echo %aSum a numeric column
awk '{s+=$2} END {print s}' file.txt(Get-Content file.txt | ForEach-Object { [double](($_ -split '\s+')[1]) } | Measure-Object -Sum).SumPrint lines where a field matches
awk '$3 == "ERROR" {print}' app.logGet-Content app.log | Where-Object { ($_ -split '\s+')[2] -eq 'ERROR' }Gotchas
- awk fields are 1-indexed (`$1` is the first); PowerShell array indices are 0-based.
- awk auto-splits on runs of whitespace and ignores leading/trailing whitespace; PowerShell `-split '\s+'` produces an empty leading element when the line starts with whitespace — `.Trim()` first or filter empties.
- Multi-block awk scripts (BEGIN/END, associative arrays) rarely translate to a clean one-liner — write a small PowerShell script for non-trivial cases.
WSL & PowerShell Core notes
Common tasks using awk
- Convert line endings between CRLF and LF
Strip or insert carriage returns to switch a file between Windows (CRLF, \r\n) and Unix (LF, \n) line endings.
- Dedupe lines while preserving order
Remove duplicate lines from input but KEEP the first occurrence in its original position — for unique-but-sorted-by-recency lists, `$PATH` cleanup, and history dedup.
- Extract a substring by regex
Pull a substring out of a string or a stream of input using a regex — for parsing log lines, extracting an ID from a URL, scraping a version number, or tokenizing a config.
- Extract email addresses from text
Pull every email address out of a log file or block of text.
- Find and replace text in files
Substitute one string for another inside a file (or every file in a tree), in place.
- Format text into aligned columns
Take tab- or whitespace-separated text and produce aligned columns — for human-readable tables, `mount` / `df` output reformatting, and CSV pretty-printing.
- Normalize whitespace in text
Collapse multiple spaces / tabs into single spaces and strip line-leading / trailing whitespace — for diff-friendly text, CSV cleanup, and config-file canonicalization.
- Remove duplicate lines from a file
Strip duplicate lines from a file, optionally preserving original order.
- Sort a file by a specific column
Sort lines of a file by the value in a chosen column (numeric or alphabetic).
- Trim leading and trailing whitespace
Remove only the whitespace at the start and end of each line (preserving internal spaces) — for cleaning user input, config-file values, and form fields.