uniq — Filter adjacent duplicate lines across all 5 shells
Equivalents in every shell
Bashunix
sort file | uniqZshunix
sort file | uniqFishunix
sort file | uniqPowerShellwindows
Get-Content file | Sort-Object -Unique`Sort-Object -Unique` deduplicates and sorts in one step.
Worked examples
Remove duplicate lines
Bash
sort file.txt | uniqPowerShell
Get-Content file.txt | Sort-Object -UniqueCount occurrences and sort most-frequent first
Bash
sort file.txt | uniq -c | sort -rnPowerShell
Get-Content file.txt | Group-Object | Sort-Object Count -Descending | ForEach-Object { "$($_.Count) $($_.Name)" }Show only lines that appear more than once
Bash
sort file.txt | uniq -dPowerShell
Get-Content file.txt | Group-Object | Where-Object Count -gt 1 | ForEach-Object NameGotchas
- `uniq` only collapses ADJACENT duplicates — always pipe through `sort` first unless you specifically want consecutive-run behavior.
- `uniq -c` counts adjacent runs; if input is not sorted, the counts will be wrong.
- cmd has no native uniq — pipe to PowerShell `Sort-Object -Unique` or install GNU coreutils.
WSL & PowerShell Core notes
pwshpwsh has no `uniq` cmdlet. `Sort-Object -Unique` is the one-cmdlet form that combines sort + dedup; for the `uniq -c` (count adjacent runs) idiom, use `Group-Object | ForEach-Object { "$($_.Count) $($_.Name)" }`, and for `uniq -d` (duplicates only) use `Group-Object | Where-Object Count -gt 1 | ForEach-Object Name`. All three are fully in-memory — same multi-GB heap caveat as `Sort-Object` applies for very large inputs.
WSLGNU `uniq` requires adjacent duplicates, so the canonical bash idiom is `sort file | uniq` (with `LC_ALL=C` if you want byte-exact dedup that ignores locale-equivalent characters like `café` vs the NFD form `cafe\u0301`). On `/mnt/c/...` paths the cost is dominated by the sort's read pass over DrvFs; `uniq` itself streams and is cheap. WSL2 BusyBox `uniq` (Alpine images) is feature-compatible with GNU `uniq` for everyday `-c`/`-d`/`-u`/`-i` usage, but lacks `--group` mode if you rely on grouped output.
Common tasks using uniq
- Dedupe lines while preserving order
Remove duplicate lines from input but KEEP the first occurrence in its original position — for unique-but-sorted-by-recency lists, `$PATH` cleanup, and history dedup.
- Remove duplicate lines from a file
Strip duplicate lines from a file, optionally preserving original order.