uniq

all

Output the unique lines from an input or file. Since it does not detect repeated lines unless they are adjacent, we need to sort them first. See also: `sort`.

More info →

Options (6)

-u, --uniqueboolean

Display only unique lines

Example: sort {{path/to/file}} | uniq {{[-u|--unique]}}

-d, --repeatedboolean

Display only duplicate lines

Example: sort {{path/to/file}} | uniq {{[-d|--repeated]}}

-c, --countboolean

Display number of occurrences of each line along with that line

Example: sort {{path/to/file}} | uniq {{[-c|--count]}}

boolean

Display number of occurrences of each line, sorted by the most frequent

Example: sort {{path/to/file}} | uniq {{[-c|--count]}} | sort {{[-nr|--numeric-sort --reverse]}}

-w, --check-charsboolean

Compare only the first 10 characters on each line for uniqueness

Example: sort {{path/to/file}} | uniq {{[-w|--check-chars]}} 10

-s, --skip-charsboolean

Compare text after the first 5 characters on each line for uniqueness

Example: sort {{path/to/file}} | uniq {{[-s|--skip-chars]}} 5

Examples (7)

Display each line once

sort path/to/file | uniq

Display only unique lines

sort path/to/file | uniq [-u|--unique]

Display only duplicate lines

sort path/to/file | uniq [-d|--repeated]

Display number of occurrences of each line along with that line

sort path/to/file | uniq [-c|--count]

Display number of occurrences of each line, sorted by the most frequent

sort path/to/file | uniq [-c|--count] | sort [-nr|--numeric-sort --reverse]

Compare only the first 10 characters on each line for uniqueness

sort path/to/file | uniq [-w|--check-chars] 10

Compare text after the first 5 characters on each line for uniqueness

sort path/to/file | uniq [-s|--skip-chars] 5

made by @shridhargupta | data from tldr-pages