Does removing duplicates change the order of my lines?

No. Order is always preserved. With keep-first, every line stays where its first appearance was and later repeats are dropped. With keep-last, the final appearance keeps its position and earlier repeats are dropped.

What counts as a duplicate?

Only lines that are exactly identical after applying your optional trim and case rules. By default that means a character-for-character match. Turn on case-insensitive matching to treat Apple and apple as the same line.

Why would I keep the last occurrence instead of the first?

Keep-last is useful for logs or change lists where the most recent entry for a key should win. For example, a list of setting=value lines where a later line overrides an earlier one for the same setting.

What does the trim option do?

It strips leading and trailing whitespace from each line before comparing and before output, so two lines that differ only by a trailing space or tab are correctly recognised as duplicates.

Are blank lines treated as duplicates?

Yes. After the first blank line, additional blank lines are removed like any other repeat. If you trim whitespace, indented blank lines also collapse to a single empty line.

Duplicate Line Remover

Long lists copied from spreadsheets, logs, or exports often contain the same entry many times. This tool removes exact duplicate lines while keeping your original ordering intact, and tells you exactly how many it stripped.

How it works

The input is split on newlines. For keep-first, a set tracks which line keys have already been seen, and any line whose key is already in the set is skipped:

for each line:
  key = caseInsensitive ? line.toLowerCase() : line
  if key not in seen: keep line, add key to seen

For keep-last, the tool records the index of the final occurrence of each key, then emits only the lines sitting at those indices. Either way the surviving lines stay in their natural position, so nothing is re-sorted.

Tips and examples

Use case-insensitive matching when cleaning email lists, where [email protected] and [email protected] are the same address. Enable trimming for data copied from terminals or PDFs, where invisible trailing spaces would otherwise hide duplicates. To both sort and de-duplicate, run this tool first and then the Multi-line Text Sorter, or vice versa — the removed-count readout lets you confirm the cleanup did what you expected.