How to use Remove Duplicate Lines
- Paste one item per line into the Lines box.
- Choose what to remove in Duplicates to remove.
- Turn on Ignore letter case or Ignore leading and trailing spaces if the list is inconsistent.
- The cleaned list updates as you change the options. Copy the result.
Example: Remove Duplicate Lines
Clean a five-line list in which two items repeat.
Options
- Duplicates to remove
- Keep the first copy of each line leaves the earliest occurrence, Keep the last copy leaves the latest, and Keep only lines that appear once removes every line that repeats at all.
- Ignore leading and trailing spaces
- Compares each line with its surrounding spaces removed, but writes out the original text of the line it keeps. That matters when the indentation in your list carries meaning.
- Ignore letter case
- Treats Apple and apple as a match while keeping the spelling of the copy that remains.
- Remove empty lines
- Drops empty lines before duplicate matching. Leave it off when blank lines separate groups in your list.
- Sort the result alphabetically
- Sorts with a locale-aware comparison that reads numbers inside text, so item9 comes before item10. Sorting runs after duplicates are removed.
Supported inputs and limits
Where your input is processed
This tool processes your input in this browser. Your text and files are not uploaded to UseFreeTools. Check this tool's limits for anything it may save on your device.
What the comparison ignores, and what it does not
Matching can ignore letter case and spaces at either end of a line. The output keeps the original text of the retained copy. Spaces inside a line, tabs and non-breaking spaces remain significant. If a list still has apparent duplicates, check for invisible whitespace within the lines.
Questions about Remove Duplicate Lines
Does it keep the first or the last copy of a repeated line?
The default keeps the first. Switch Duplicates to remove to Keep the last copy of each line when the newest entry should win, which suits logs and append-only lists.
Is the comparison case-sensitive by default?
Yes. Apple and apple are treated as different lines until you turn on Ignore letter case. Turning it on compares the lines in lowercase but keeps the original wording in the output.
What does 'Keep only lines that appear once' actually leave?
Only lines that occur exactly once in the whole list. A line that appears twice is removed in both places, so this option often shortens a list far more than the other two.
Will the tool change my line endings or add a trailing newline?
Line endings are detected and kept. The result joins the kept lines with the same ending your input used, so a CRLF list stays CRLF.
How are numbers sorted inside lines?
Sorting uses a locale-aware, number-aware comparison, so item2, item9 and item10 follow numeric order rather than plain character order. If you need a strict character sort, sort the output in your spreadsheet or editor instead.