Remove Duplicate Lines

Remove repeated lines from a list while keeping the order you pasted.

On-device No sign-up Free to use

Clean controls

Showing an example. Edit to see your own.

?

Paste one item per line.

?

Treats Apple and apple as the same line.

?

Compares trimmed lines but keeps the original text in the output.

Processed in your browser. Your inputs stay on this device.

Result

Showing an example.

Clean a five-line list in which two items repeat.

apple, banana and cherry on three lines. The result reports 3 lines kept from 5, 2 lines removed and 2 repeated values found.

Loading the tool. Your result will appear here.

https://usefreetools.net/remove-duplicate-lines · 20 November 2026

Was this helpful?

How to use Remove Duplicate Lines

  1. Paste one item per line into the Lines box.
  2. Choose what to remove in Duplicates to remove.
  3. Turn on Ignore letter case or Ignore leading and trailing spaces if the list is inconsistent.
  4. The cleaned list updates as you change the options. Copy the result.

Example: Remove Duplicate Lines

Clean a five-line list in which two items repeat.

You add
apple, banana, apple, cherry and banana on five separate lines, with Duplicates to remove left on Keep the first copy of each line.
You get
apple, banana and cherry on three lines. The result reports 3 lines kept from 5, 2 lines removed and 2 repeated values found.

Options

Duplicates to remove
Keep the first copy of each line leaves the earliest occurrence, Keep the last copy leaves the latest, and Keep only lines that appear once removes every line that repeats at all.
Ignore leading and trailing spaces
Compares each line with its surrounding spaces removed, but writes out the original text of the line it keeps. That matters when the indentation in your list carries meaning.
Ignore letter case
Treats Apple and apple as a match while keeping the spelling of the copy that remains.
Remove empty lines
Drops empty lines before duplicate matching. Leave it off when blank lines separate groups in your list.
Sort the result alphabetically
Sorts with a locale-aware comparison that reads numbers inside text, so item9 comes before item10. Sorting runs after duplicates are removed.

Supported inputs and limits

  • Matching follows the options you choose, so a list that looks clean can still hold near-duplicates such as Apple and apple unless you turn on Ignore letter case.
  • Lists are capped at 5,000,000 characters and 200,000 lines.
  • Line endings are kept: Windows CRLF input stays CRLF and Unix LF input stays LF.
  • Tabs and non-breaking spaces inside a line are not trimmed by Ignore leading and trailing spaces.
  • Keep a copy of the original if capitalization or indentation matters to you.
More details

The result reports occurrence counts for up to one hundred repeated values; original line-count measures remain available.

Where your input is processed

This tool does not upload your inputs. See the limits for anything saved on your device.

What the comparison ignores, and what it does not

Matching can ignore letter case and spaces at either end of a line. The output keeps the original text of the retained copy. Spaces inside a line, tabs and non-breaking spaces remain significant. If a list still has apparent duplicates, check for invisible whitespace within the lines.

Questions about Remove Duplicate Lines

Does it keep the first or the last copy of a repeated line?

The default keeps the first. Switch Duplicates to remove to Keep the last copy of each line when the newest entry should win, which suits logs and append-only lists.

Is the comparison case-sensitive by default?

Yes. Apple and apple are treated as different lines until you turn on Ignore letter case. Turning it on compares the lines in lowercase but keeps the original wording in the output.

What does 'Keep only lines that appear once' actually leave?

Only lines that occur exactly once in the whole list. A line that appears twice is removed in both places, so this option often shortens a list far more than the other two.

Will the tool change my line endings or add a trailing newline?

Line endings are detected and kept. The result joins the kept lines with the same ending your input used, so a CRLF list stays CRLF.

How are numbers sorted inside lines?

Sorting uses a locale-aware, number-aware comparison, so item2, item9 and item10 follow numeric order rather than plain character order. If you need a strict character sort, sort the output in your spreadsheet or editor instead.

Page updated · Project managed by Tony Hines · Report a problem with Remove Duplicate Lines

Related reading

All guides