Live demo (works without JavaScript):
Sample input — paste anything and watch the result update.Sample input — paste anything and watch the result update.Written by Jane Smith · Reviewed by Sarah Mitchell · July 23, 2026
Strip duplicate lines from your text while preserving the original order.
Remove Duplicate Lines scans a pasted list and keeps the first occurrence of every distinct line, dropping everything that repeats later. The order of the survivors matches the order they first appeared in the input, so a list is never alphabetized behind your back. The original text stays in the source box, and the cleaned result appears in the output box.
Start with this five-line list, which hides two kinds of duplicates:
apple Banana apple banana kiwi
With trim whitespace on and case-insensitive matching on, the page compares the lowercased, trimmed form of every line. The second "apple" repeats the first and drops, and "banana" matches "Banana" and drops as well. Three lines survive, in their original order:
apple Banana kiwi
Switch case-insensitive matching off and rerun. Now "Banana" and "banana" differ by case, so both stay and the output holds four lines. The lowercase "apple" still disappears, because its key is identical to the first line's either way.
Trim whitespace before comparing is checked by default. It strips leading and trailing spaces from each line before the duplicate check, so "note" and "note " are treated as the same line and the version that appears first is what survives. Unchecking it makes the comparison exact, which matters when trailing spaces carry meaning, as in some fixed-width exports. Case-insensitive matching is off by default and lowercases both sides before comparing, which merges "SEO Tools" and "seo tools" into one entry while leaving the casing of the survivor intact.
The comparison is all or nothing: either two keys match exactly or both lines stay. This four-line list makes the boundary visible:
New York new york New York NewYork
With trimming and case-insensitive matching both on, three of the four entries share the key "new york" and collapse into the first, while "NewYork" keeps its own key. Two lines survive. Turn case-insensitive matching off and the list holds three, because "New York" and "new york" now differ by case. The trailing-space variant is still folded in by the trim option, so the same four lines never produce a four-line output. Names with different spellings, such as "Smith" and "Smyth", or addresses with formatting drift, survive every configuration.
Consider a mailing list exported from a signup form. The raw export holds 1,200 rows: 40 are empty lines left by trailing newlines, 80 repeat an email address with different casing, 25 repeat an address with a stray trailing space, and 15 are genuinely separate people whose names happen to match. Running the list with trimming and case-insensitive matching on collapses the 40 blanks into 1, folds the 80 casing variants into their first occurrence, and merges the 25 stray-space rows as well, so 142 rows disappear and 1,058 remain. The 15 name matches survive, because their email addresses differ, and that is usually the desired result: the page deduplicates the identifier you chose, not the person behind it.
A second export a day later adds rows that repeat the first run's survivors; running the combined file collapses those too, because the keys are identical, so the merge stays clean. Addresses that differ by a middle initial or a plus-tag such as "ada+news@example.com" keep both versions, since those keys are genuinely distinct.
The key decision is which version of a duplicated line survives, and the answer is always the first one seen from the top of the input. If a document lists "draft" on line 2 and "DRAFT" on line 900, the output keeps line 2's "draft" and drops the later one, even when the later version looks more complete. To keep the newest spelling instead, reverse the list before running, deduplicate, and reverse again; the page has no built-in last-occurrence mode. This matters when earlier versions of a line are known to be stale.
Blank lines are ordinary lines to this tool. An empty string is a valid entry, so the first blank line is kept and every later blank line is removed as a duplicate. A block of text separated by a single blank line comes back with one blank line instead of several, which is usually the cleanup that was wanted. When trimming is on, lines that differ only by indentation collapse into one, and the first variant, spaces and all, is what the output keeps. Lines with different content but the same trimmed form, such as "cat" and "cat ", also collapse, so trimming should stay off when the whitespace is data.
No. The output preserves first-occurrence order, so the input's sequence is never rearranged. The sort-lines tool can sort and deduplicate in one pass if that is what the task needs.
Two lines are duplicates when their comparison keys match: the raw line by default, the trimmed line with the trim option on, and the lowercased version of either when case-insensitive matching is enabled.
Because blank lines count as entries. One blank line survives as the first occurrence, and the rest are dropped, so a run of several blank lines compresses to one.
There is nothing to undo, since the source box never changes. Rerun with different options or copy from the input to start fresh.
They still drop. The page remembers every key it has seen from the top of the list, so a duplicate at line 1 and line 4,000 collapses to the line 1 version.
The deduplication runs in memory, and the pasted list never leaves the page.
No. Only the comparison key is lowercased; the line that appears first keeps its original spelling, so "Banana" stays "Banana" and never becomes "banana".
No. Each surviving line is copied out exactly as typed, with leading and trailing spaces intact. Trimming only shapes the key used for the comparison.
Windows text carries a carriage return at the end of each line. Trimming strips it before the key is built, so "note" and a "note" with a carriage return collapse into one; with trimming off they are treated as different lines, because the carriage return stays part of the comparison.
Each distinct line occupies one slot in a lookup table, and every incoming line needs a single check against it, so the work grows with the number of unique entries rather than the total length of the input. A mailing list of ten thousand rows where most lines repeat processes quickly, because the table stays small once the common entries are recorded. A list where nearly every line is different stores a key for each one, and the page can feel sluggish with several million genuinely unique rows. Repeated runs are just as cheap as the first, since nothing is cached between clicks.
Use the tool when merging keyword lists, cleaning exported addresses, or preparing a column of IDs for import. Skip it when duplicates are only approximate, such as names with different spellings or addresses with slight formatting drift, because the comparison is exact and those lines will both survive. Skip it too when the document must keep its full blank-line structure, and pair it with sort-lines or remove-extra-spaces when the input also needs ordering or whitespace cleanup.