Remove Duplicate Lines
Remove duplicate lines from any text or list instantly online. Free duplicate line remover - paste, deduplicate, copy.
Three steps to get started
Paste your list
Paste any multi-line text or list. Each line is treated as a separate item.
Duplicates removed
First occurrence of each line kept, all duplicates removed. Order preserved.
Copy or download
Click Copy for the clean list, or download as a .txt file.
Clean up lists and text files instantly
Deduplication is the process of reducing a list so that each distinct value appears exactly once. This tool does it by scanning your input top to bottom, keeping the first occurrence of every unique line and discarding every later repeat - which means the original ordering survives intact. That is a meaningful difference from the classic Unix pipeline sort | uniq, which requires sorting first and therefore destroys the order you pasted in.
Common use cases:
- Collapsing overlapping keyword exports from Ahrefs, Semrush, and Search Console into one clean list
- Cleaning email lists before a Mailchimp or HubSpot import, where duplicates inflate contact counts and billing
- Removing repeated rows from a CSV column pasted out of Excel or Google Sheets
- Trimming repeated stack traces out of an application log before filing a bug
- Building wordlists and deduplicating IP or domain blocklists merged from several feeds
Why some "duplicates" survive
The comparison is an exact string match on the whole line, so lines that look identical on screen can still differ underneath. The three usual culprits are trailing whitespace (apple vs apple ), inconsistent line endings when files from Windows (CRLF) and macOS or Linux (LF) are merged, and invisible characters such as a UTF-8 byte-order mark on the very first line or a non-breaking space pasted from a web page. If a duplicate refuses to disappear, running the text through a whitespace-trimming step first almost always explains it.
Case handling is the other decision to make. Case-sensitive mode treatsApple, apple, and APPLE as three separate values, which is correct for anything case-significant like passwords, base64 strings, or Linux filenames. Case-insensitive mode folds them into one and keeps whichever spelling appeared first - the right choice for email addresses, where the domain is defined as case-insensitive, and for most keyword and tag lists.
Deduplication uses a hash set, so processing stays fast even on lists with hundreds of thousands of lines. It all runs locally in the page, which is what makes it safe to paste customer email exports or internal log excerpts here.