Remove Duplicate Lines
Remove Duplicate Lines — runs in your browser. Inputs are not stored.
Remove duplicate lines
Enter values, then press Calculate.
Input stays in the browser and is not sent to the server.
How to use
- Paste a list into Text, one item per line. Line-oriented data such as emails, URLs, or SKUs works well.
- Turn on Trim leading and trailing spaces before compare if you want lines that differ only by padding to count as the same.
- Press Remove duplicates.
- Check the Original lines, Result lines, and Removed counts, plus the result that keeps the first occurrence.
- Use Copy result to take it with you, or Reset to clear.
Key concepts
Remove duplicate lines scans a mailing list, log URLs, or product codes from top to bottom and keeps only the first time each line appears. Order is preserved; later copies are dropped. To sort as well, send the result to the list sorter. Comparison is case-sensitive, so Apple and apple are different lines.
The trim option strips leading and trailing spaces only for comparison. Spaces in the middle of a line stay. Stats show how many lines were dropped. Processing stays in the browser and is not sent to a server. Empty lines count as a value too—clean them first if you do not want them.
Example
If the input is five lines A, B, A, B, C, the result is three lines A, B, C and Removed is 2. If the second line is B (trailing space) and trim is on, it matches the first B. With trim off, the space makes them different and both stay. Duplicates that differ only by letter case, such as email addresses, are not caught here—normalize to lowercase first if you need that. In a log, the same URL on lines that also differ by a timestamp is not a duplicate because the whole line differs. If Removed is 0, the input is already unique, or the line breaks may not match what you expect—check the original. When you retry the same input, reset first so it does not mix with the previous output. On a shared computer, clear the fields when you finish. Closing the tab discards what you pasted; you do not need to send a delete request. Button names match the tool above. If the result is empty, check required fields and that an error is not hidden.
Related: List sorter · Text replace · Text compare
Frequently asked questions
Does remove-duplicate-lines ignore case?
No. Lines are duplicates only when the strings match exactly. Trim only aligns spaces.
Does it keep the first occurrence or the last?
It keeps the first occurrence from top to bottom. There is no keep-last mode.
Are empty lines treated as duplicates?
An empty line is one value. Several empty lines collapse to one.
Does the trim option also cut spaces in the result?
It aligns spaces for comparison only. Kept lines stay in the form of the first occurrence.
Is the list uploaded to a server?
No. Processing runs only in the browser.
Does it sort and remove duplicates in one step?
This screen only removes duplicates. Use the list sorter for alphabetical order.
Last reviewed: 2026-09-04