Tool to deduplicate a list. Doubles or duplicates are a repeated redundant information present twice (or more) in a list, usually useless.
Duplicates in a List - dCode
Tag(s) : Data Processing
dCode is free and its tools are a valuable help in games, maths, geocaching, puzzles and problems to solve every day!
A suggestion ? a feedback ? a bug ? an idea ? Write to dCode!
Duplicates in a List
Duplicates Detector
Answers to Questions (FAQ)
What is a duplicate? (Definition)
A duplicate is a redundant element (double or triple or more) in a set of elements (a list).
How to remove doubles in a list?
Indicate / copy-paste items in the List field. dCode detects if the list is a list of terms (one per line) or a list of words (with separator) and finds redundant items repeated more than once in order to un-duplicate.
Example: A,B,C,D,A,B,C,A,B,A can be deduplicated as A,B,C,D (repeated terms were removed)
Be carefull to take into account all writing variants of a same element: ignore accents and diacritics allow to treat words like item and ìtém as a same word twice. Also, ignoring uppercase and lowercase allow to detect/find item and ITEM as a duplication of the same word and delete them.
How to count doubles in a list?
By listing items, dCode detector counts the number of appearance of each, the list can be sorted to detect the most duplicated ones.
dCode counts duplicates/redundancies, so 2 identical items is counted 1 duplicate. N identical elements is counted N-1.
Why counting doubles?
There are many applications where the presence of duplicates can be a problem:
— detect and delete duplicate files to free up disk space
— deduplicate data from a database, retaining only one occurrence of each unique/distinct record.
— sending mailings by checking if a list of email addresses contains duplicates in order to filter the list and delete identical emails.
Source code
dCode retains ownership of the "Duplicates in a List" source code. Any algorithm for the "Duplicates in a List" algorithm, applet or snippet or script (converter, solver, encryption / decryption, encoding / decoding, ciphering / deciphering, breaker, translator), or any "Duplicates in a List" functions (calculate, convert, solve, decrypt / encrypt, decipher / cipher, decode / encode, translate) written in any informatic language (Python, Java, PHP, C#, Javascript, Matlab, etc.) or any database download or API access for "Duplicates in a List" or any other element are not public (except explicit open source licence). Same with the download for offline use on PC, mobile, tablet, iPhone or Android app.
Reminder: dCode is an educational and teaching resource, accessible online for free and for everyone.
Cite dCode
The content of the page "Duplicates in a List" and its results may be freely copied and reused, including for commercial purposes, provided that dCode.fr is cited as the source (Creative Commons CC-BY free distribution license).
Exporting the results is free and can be done simply by clicking on the export icons ⤓ (.csv or .txt format) or ⧉ (copy and paste).
To cite dCode.fr on another website, use the link:
In a scientific article or book, the recommended bibliographic citation is: Duplicates in a List on dCode.fr [online website], retrieved on 2025-06-15,
