Filter duplicate records

Author: Oscar Cronquist Article last updated on January 11, 2019

In this article I will demonstrate a technique to filter duplicate records. The picture below shows you a data set in columns B to E. Next step is to create an Excel defined table.

Press with left mouse button on a cell in the table, go to tab "Insert" on the ribbon. Press with left mouse button on "Insert Table" button. The following dialog box appears.

Press with left mouse button on OK button.

Type Duplicate in cell F2 and then use the following formula in cell F3:

=COUNTIFS($B$3:B3, B3, $C$3:C3, C3, $D$3:D3, D3, $E$3:E3, E3)>1

How the formula works in cell F13

Step 1 - Understand how relative and absolute cell references work

The formula uses absolute and relative cell references. In cell G3 the formula is:

=COUNTIFS($B$3:B3, B3, $C$3:C3, C3, $D$3:D3, D3, $E$3:E3, E3)>1

In cell G13 the formula has changed to:

=COUNTIFS($B$3:B13, B13, $C$3:C13, C13, $D$3:D13, D13, $E$3:E13, E13)>1

The cell ranges expand as the formula is copied to cells below. This lets you count the current record in above records.

How to use absolute and relative references

What is a reference in Excel? Excel has an A1 reference style meaning columns are named letters A to XFD […]

How to use absolute and relative references

Step 2 - Find duplicates

COUNTIFS($B$3:B13, B13, $C$3:C13, C13, $D$3:D13, D13, $E$3:E13, E13)>1

becomes

COUNTIFS({"Sample0"; "Sample0"; "Sample1"; "Sample0"; "Sample0"; "Sample1"; "Sample1"; "Sample0"; "Sample1"; "Sample1"; "Sample1"}, B13, {"B"; "B"; "A"; "A"; "B"; "B"; "B"; "A"; "A"; "A"; "A"}, C13, {11; 11; 11; 10; 10; 10; 11; 11; 10; 11; 11}, D13, {"AA111";"AA110";"AA111";"AA111";"AA110";"AA111";"AA111";"AA110";"AA110";"AA110";"AA111"}, E13)>1

becomes

=IF(COUNTIFS({"Sample0"; "Sample0"; "Sample1"; "Sample0"; "Sample0"; "Sample1"; "Sample1"; "Sample0"; "Sample1"; "Sample1"; "Sample1"}, "Sample1", {"B"; "B"; "A"; "A"; "B"; "B"; "B"; "A"; "A"; "A"; "A"}, "A", {11; 11; 11; 10; 10; 10; 11; 11; 10; 11; 11}, 11, {"AA111";"AA110";"AA111";"AA111";"AA110";"AA111";"AA111";"AA110";"AA110";"AA110";"AA111"}, "AA111")>1, "Duplicate", "")

becomes

=2>1

and returns TRUE in cell G13

How to use the COUNTIFS function

Checks multiple conditions against the same number of cell ranges and counts how many times all criteria are met.

How to use the COUNTIFS function

Filter records based on formula

Press with mouse on black arrow on cell F2
Enable only value TRUE
Press with left mouse button on OK button

Get the Excel file

Find-duplicate-recordsv2.xlsx

How to use an Excel Table name in Data Validation Lists and Conditional Formatting formulas

This article demonstrates different ways to reference an Excel defined Table in a drop-down list and Conditional Formatting. There are […]

Extract unique distinct values from a filtered Excel defined Table [UDF and Formula]

This article demonstrates two formulas that extract distinct values from a filtered Excel Table, one formula for Excel 365 subscribers […]

Count unique distinct values in a filtered Excel defined Table

This article demonstrates a formula that counts unique distinct values filtered from an Excel defined Table. Debra Dalgleish described in […]

Populate drop down list with filtered Excel Table values

This article demonstrates how to populate a drop down list with filtered values from an Excel defined Table. The animated […]

Remove common records between two data sets

This article demonstrates how to filter records occurring in only one out of two Excel defined tables. It also shows […]

Hide specific columns programmatically

This article describes a macro that hides specific columns automatically based on values in two given cells. I am also […]

How to copy Excel tables programmatically

The image above demonstrates a macro linked to a button. Press with left mouse button on the button and the […]

Highlight duplicates in a filtered Excel Table

The image above demonstrates a conditional formatting formula applied to an Excel Table containing random data. The Excel Table has […]

Copy Excel Table filter criteria programmatically

I will in this article demonstrate a macro that copies criteria from one Excel Table and applies them to another […]

Comments (3)

3 Responses to “Filter duplicate records”

Shaz says:

March 3, 2011 at 6:37 pm

How do I get the sheet names in a file to becomes the row description in a summary file.
Thxs

Reply
Oscar says:

March 5, 2011 at 11:08 pm

Shaz,

Here is an user defined function:

VBA code:
```
Function List_sheets()

Dim i As Integer
Dim temp() As String
ReDim temp(1 To Application.Sheets.Count)

For i = 1 To Application.Sheets.Count
  temp(i) = Worksheets(i).name
Next i

List_sheets = temp

End Function
```
Where to put the code

Press Alt-F11 to open visual basic editor
Press with left mouse button on Module on the Insert menu
Copy and paste the code above
Exit visual basic editor

How to use user defined function

Example,

Select cell A1
Type =List_sheets() in formula bar and then press CTRL + SHIFT + ENTER
Copy cell A1 and paste it to the right

Reply
Sangam says:

July 27, 2012 at 11:19 pm

Its working...! really good solution
Thanks,
Sangam R

Reply

How to comment

How to add a formula to your comment
<code>Insert your formula here.</code>

Convert less than and larger than signs
Use html character entities instead of less than and larger than signs.
< becomes < and > becomes >

How to add VBA code to your comment
[vb 1="vbnet" language=","]
Put your VBA code here.
[/vb]

How to add a picture to your comment:
Upload picture to postimage.org or imgur
Paste image link to your comment.

Contact Oscar

You can contact me through this contact form