search
Search
Join our weekly DS/ML newsletter layers DS/ML Guides
menu
menu search toc more_vert
Robocat
Guest 0reps
Thanks for the thanks!
close
Comments
Log in or sign up
Cancel
Post
account_circle
Profile
exit_to_app
Sign out
help Ask a question
Share on Twitter
search
keyboard_voice
close
Searching Tips
Search for a recipe:
"Creating a table in MySQL"
Search for an API documentation: "@append"
Search for code: "!dataframe"
Apply a tag filter: "#python"
Useful Shortcuts
/ to open search panel
Esc to close search panel
to navigate between search results
d to clear all current filters
Enter to expand content preview
icon_star
Doc Search
icon_star
Code Search Beta
SORRY NOTHING FOUND!
mic
Start speaking...
Voice search is only supported in Safari and Chrome.
Navigate to
A
A
brightness_medium
share
arrow_backShare
Twitter
Facebook

Pandas DataFrame | duplicated method

Pandas
chevron_right
Documentation
chevron_right
DataFrame
chevron_right
Data Selection and Renaming
schedule Jul 1, 2022
Last updated
local_offer PythonPandas
Tags

Pandas DataFrame.duplicated(~) method returns a Series of booleans where True represents duplicate rows.

Parameters

1. subset | string or array-like of string | optional

The label of the columns to consider. By default, all columns are considered.

2. keep | boolean or string | optional

The marking rule for duplicates:

Value

Description

"first"

All duplicates are marked as True except the first one.

"last"

All duplicates are marked as True except the last one.

False

All duplicates are marked as True.

By default, keep="first".

Return value

A Series where True represents duplicate rows.

Examples

Consider the following DataFrame:

df = pd.DataFrame({"A":[1,2,1], "B":[3,4,3]})
df
A B
0 1 3
1 2 4
2 1 3

Here, the 1st and 3rd rows are duplicate.

Specifying the keep parameter

first

To mark all duplicate rows except the first one:

df.duplicated() # or explicitly set keep="first"
0 False
1 False
2 True
dtype: bool

last

To mark all duplicate rows except the last one:

df.duplicated(keep="last")
0 True
1 False
2 False
dtype: bool

False

To mark all duplicate rows as True:

df.duplicated(keep=False)
0 True
1 False
2 True
dtype: bool
mail
Join our newsletter for updates on new DS/ML comprehensive guides (spam-free)
robocat
Published by Isshin Inada
Edited by 0 others
Did you find this page useful?
thumb_down
Ask a question or leave a feedback...
0
thumb_down
0
chat_bubble_outline
0
settings
Enjoy our search
Hit / to insta-search docs and recipes!