Handling Missing Values

Adding missing dates in Datetime Index Checking if a certain value in a DataFrame is NaN Checking if a DataFrame contains any missing values Converting a column with missing values to integer type Counting non-missing values Counting number of rows with missing values Counting the number of NaN in each row of a DataFrame Counting number of NaN values in each column of a DataFrame Counting the total number of NaN values of a DataFrame Filling missing values using another column Filling missing values with the mean of the column Finding columns with missing values Getting integer indexes of rows with NaN Getting rows with missing values Getting rows with missing values in certain columns Getting index of rows with missing values (NaNs)Getting index of rows without missing values Mapping NaN values to 0 and non-NaN values to 1 Mapping NaN values to False and non-NaN values to True Removing columns where some rows contain missing values Removing rows from a DataFrame with missing values Replacing all NaN values of a DataFrame Replacing all NaN values with zeros in a DataFrame Replacing missing values Replacing missing values with constants Replacing NaN with blank string Replacing NaNs for certain columns Replacing NaNs with preceding values Replacing values with NaNs Using interpolation to fill missing values

thumb_up

thumb_down

chat_bubble_outline
0
 auto_stories new 
settings

Replacing values with NaNs in Pandas DataFrame

Programming

Python

Pandas

Cookbooks

DataFrame Cookbooks

Handling Missing Values

Mar 9, 2022

Last updated

local_offer Python●Pandas

Replacing value with NaN

Consider the following DataFrame:


        
        
            
                
                
                    df = pd.DataFrame({"A":[3,"NONE"]})
df
                
            
               A
0  3
1  NONE

To replace "NONE" values with NaN:


        
        
            
                
                
                    import numpy as np
df.replace("NONE", np.nan)
                
            
               A
0  3.0
1  NaN

Note that the replacement is not done in-place, that is, a new DataFrame is returned and the original df is kept intact. To perform the replacement in-place, set inplace=True.

Replacing multiple values with NaN

Consider the following DataFrame:


        
        
            
                
                
                    df = pd.DataFrame({"A":[3,5],"B":[4,6]}, index=["a","b"])
df
                
            
               A  B
a  3  4
b  5  6

To replace values 3 and 4 with np.nan:


        
        
            
                
                
                    df.replace([3,4], np.nan)
                
            
               A    B
a  NaN  NaN
b  5.0  6.0

Conditionally replacing values with NaN

Consider the following DataFrame:


        
        
            
                
                
                    df = pd.DataFrame({"A":[3,5],"B":[4,6]}, index=["a","b"])
df
                
            
               A  B
a  3  4
b  5  6

To replace values larger than 4 with NaN:


        
        
            
                
                
                    df[df > 4] = np.nan
df
                
            
               A    B
a  3.0  4.0
b  NaN  NaN