Creating DataFrames Cookbook

Combining multiple Series into a DataFrame Combining multiple Series to form a DataFrame Converting a Series to a DataFrame Converting list of lists into DataFrame Converting list to DataFrame Converting percent string into a numeric for read_csv Converting scikit-learn dataset to Pandas DataFrame Converting string data into a DataFrame Creating a DataFrame from a string Creating a DataFrame using lists Creating a DataFrame with different type for each column Creating a DataFrame with empty values Creating a DataFrame with missing values Creating a DataFrame with random numbers Creating a DataFrame with zeros Creating a MultiIndex DataFrame Creating a Pandas DataFrame Creating a single DataFrame from multiple files Creating empty DataFrame with only column labels Filling missing values when using read_csv Importing Dataset Initialising a DataFrame using a constant Initialising a DataFrame using a dictionary Initialising a DataFrame using a list of dictionaries Inserting lists into a DataFrame cell Keeping leading zeroes when using read_csv Parsing dates when using read_csv Preventing strings from getting parsed as NaN for read_csv Reading data from GitHub Reading file without header Reading large CSV files in chunks Reading n random lines using read_csv Reading space-delimited files Reading specific columns from file Reading tab-delimited files Reading the first few lines of a file to create DataFrame Reading the last n lines of a file Reading URL using read_csv Reading zipped csv file as a DataFrame Removing Unnamed:0 column Resolving ParserError: Error tokenizing data Saving DataFrame as zipped csv Skipping rows without skipping header for read_csv Specifying data type for read_csv Treating missing values as empty strings rather than NaN for read_csv

thumb_up

thumb_down

chat_bubble_outline
0
 auto_stories new 
settings

Creating a single DataFrame from multiple files in Pandas

Programming

Python

Pandas

Cookbooks

DataFrame Cookbooks

Creating DataFrames Cookbook

Mar 9, 2022

Last updated

local_offer Python●Pandas

When files contain columns

Consider the following my_data_one.txt file:

And my_data_two.txt file:

To read and combine the two files into a single DataFrame:


        
        
            
                
                
                    df_one = pd.read_csv("my_data_one.txt")
df_two = pd.read_csv("my_data_two.txt")
pd.concat([df_one, df_two], axis=1)
                
            
               A  B  C   D
0  3  4  10  11
1  5  6  12  13

Here, axis=1 indicates that we want to concatenate the DataFrames horizontally, as opposed to vertically.

When files contain rows

Consider the following my_data_one.txt file:

And my_data_two.txt:

To construct a single DataFrame from the two files:


        
        
            
                
                
                    df_one = pd.read_csv("my_data_one.txt")
df_two = pd.read_csv("my_data_two.txt")
df = pd.concat([df_one, df_two], axis=0, ignore_index=True)
df
                
            
               A  B
0  3  4
1  5  6
2  10  11
3  12  13

Note the following:

axis=0 for concat(~) means that we want to stack the DataFrames vertically, as opposed to horizontally.
ignore_index=True for concat(~) resets the index of the resulting DataFrame to the default integer indices ([0,1,2,3]). Without this parameter, we would end up with duplicate index values ([0,1,0,1]),

Pandas | read_csv method

Reads a file, and parses its content into a DataFrame.

Pandas | concat method

Concatenates a list of Series or DataFrame, either horizontally or vertically.