Add Rows with Default Values in Pandas DataFrame

Adding rows with default values can be useful when you’re working with missing or incomplete data.

In this tutorial, you’ll learn how to add rows with default values to your DataFrames, whether you’re starting from scratch or working with an existing DataFrame.

 

 

Using loc[]

The loc[] property allows you to add new rows to your DataFrame, and you can specify default values for these new rows.

Let’s start by creating a sample DataFrame:

import pandas as pd
data = {'User_ID': [1, 2, 3],
        'Plan_Type': ['Basic', 'Premium', 'Basic'],
        'Monthly_Charge': [50, 80, 50]}
df = pd.DataFrame(data)
print(df)

Output:

   User_ID Plan_Type  Monthly_Charge
0        1     Basic              50
1        2   Premium              80
2        3     Basic              50

Here we have a DataFrame with columns ‘User_ID’, ‘Plan_Type’, and ‘Monthly_Charge’.

Now, let’s add a new row with a default value using loc[]:

df.loc[3] = [5, 'Premium', 80]
print(df)

Output:

   User_ID Plan_Type  Monthly_Charge
0        1     Basic              50
1        2   Premium              80
2        3     Basic              50
3        5   Premium              80

 

Using iloc[] (Insert at Specific Position)

The iloc property is useful when you want to insert a row at a specific position rather than at the end of the DataFrame.

Let’s add a new row at the second position (index 1) with default values using iloc[]:

# Create a temporary DataFrame with the new row
temp_df = pd.DataFrame([[6, 'Basic', 50]], columns=df.columns)

# Split the original DataFrame and append the new row
df1 = df.iloc[:1]
df2 = df.iloc[1:]

# Concatenate all three DataFrames
df = pd.concat([df1, temp_df, df2]).reset_index(drop=True)
print(df)

Output:

0        1     Basic              50
1        6     Basic              50
2        2   Premium              80
3        3     Basic              50
4        5   Premium              80

You can see that a new row is inserted at the second position (index 1).

 

Using concat() for Multiple Row Addition

The concat() function is highly useful for this operation, offering a fast and efficient way to combine DataFrames.

Continuing with our DataFrame, let’s add multiple new rows with default values using concat():

new_rows = pd.DataFrame({
    'User_ID': [7, 8],
    'Plan_Type': ['Basic', 'Premium'],
    'Monthly_Charge': [50, 80]
})

# Concatenate the original DataFrame with the new DataFrame
df = pd.concat([df, new_rows]).reset_index(drop=True)
print(df)

Output:

   User_ID Plan_Type  Monthly_Charge
0        1     Basic              50
1        6     Basic              50
2        2   Premium              80
3        3     Basic              50
4        5   Premium              80
5        7     Basic              50
6        8   Premium              80

The reset_index(drop=True) part ensures that the DataFrame index is reset, maintaining a clean, sequential numbering.

 

Using reindex()

The reindex() method offers another way to add rows with default values to a DataFrame.

With reindex(), you can expand the DataFrame’s index and automatically fill the new rows with default values or NaN (Not a Number) if not specified.

Let’s proceed with our ongoing DataFrame example by using reindex() to add new rows:

new_index_range = list(range(0, 10))
fill_vals = pd.Series([9, 'Basic', 50], index=df.columns)
df = df.reindex(new_index_range).fillna(fill_vals)
print(df)

Output:

   User_ID Plan_Type  Monthly_Charge
0      1.0     Basic            50.0
1      6.0     Basic            50.0
2      2.0   Premium            80.0
3      3.0     Basic            50.0
4      5.0   Premium            80.0
5      7.0     Basic            50.0
6      8.0   Premium            80.0
7      9.0     Basic            50.0
8      9.0     Basic            50.0
9      9.0     Basic            50.0

In this code, I’ve created a fill_vals Series with scalar values, using the column names from the DataFrame as the index.

Then, I use reindex() to expand the DataFrame with the new rows and fillna() to fill the new rows with the default values from the fill_vals Series.

 

Dynamic Row Addition with Default Values

Let’s say you want to dynamically add rows where the ‘Monthly_Charge’ is less than 60.

Here’s how you can do it:

# Initialize the next user ID
next_user_id = 10
for index, row in df.iterrows():
    if row['Monthly_Charge'] < 60:
        df.loc[len(df)] = [next_user_id, 'Basic', 50]
        next_user_id += 1
print(df)

Output:

    User_ID Plan_Type  Monthly_Charge
0       1.0     Basic            50.0
1       6.0     Basic            50.0
2       2.0   Premium            80.0
3       3.0     Basic            50.0
4       5.0   Premium            80.0
5       7.0     Basic            50.0
6       8.0   Premium            80.0
7       9.0     Basic            50.0
8       9.0     Basic            50.0
9       9.0     Basic            50.0
10     10.0     Basic            50.0
11     11.0     Basic            50.0
12     12.0     Basic            50.0
13     13.0     Basic            50.0
14     14.0     Basic            50.0
15     15.0     Basic            50.0
16     16.0     Basic            50.0

New rows have been added dynamically for each existing row where the ‘Monthly_Charge’ is less than 60.

Each new row is assigned the next available ‘User_ID’ and filled with default values for ‘Plan_Type’ and ‘Monthly_Charge’.