python Pandas DataFrame copy(deep=False) vs copy(deep=True) vs '='
If you see the object IDs of the various DataFrames you create, you can clearly see what is happening.
When you write df2 = df1, you are creating a variable named df2, and binding it with an object with id 4541269200. When you write df1 = pd.DataFrame([9,9,9]), you are creating a new object with id 4541271120 and binding it to variable df1, but the object with id 4541269200 which was previously bound to df1 continues to live. If there were no variables bound to that object, it will get garbage collected by Python.
In[33]: import pandas as pd
In[34]: df1 = pd.DataFrame([1,2,3,4,5])
In[35]: id(df1)
Out[35]: 4541269200
In[36]: df2 = df1
In[37]: id(df2)
Out[37]: 4541269200 # Same id as df1
In[38]: df3 = df1.copy()
In[39]: id(df3)
Out[39]: 4541269584 # New object, new id.
In[40]: df4 = df1.copy(deep=False)
In[41]: id(df4)
Out[41]: 4541269072 # New object, new id.
In[42]: df1 = pd.DataFrame([9, 9, 9])
In[43]: id(df1)
Out[43]: 4541271120 # New object created and bound to name 'df1'.
In[44]: id(df2)
Out[44]: 4541269200 # Old object's id not impacted.
Edit: Added on 7/30/2018
Deep copying doesn't work in pandas and the devs consider putting mutable objects inside a DataFrame as an antipattern. Consider the following:
In[10]: arr1 = [1, 2, 3]
In[11]: arr2 = [1, 2, 3, 4]
In[12]: df1 = pd.DataFrame([[arr1], [arr2]], columns=['A'])
In[13]: df1.applymap(id)
Out[13]:
A
0 4515714832
1 4515734952
In[14]: df2 = df1.copy(deep=True)
In[15]: df2.applymap(id)
Out[15]:
A
0 4515714832
1 4515734952
In[16]: df2.loc[0, 'A'].append(55)
In[17]: df2
Out[17]:
A
0 [1, 2, 3, 55]
1 [1, 2, 3, 4]
In[18]: df1
Out[18]:
A
0 [1, 2, 3, 55]
1 [1, 2, 3, 4]
df2, if it was a true deep copy should have had new ids for the lists contained within it. As a result, when you modify a list inside df2, it affects the list inside df1 as well, because they are the same objects.
Deep copy creates new id's of every object it contains while normal copy only copies the elements from the parent and creates a new id for a variable to which it is copied to.
The reason for none of df2, df3 and df4 displaying [9,9,9] is:
In[33]: import pandas as pd
In[34]: df1 = pd.DataFrame([1,2,3,4,5])
In[35]: id(df1)
Out[35]: 4541269200
In[36]: df2 = df1
In[37]: id(df2)
Out[37]: 4541269200 # Same id as df1
In[38]: df3 = df1.copy()
In[39]: id(df3)
Out[39]: 4541269584 # New object, new id.
In[40]: df4 = df1.copy(deep=False)
In[41]: id(df4)
Out[41]: 4541269072 # New object, new id.
In[42]: df1 = pd.DataFrame([9, 9, 9])
In[43]: id(df1)
Out[43]: 4541271120 # New object created and bound to name 'df1'.
You need to modify df's elements individually. Try the following
df1 = pd.DataFrame([1,2,3,4,5])
df2 = df1
df3 = df1.copy()
df4 = df1.copy(deep=False)
df1.iloc[0,0] = 6
df2.iloc[1,0] = 7
df4.iloc[2,0] = 8
print(df1)
print(df2)
print(df3)
print(df4)
df1: df2: df3: df4:
0 0 0 0
0 6 0 6 0 1 0 6
1 7 1 7 1 2 1 7
2 8 2 8 2 3 2 8
3 4 3 4 3 4 3 4
4 5 4 5 4 5 4 5