how to merge some csv files into one file

Solution 1:

This sounds like a direct implementation of one of the Pandas examples for concat(). Copying the relevant example from their documentation:

>>> df1 = pd.DataFrame([['a', 1], ['b', 2]],
                   columns=['letter', 'number'])
>>> df1
  letter  number
0      a       1
1      b       2
>>> df3 = pd.DataFrame([['c', 3, 'cat'], ['d', 4, 'dog']],
                   columns=['letter', 'number', 'animal'])
>>> df3
  letter  number animal
0      c       3    cat
1      d       4    dog
>>> pd.concat([df1, df3], sort=False)
  letter  number animal
0      a       1    NaN
1      b       2    NaN
0      c       3    cat
1      d       4    dog

I usually like to call df.reset_index() on the resulting Dataframe df as well, since having duplicate values in the index can cause unexpected behavior. If you're about to do a join on one of the columns, though, it won't matter.... although you've already got ignore_index=True in your sample code, so you should be fine.