How to load only newly added file in Azure SQL DB
I am trying to load data from csv file to Azure SQL DB using copy activity. First I loaded three files from blob storage to Azure SQL DB. Then again Three new files are uploaded to blob storage and now I want to load only newly added file to azure sql db. File name are in this format: "student_index_date" where index are from 1-6 and I have to make use of this index.
Solution 1:
You can use Getmetadata activity to get the list of child items and then filter based on the latest modified dt for the latest file and use it as source in copy activity.