Hive: Add partitions for existing folder structure
I have a folder structure in HDFS like below. However, no partitions were actually created on the table using the ALTER TABLE ADD PARTITION
commands, even though the folder structure was setup as if the table had partitions.
How can I automatically add all the partitions to the Hive table? (Hive 1.0, external table)
/user/frank/clicks.db
/date=20190401
/file0004.csv
/date=20190402
/file0009.csv
/date=20190501
/file0000.csv
/file0001.csv
...etc
Solution 1:
Use msck repair table command:
MSCK [REPAIR] TABLE tablename;
or
ALTER TABLE tablename RECOVER PARTITIONS;
if you are running Hive on EMR.
Read more details about both commands here: RECOVER PARTITIONS