How to configure druid batch indexing jobs dynamic EMR cluster for batch ingestion?

I have tried out overriding the parameters ( Hadoop configuration) in core-site.xml,yarn-site.xml,mapred-site.xml,hdfs-site.xml as Job properties in druid indexing job. It worked. In that case no need of copying the above files in druid server.

Just used below python program to convert the properties to json key value pairs from xml files. Can do the same for all the files and pass everything as indexing job payload. The below thing can be automated using airflow after creating different EMR clusters.

import json
import xmltodict
path = 'mypath'
file = 'yarn-site.xml'
with open(os.path.join(path,file)) as xml_file:
    data_dict = xmltodict.parse(xml_file.read())
    xml_file.close()
    druid_dict = {property.get('name'):property.get('value') for property in data_dict.get('configuration').get('property') }
    print(json.dumps(druid_dict)) ```