Configure GC in Azure Databricks
Set these in the cluster configuration
spark.driver.extraJavaOptions -XX:+G1HeapRegionSize
spark.executor.extraJavaOptions -XX:+G1HeapRegionSize
Set these in the cluster configuration
spark.driver.extraJavaOptions -XX:+G1HeapRegionSize
spark.executor.extraJavaOptions -XX:+G1HeapRegionSize