Airflow: how to delete a DAG?
I have started the Airflow webserver and scheduled some dags. I can see the dags on web GUI.
How can I delete a particular DAG from being run and shown in web GUI? Is there an Airflow CLI command to do that?
I looked around but could not find an answer for a simple way of deleting a DAG once it has been loaded and scheduled.
Edit 8/27/18 - Airflow 1.10 is now released on PyPI!
https://pypi.org/project/apache-airflow/1.10.0/
How to delete a DAG completely
We have this feature now in Airflow ≥ 1.10!
The PR #2199 (Jira: AIRFLOW-1002) adding DAG removal to Airflow has now been merged which allows fully deleting a DAG's entries from all of the related tables.
The core delete_dag(...) code is now part of the experimental API, and there are entrypoints available via the CLI and also via the REST API.
CLI:
airflow delete_dag my_dag_id
REST API (running webserver locally):
curl -X "DELETE" http://127.0.0.1:8080/api/experimental/dags/my_dag_id
Warning regarding the REST API: Ensure that your Airflow cluster uses authentication in production.
Installing / upgrading to Airflow 1.10 (current)
To upgrade, run either:
export SLUGIFY_USES_TEXT_UNIDECODE=yes
or:
export AIRFLOW_GPL_UNIDECODE=yes
Then:
pip install -U apache-airflow
Remember to check UPDATING.md first for the full details!
This is my adapted code using PostgresHook with the default connection_id.
import sys
from airflow.hooks.postgres_hook import PostgresHook
dag_input = sys.argv[1]
hook=PostgresHook( postgres_conn_id= "airflow_db")
for t in ["xcom", "task_instance", "sla_miss", "log", "job", "dag_run", "dag" ]:
sql="delete from {} where dag_id='{}'".format(t, dag_input)
hook.run(sql, True)
Not sure why Apache Airflow doesn't have an obvious and easy way to delete a DAG
Filed https://issues.apache.org/jira/browse/AIRFLOW-1002