Apache airflow macro to get last dag run execution time

by

Yes, you can define your own custom macro for this as follows:

# custom macro function
def get_last_dag_run(dag):
    last_dag_run = dag.get_last_dagrun()
    if last_dag_run is None:
        return "no prev run"
    else:
        return last_dag_run.execution_date.strftime("%Y-%m-%d")

# add macro in user_defined_macros in dag definition
dag = DAG(dag_id="my_test_dag",
      schedule_interval="@daily",
      user_defined_macros={
          'last_dag_run_execution_date': get_last_dag_run
      }
)

# example of using it in practice
print_vals = BashOperator(
    task_id='print_vals',
    bash_command='echo {{ last_dag_run_execution_date(dag) }}',
    dag=dag
)

Note that the dag.get_last_run() is just one of the many functions available on the Dag object. Here’s where I found it: https://github.com/apache/incubator-airflow/blob/v1-10-stable/airflow/models.py#L3396

You can also tweak the formatting of the string for the date format, and what you want output if there is no previous run.

More Related Contents:

  1. Airflow 1.9.0 is queuing but not launching tasks
  2. Problem with start date and scheduled date in Apache Airflow
  3. Wiring top-level DAGs together
  4. Airflow – How to pass xcom variable into Python function
  5. Airflow: how to delete a DAG?
  6. For Apache Airflow, How can I pass the parameters when manually trigger DAG via CLI?
  7. Airflow: changing the crontab time for a DAG in Airflow
  8. Why is it recommended against using a dynamic start_date in Airflow?
  9. Create and use Connections in Airflow operator at runtime [duplicate]
  10. Make custom Airflow macros expand other macros
  11. How to get last two successful execution dates of Airflow job?
  12. execution_date in airflow: need to access as a variable
  13. Airflow BashOperator return exit code 0 even when task failed and return exit code 1
  14. Pip error even Microsoft Visual C++ 14.0 is installed
  15. How to submit Spark jobs to EMR cluster from Airflow?
  16. Proper way to create dynamic workflows in Airflow
  17. setting up s3 for logs in airflow
  18. Airflow: how to get pip packages installed via their docker-compose.yml?
  19. How to create a conditional task in Airflow
  20. How to run Airflow on Windows
  21. pass task instance variables as op_args
  22. How to prevent airflow from backfilling dag runs?
  23. How to control the parallelism or concurrency of an Airflow installation?
  24. Airflow “This DAG isnt available in the webserver DagBag object “
  25. Apache Airflow scheduler does not trigger DAG at schedule time
  26. Airflow s3 connection using UI

Leave a Comment