airflow-hello world

1. 编写dag

在dags目录下新建hello_world.py文件，编写代码如下：

# -*- coding: utf-8 -*-
import airflow
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.python_operator import PythonOperator
from datetime import timedelta

default_args = {
    'owner': '4change',
    'depends_on_past': False,
    'start_date': airflow.utils.dates.days_ago(2),
    'email': ['[email protected]'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
    # 'queue': 'bash_queue',
    # 'pool': 'backfill',
    # 'priority_weight': 10,
    # 'end_date': datetime(2016, 1, 1),
    # 'wait_for_downstream': False,
    # 'dag': dag,
    # 'adhoc':False,
    # 'sla': timedelta(hours=2),
    # 'execution_timeout': timedelta(seconds=300),
    # 'on_failure_callback': some_function,
    # 'on_success_callback': some_other_function,
    # 'on_retry_callback': another_function,
    # 'trigger_rule': u'all_success'
}


def print_hello():
    return 'Hello world!'


dag = DAG('hello_world_dag', default_args=default_args, description='hello world dag', schedule_interval=timedelta(days=1))

date_operator = BashOperator(task_id='date_task', bash_command='date', dag=dag)
sleep_operator = BashOperator(task_id='sleep_task', depends_on_past=False, bash_command='sleep 5', dag=dag)
hello_operator = PythonOperator(task_id='hello_task', python_callable=print_hello, dag=dag)

# 设置各任务间的依赖关系
sleep_operator.set_upstream(date_operator)
hello_operator.set_upstream(date_operator)

2. 测试dag

python3 hello_world.py								# 测试python文件的编写是否正常
airflow list_tasks hello_world_dag					# 列出hello_world_dag这个dag下的相关任务
airflow test hello_world_dag data_task 20190325		# 测试hello_world_dag下的data_task任务，时间为20190325
airflow test hello_world_dag hello_task 20190325	# 测试hello_world_dag下的hello_task任务，时间为20190325

3. 运行dag

airflow scheduler				# 开始调度airflow相关任务

参考文档

[AirFlow]AirFlow使用指南三 第一个DAG示例
[AirFlow]AirFlow使用指南二 DAG定义文件