Apache Airflow Interview Questions Interview Guide

Sample Answer:

Built ETL DAG with 25 tasks: data extraction, transformation, validation, loading. Used task groups for modular design, set dependencies with cross-dependencies. Implemented failure notifications and retry logic.

Sample Answer:

Enabled task logging, checked logs in UI, inspected sensor status. Found issue: downstream task failing due to upstream data format change. Added schema validation task to catch earlier.

Sample Answer:

Idempotent tasks, clear naming, logical dependencies, task groups for organization, proper error handling, monitoring, alerts. Avoid complex logic in DAG definition, use PythonOperator for computation.

Sample Answer:

Cron: fixed schedule (e.g., daily 2am). Dynamic: triggered by events or data arrival. Use cron for regular jobs, dynamic for reactive workflows.

Sample Answer:

Use catchup=True, specify start_date. Airflow backfills automatically. For large backfills, disable catchup, manually trigger with clear=True. Monitor resources during backfill.

Sample Answer:

Set retries and retry_delay per task. Use exponential backoff. Implement on_failure callbacks for alerts. For critical tasks, set trigger_rule=all_done to continue pipeline.