LEVERAGING DATA ASSETS FEATURES IN AIRFLOW 3.0 TO OPTIMISE RESOURCE UTILIZATION BY MORE THAN 30%

Summary from the TLDR Data newsletter

Halodoc migrated Airflow workloads from polling sensors and synchronous Redshift loads to Airflow 3.x Assets and deferrable operators. Across 160 DAGs, the team reduced worker CPU from 26.1% to 7.71% and worker memory from 49.2% to 30.8%, while also lowering scheduler load and reducing Redshift table-locking errors by about 38%.