Climate forecasting ML isn’t a notebook problem it’s a pipeline problem. This guide shows an end-to-end MLOps setup on Kubernetes + Airflow: NetCDF/Zarr preprocessing, PyTorch training, MLflow tracking, CI/CD, and monitoring for drift + failures.
#ClimateTech #MLOps #Kubernetes #ApacheAirflow #PyTorch #DataEngineering
💡 Apache Airflow 2025 Recap
2026 has arrived which marks a great opportunity to review and assess the changes the last year has brought to the ever evolving landscape of open source data tools.
In our new #blog post we look at Apache Airflow and how the leading open source orchestration platform has changed in the last year with the bit v3 major release.
#apacheairflow #airflow #opensource #dataengineering #datascience
Building data pipelines with Java and open source
"The release served as a crucial turning point for the project. Downloads from its GitHub repository increased, and more enterprises adopted the software. Encouraged by this growth, the team envisioned the next generation of Airflow: a modular architecture, a more modern user interface, and a “run anywhere, anytime” feature, enabling it to operate on premises, in the cloud, or on edge devices and handle event-driven and ad hoc scenarios in addition to scheduled tasks. The team delivered on this vision with the launch of Airflow 3.0 last April.
“It was amazing that we managed to ‘rebuild the plane while flying it’ when we worked on Airflow 3—even if we had some temporary issues and glitches,” says Jarek Potiuk, one of the foremost contributors to Airflow and now a member of its project-management committee. “We had to refactor and move a lot of pieces of the software while keeping Airflow 2 running and providing some bug fixes for it.”
Compared with Airflow’s second version, which Koka says had only a few hundred to a thousand downloads per month on GitHub, “now we’re averaging somewhere between 35 to 40 million downloads a month,” he says. The project’s community also soared, with more than 3,000 developers of all skill levels from around the world contributing to Airflow."
https://spectrum.ieee.org/apache-airflow-3-programmatic-workflows
#AirFlow #ApacheAirflow #AirBnB #OpenSource #FLOSS #WorkflowOrchestratror #Python #DataPipelines
Release v1.0.0 · terraform-provider-airflow
🚨 Apache Airflow 3.0.3 flaw (CVE-2025-54831) exposes sensitive connection info, including passwords, tokens & API keys, to read-only users.
🔧 Fix: upgrade to v3.0.4 or later.
💬 Should open-source software handling sensitive data undergo mandatory independent audits before release?
👉 Follow @technadu for more cybersecurity updates.
#CyberSecurity #ApacheAirflow #CVE202554831 #OpenSourceSecurity #DevSecOps #InfoSec #Vulnerability #DataSecurity
Nothing screams "we're a serious data company" like having your CEO resign over a Kiss Cam incident, then hiring your performer's ex-wife to pitch Apache Airflow. Modern corporate communications, everyone! 🎪
The "Chief People Officer" of dataops company Astronomer resigned this week from her position after apparently being caught on that "Kiss Cam" at a Coldplay concert with the company's CEO, reports the BBC. That CEO has also resigned, with Astronomer appointing their original co-founder and chief p...
New to #ApacheAirflow? Build your first workflow!
Learn how this #opensource orchestration tool helps you schedule, monitor & manage data pipelines in #Python. Watch Ricardo Sueiras guide beginners through hands-on steps.
Click here: https://youtu.be/sdBimSGxAcs
Neues Spielzeug gefunden #airflow ;P
Code Labs Academy
NextLytics AG
Marcus Fihlon (McPringle)
Miguel Afonso Caetano
lens0021
TechNadu
Offensive Sequence
Loki the Cat
FOSSASIA
CoreSec