Git Clone Workflow

To avoid the dif­fi­cul­ties that we caused ourselves with Airflow 1, we created a fresh repository for Airflow 2, which does not have a copy of the upstream repo’s code. We now maintain a set of patches for each upstream release that we care about. This new repo has build scripts and patches only.

When I first set this up, I had the CI build script create a shallow clone of the upstream repo, then check out each tag, and apply our patches.

# NOT SHOWN: create a virtualenv with Hatch and other build dependencies
# from Airflow's pyproject.toml

git clone --depth=1 https://github.com/apache/airflow.git worktree
cd worktree

for tag in ("2.10.2" "2.10.4"); do
    git reset --hard HEAD
    rm -rf dist
    git fetch --depth 1 origin "$tag"
    git checkout --quiet FETCH_HEAD

    for p in ../patches/"$tag"/*.patch; do
        git am < "$p"
    done

    python3 -m build --wheel
    cp dist/* ../build
done

The first patch for each tag changes the version in­for­ma­tion so that our wheel won’t conflict with the official wheel from upstream. It updates tool.hatch.version in pyproject.toml to read:

[tool.hatch.version]
source = "code"
expression = "stripe_airflow_version()"
path = "stripe_version.py"

instead of extracting the version in­for­ma­tion from airflow/__init__.py.

The stripe_ver­sion.py script uses git describe to get the number of additional commits in our branch and the ab­bre­vi­at­ed SHA of the most recent commit, then prefixes these items with +stripe.${MAJOR}. All of this is suffixed to the actual version number from upstream, so we build a wheel that is named something like apache_air­flow-${TAG}+stripe.1.${COUNT}.g${SHA}-py3-none-any.whl.

While this system produced a working wheel, there was one critical omission. The official upstream wheel contained an extra 37MB of UI code in www/static, which is used by the various Airflow website UIs.

I spent quite a bit of effort to make our build generate this extra payload, but it turned out to be very difficult. python3 -m hatch build -t custom requires Node.js and does a lot of extra steps that didn’t interact well with the locked down egress rules of our CI.

Source Dis­tri­b­u­tion Workflow

I realized that all of the www/static tree could be extracted from the official release, and that we didn’t have to generate it in CI.

Instead of checking out a tag, our CI downloads the official source dis­tri­b­u­tion tarball, apache_air­flow-${RELEASE}.tar.gz, untars the tarball, applies our patches, and builds a new wheel.

It took me a while to figure out why our custom versioning wasn’t working. Because the sdist contains a file called PKG-INFO at the root, Hatch takes the version from that. I had to update the stripe_ver­sion.py script to modify the Version: line in PKG-INFO.

Format-Patch Workflow

So far, I’ve covered how the patched wheel is built in CI, but not how you would create new patches.

For local de­vel­op­ment, you can check out the upstream tag (see FETCH_HEAD above), then apply any existing patches that are relevant. Make other changes, commit them locally, and build the wheel by hand. When you have tested and have something that you’re happy with, you can use git format-patch to create a series of patches. These patches can then be committed to the repo that we use to build the wheels.

This workflow is less convenient than making changes directly in the forked code, as we did with Airflow 1. But now we only have a moderate amount of friction to upgrade to a newer release from upstream, instead of ever-increasing difficulty.