bazel: refresh the expired macOS SDK pin (#16128)
## Why
macOS BuildBuddy started failing before target analysis because the
Apple CDN object pinned in
[`MODULE.bazel`](https://github.com/openai/codex/blob/fce0f76d577b5070f1e2b4a2abaa8350acfc38ff/MODULE.bazel#L28-L36)
now returns `403 Forbidden`. The failure report that triggered this
change was this [BuildBuddy
invocation](https://app.buildbuddy.io/invocation/c57590e0-1bdb-4e19-a86f-74d4a7ded228).
This repo uses `@llvm//extensions:osx.bzl` via `osx.from_archive(...)`,
and that API does not discover a current SDK URL for us. It fetches
exactly the `urls`, `sha256`, and `strip_prefix` we pin. Once Apple
retires that `swcdn.apple.com` object, `@macos_sdk` stops resolving and
every downstream macOS build fails during external repository fetch.
This is the same basic failure mode we hit in
[b9fa08ec619c96617a9ae2041c9ddb02d2c02434](https://github.com/openai/codex/commit/b9fa08ec619c96617a9ae2041c9ddb02d2c02434):
the pin itself aged out.
## How I tracked it down
1. I started from the BuildBuddy error and copied the exact
`swcdn.apple.com/.../CLTools_macOSNMOS_SDK.pkg` URL from the failure.
2. I reproduced the issue outside CI by opening that URL directly in a
browser and by running `curl -I` against it locally. Both returned `403
Forbidden`, which ruled out BuildBuddy as the root cause.
3. I searched the repo for that URL and found it hardcoded in
`MODULE.bazel`.
4. I inspected the `llvm` Bzlmod `osx` extension implementation to
confirm that `osx.from_archive(...)` is just a literal fetch of the
pinned archive metadata. There is no automatic fallback or catalog
lookup behind it.
5. I queried Apple's software update catalogs to find the current
Command Line Tools package for macOS 26.x. The useful catalog was:
-
`https://swscan.apple.com/content/catalogs/others/index-26-15-14-13-12-10.16-10.15-10.14-10.13-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz`
This is scriptable; it does not require opening a website in a browser.
The catalog is a gzip-compressed plist served over HTTP, so the workflow
is just:
1. fetch the catalog,
2. decompress it,
3. search or parse the plist for `CLTools_macOSNMOS_SDK.pkg` entries,
4. inspect the matching product metadata.
The quick shell version I used was:
```shell
curl -L <catalog-url> \
| gzip -dc \
| rg -n -C 6 'CLTools_macOSNMOS_SDK\.pkg|PostDate|English\.dist'
```
That is enough to surface the current product id, package URL, post
date, and the matching `.dist` file. If we want something less
grep-driven next time, the same catalog can be parsed structurally. For
example:
```python
import gzip
import plistlib
import urllib.request
url =
"https://swscan.apple.com/content/catalogs/others/index-26-15-14-13-12-10.16-10.15-10.14-10.13-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz"
with urllib.request.urlopen(url) as resp:
catalog = plistlib.loads(gzip.decompress(resp.read()))
for product_id, product in catalog["Products"].items():
for package in product.get("Packages", []):
package_url = package.get("URL", "")
if package_url.endswith("CLTools_macOSNMOS_SDK.pkg"):
print(product_id)
print(product.get("PostDate"))
print(package_url)
print(product.get("Distributions", {}).get("English"))
```
In practice, `curl` was only the transport. The important part is that
the catalog itself is a machine-readable plist, so this can be
automated.
6. That catalog contains the newer `047-96692` Command Line Tools
release, and its distribution file identifies it as [Command Line Tools
for Xcode
26.4](https://swdist.apple.com/content/downloads/32/53/047-96692-A_OAHIHT53YB/ybtshxmrcju8m2qvw3w5elr4rajtg1x3y3/047-96692.English.dist).
7. I downloaded that package locally, computed its SHA-256, expanded it
with `pkgutil --expand-full`, and verified that it contains
`Payload/Library/Developer/CommandLineTools/SDKs/MacOSX26.4.sdk`, which
is the correct new `strip_prefix` for this pin.
The core debugging loop looked like this:
```shell
curl -I <stale swcdn URL>
rg 'swcdn\.apple\.com|osx\.from_archive' MODULE.bazel
curl -L <apple 26.x sucatalog> | gzip -dc | rg 'CLTools_macOSNMOS_SDK.pkg'
pkgutil --expand-full CLTools_macOSNMOS_SDK.pkg expanded
find expanded/Payload/Library/Developer/CommandLineTools/SDKs -maxdepth 1 -mindepth 1
```
## What changed
- Updated `MODULE.bazel` to point `osx.from_archive(...)` at the
currently live `047-96692` `CLTools_macOSNMOS_SDK.pkg` object.
- Updated the pinned `sha256` to match that package.
- Updated the `strip_prefix` from `MacOSX26.2.sdk` to `MacOSX26.4.sdk`.
## Verification
- `bazel --output_user_root="$(mktemp -d
/tmp/codex-bazel-sdk-fetch.XXXXXX)" build @macos_sdk//sysroot`
## Notes for next time
As long as we pin raw `swcdn.apple.com` objects, this will likely happen
again. When it does, the expected recovery path is:
1. Reproduce the `403` against the exact URL from CI.
2. Find the stale pin in `MODULE.bazel`.
3. Look up the current CLTools package in the relevant Apple software
update catalog for that macOS major version.
4. Download the replacement package and refresh both `sha256` and
`strip_prefix`.
5. Validate the new pin with a fresh `@macos_sdk` fetch, not just an
incremental Bazel build.
The important detail is that the non-`26` catalog did not surface the
macOS 26.x SDK package here; the `index-26-15-14-...` catalog was the
one that exposed the currently live replacement. M
Michael Bolin committed
ae8a3be9585d0918fe38fa199c963aeb80ee4f55
Parent: bc53d42
Committed by GitHub <noreply@github.com>
on 3/28/2026, 9:08:19 PM