summaryrefslogtreecommitdiff
path: root/Docs
diff options
context:
space:
mode:
authorChristian Cleberg <[email protected]>2026-07-20 17:50:56 -0500
committerChristian Cleberg <[email protected]>2026-07-20 17:55:24 -0500
commit1e25fb947d3c01fc983b481fff5b064ed82349ab (patch)
treead69f714e93dc3228131f0221f297fad53e00e6a /Docs
parentea57052e998007dc914257f84f383c63f945cf08 (diff)
downloaddomain-dig-1e25fb947d3c01fc983b481fff5b064ed82349ab.tar.gz
domain-dig-1e25fb947d3c01fc983b481fff5b064ed82349ab.tar.bz2
domain-dig-1e25fb947d3c01fc983b481fff5b064ed82349ab.zip
ci: split accessibility coverage between local runtimes and CI
The two-job CI matrix was buying two near-identical iOS 26.x runs at double the macOS minutes. GitHub's macos-26 image ships only 26.x simulator runtimes, so it cannot test the 17.6 floor at all, and floor coverage was the entire justification for the second job. Split the work by what each side can uniquely do instead: CI keeps one job on the newest runtime. Its real value is not the runtime — it is building a clean checkout of the merge result, which catches a file that was never committed. A local run cannot, and that failure mode is live here: DomainDig.xcodeproj is hand-edited and uses file-system-synchronized groups, where an entire missing folder still builds locally. sr.ht cannot run macOS, so this is the only place that check exists. Collapsing the matrix also removed the deployment-target math, since "newest" is always above the floor. Scripts/audit-a11y.sh runs the audit against real runtimes, defaulting to floor + current. It reads the deployment target from the project rather than hard-coding it, selects the oldest runtime at or above it (one below is useless — the app cannot install), and says so plainly when the nearest installed runtime is a major version above the target rather than implying floor coverage it does not have. .githooks/pre-push runs the floor tier, and only when Swift, asset, or project files changed. Pre-push rather than pre-commit because the suite takes ~85s: at pre-commit that blocks every commit, and a hook routinely bypassed with --no-verify is worse than none. Opt in per clone with 'git config core.hooksPath .githooks'. Docs/ACCESSIBILITY.md records the split, the measured non-nested coverage that motivates it, and the enforcement ratchet.
Diffstat (limited to 'Docs')
-rw-r--r--Docs/ACCESSIBILITY.md96
1 files changed, 96 insertions, 0 deletions
diff --git a/Docs/ACCESSIBILITY.md b/Docs/ACCESSIBILITY.md
new file mode 100644
index 0000000..1045df9
--- /dev/null
+++ b/Docs/ACCESSIBILITY.md
@@ -0,0 +1,96 @@
+# Accessibility Audit
+
+`DomainDigUITests` runs Apple's `performAccessibilityAudit()` across every
+primary screen. The audit checks contrast, hit-region size, clipped text at
+large Dynamic Type, element descriptions, trait correctness, and Dynamic Type
+support — the same ground the accessibility pass tracked in
+[issue #21](https://github.com/zerolabsco/domain-dig/issues/21) covers.
+
+## Findings are reported, not enforced
+
+The audit surfaces violations that exist today, so failing on all of them would
+block every unrelated change until the whole pass lands. Instead, findings are
+logged and attached to the result bundle tagged `[report]` or `[FAIL]`.
+
+Enforcement is the committed constant
+`AccessibilityAuditHarness.enforcedAuditTypes`. Widen it as each phase clears a
+category:
+
+| After phase | Enforce |
+| --- | --- |
+| 2 — semantic colors + light mode | `.contrast` |
+| 3 — Dynamic Type + reflow | `.textClipped`, `.dynamicType`, `.hitRegion` |
+| 4 — VoiceOver | `.elementDetection`, `.sufficientElementDescription`, `.trait` |
+
+A constant rather than a CI setting, for two reasons. Environment variables do
+not work: neither a plain `xcodebuild` env var nor a `TEST_RUNNER_`-prefixed
+build setting reaches the UI test process, so the toggle silently did nothing.
+And a committed value makes "when did contrast become enforced?" answerable with
+`git blame` instead of CI tribal knowledge.
+
+## Why coverage is split between local and CI
+
+**Audit coverage is not nested across OS versions.** Each runtime reports
+findings the others miss, in *both* directions. Measured on this project:
+
+| Screen | iOS 18.6 | iOS 27.0 |
+| --- | --- | --- |
+| Tracked Domains | 2 (text clipped) | **6** (+ contrast ×3, element detection) |
+| Settings | 2 contrast | **`dynamicType`** finding 18.6 missed |
+| Dashboard @ `AccessibilityXXXL` | **hit region** + 2 clipped | 1 clipped only |
+
+Neither runtime is a superset, so the oldest supported OS needs its own run.
+This also rules out committing per-screen baseline counts as a regression guard:
+no single number is correct on both.
+
+The catch is that **GitHub's `macos-26` image ships only iOS 26.x simulator
+runtimes.** It cannot test the 17.6 floor at all. A two-job CI matrix was tried
+and produced two near-identical 26.x runs at double the macOS minutes.
+
+So the work is split by what each side can uniquely do:
+
+| | Runtime | Uniquely provides |
+| --- | --- | --- |
+| **CI** (`.github/workflows/build.yml`) | newest available | A clean checkout of the merge result — catches a file that was never committed, which a local run cannot. Matters here because `DomainDig.xcodeproj` is hand-edited and uses file-system-synchronized groups, where a whole missing folder still builds locally. |
+| **Local** (`Scripts/audit-a11y.sh`) | oldest supported + newest | Real floor coverage, on a machine that actually has an 18.x runtime installed. |
+
+Together they cover both ends; neither duplicates the other.
+
+## Running it
+
+```sh
+./Scripts/audit-a11y.sh # floor + current
+./Scripts/audit-a11y.sh floor # oldest supported only (~85s)
+./Scripts/audit-a11y.sh current # newest installed only
+```
+
+The script reads the deployment target from the project rather than hard-coding
+it, and selects the oldest installed runtime **at or above** it — a runtime
+below the deployment target is useless, because the app cannot install there.
+If the nearest installed runtime is a major version above the target, it says
+so rather than implying floor coverage it does not have.
+
+### Pre-push hook
+
+```sh
+git config core.hooksPath .githooks
+```
+
+Runs the floor audit before a push, and only when Swift, asset, or project files
+changed. Bypass with `git push --no-verify`.
+
+Pre-push rather than pre-commit deliberately: the suite takes ~85s, and at
+pre-commit that blocks every commit. A hook routinely bypassed with
+`--no-verify` is worse than no hook, because it trains you to ignore it.
+
+## Notes
+
+- Audits retry up to three times. Slower machines can miss the audit's internal
+ deadline (`Audit failed to complete in time`, code `-56`), which is a tooling
+ timeout, not an app defect. A screen that still cannot be audited is reported
+ as an `XCTSkip`, never a pass — skips are visually distinct in CI, so an
+ unaudited screen stays visible instead of being silently counted as clean.
+- The suite launches with `DOMAIN_DIG_FORCE_PRO_PLUS` so Pro-gated screens are
+ reachable. `PurchaseService` honours that argument in `DEBUG` builds only.
+- Everything used is available at the iOS 17.6 deployment floor;
+ `performAccessibilityAudit` is `ios(17.0)`.