Faster iOS Merges with Selective Testing and Jev

Tuist Test Selection is a good starting point. Running the tests affected by a change is already a substantial improvement over running every test on every merge. Its selection works at the target level: changes to a shared dependency can select tests in its consumers.

When a module exposes one, two, or three abstractions and has a small set of consumers, that can be precise enough. A change might select five test targets instead of fifty. That is a useful saving before adding any classifier.

The harder case is a shared framework containing many components. Almost every screen imports our design system, but each screen uses only part of it. Adding a text-field component selected 40 test schemes. With the classifier enabled, two ran. The other 38 had no reachable use of the new component.

The dependency graph knew which modules imported the design system. It did not know which parts they used.

More Branches, More Merges

Agentic coding makes this gap more expensive. Consider a solo developer going from two or three merges a day to ten, fifteen, or twenty. Several agents can work on separate branches at once. Each finished branch still has to integrate with main. A team has the same problem with more contributors.

Two branches can pass their checks against the same version of main and still fail when combined. One changes an API; another adds a caller using the old API. Neither has to conflict in Git for the resulting code to break.

I prefer to merge one validated change at a time, checking it against the current destination branch. A merge queue can also validate batches or prepare later checks speculatively, provided it tests the combined revision it will land. Either way, integration checks become a shared bottleneck when branches finish faster than they can land.

Shorter checks mean less waiting and less time for finished branches to drift from main. Avoiding unnecessary tests matters more when the same check runs twenty times a day.

I added a second step to our merge checks: inspect the actual change before deciding which dependent schemes need to run. Source analysis handles cases where it finds no reachable use. Jev handles cases that need a judgment about behavior. Unresolved cases keep their tests.

The Starting Point

I use Worktrunk for the one-worktree, one-simulator setup. Before merging a branch, its pre-merge hook runs lint and affected tests on that worktree’s simulator:

AFFECTED_TESTS=1 JEV_IMPACT_APPLY=1 mise run tuist:test

Our starting selection comes from a local script that maps changed files to Tuist targets and follows their dependents. It selects from the Git diff, rather than Tuist Test Selection’s hashes of previously successful test runs. Both operate at target granularity. A test-only change stays within its own target. A production change selects tests in the changed target and its dependency consumers. Public-target changes also build the app to compile those consumers.

This already avoids testing unrelated modules. But it still over-selects when a shared module changes.

Design system adds a component.
Home imports the design system.
The graph selects Home's tests.
Home never uses the new component.

The graph gives us the candidates. The classifier decides whether the exact change can reach each candidate.

Splitting the shared framework is another option. Smaller targets help when they separate independent responsibilities and give consumers narrower dependencies. But the number of exported types alone does not tell us whether a boundary is useful. In A Philosophy of Software Design, John Ousterhout argues for modules that hide substantial complexity behind simple interfaces, and warns against splitting functionality into many shallow classes. That supports choosing boundaries for what they hide and how callers use them. It does not prescribe the number of Swift targets. His modular-design notes explain the distinction.

The classifier gives us another way to refine test selection within those boundaries. I have not benchmarked it against splitting the design system or generated protobuf code into smaller targets.

Where the Rules Live

The classifier does not replace the dependency selector. It receives its selection and can remove schemes from it. Lint and the required app build still run.

First, Keep the Tests We Know We Need

Two rules run before any model judgment:

• If the candidate scheme belongs to a module changed in this branch, keep it.
• If test infrastructure changed and the selector requested a full suite, keep that full suite.

There is no useful reason to ask a model whether a changed module should test itself. The graph and file paths already answer that.

The same applies to changes in the test runner, Tuist setup, or other infrastructure that can affect how tests are built and executed. Those changes need the broad check.

Then, Find Who Uses the Changed Code

For each changed-module and candidate-scheme pair, the classifier splits the diff into changes. It keeps additions separate from edits to existing behavior and identifies the symbols involved.

It searches the source at the branch’s commit, along the dependency paths to the candidate. An import alone is not a use site. A reference to the changed type, function, or asset is.

For a modified file, it also checks public entry points. If the change sits behind an internal wrapper, it follows one wrapper and looks for consumers of its public symbols. This catches cases where another module uses the wrapper rather than naming the implementation directly.

If the supported analysis finds no reachable use, the change is marked unlikely to affect that candidate. No API call is needed.

This is why the new text field works so well. Adding a component does not change the components other screens already use. Selecting every importing screen buys little when none of them references the addition.

There are limits. An unidentified symbol, missing source context, an unresolved wrapper, or dynamic asset lookup keeps the case unresolved. The classifier does not turn a failed search into permission to skip.

Where Jev Helps

A use site does not necessarily mean a behavior change. A consumer can use a type without depending on the part that changed. Establishing that with source searches alone quickly becomes a much larger program-analysis problem.

For these cases, Jev receives:

• the exact diff and changed symbols,
• short source excerpts around their use sites,
• any internal wrapper connecting the change to those sites,
• candidate test-file names and directly relevant test excerpts.

It answers two separate questions:

1. Can this diff change behavior at the shown use sites?
2. If behavior changes, could this scheme’s tests or snapshots observe it?

The API returns probabilities for these yes/no questions. We skip only on the first: behavior-impact probability must be at most 0.10. The test-observation answer is recorded for diagnosis. It cannot authorize a skip. Missing a direct reference in a test file does not establish that indirect coverage is absent.

The useful part is having an explicit uncertainty value at the point where deterministic analysis stops. We can keep uncertain cases instead of forcing every result into “affected” or “unaffected.” The number is still a model judgment, not a proof.

Every Trigger Must Agree

A scheme can be selected by several changed modules. One harmless change cannot excuse another change that might affect it.

The final rule, simplified:

if the scheme's module changed:
    run it
elif any trigger is unresolved or may affect behavior:
    run it
elif every trigger has no reachable use or behavior probability <= 0.10:
    skip it
else:
    run it

An API failure keeps the original selection. So does missing context. Each merge has a budget of 12 Jev requests; candidates left after that budget are tested. Large inputs are split into bounded requests, and all of their results must support the low-impact decision.

This makes the failure direction straightforward: less certainty means more testing.

To investigate a decision, I can run the graph-selected schemes without applying the classifier:

JEV_IMPACT_APPLY=0 wt merge

What Changed in Practice

As of 8 October, the filtered post-classifier sample contains 61 completed local merges. The dependency selector chose 1,128 scheme runs. We executed 779 and skipped 349: 30.9% fewer scheme runs. Eleven merges had skips.

A scheme is a module-level build-and-test command, not one test case. Avoiding a scheme also avoids starting its command and whatever build work it would need.

Some actual selections:

The reduction is concentrated in changes with many dependency consumers. Among merges selecting at least 36 schemes, we skipped 347 of 724 possible runs: 47.9%.

Most skips so far came from local source analysis. Jev judgments allowed two additional scheme skips. These results describe the combined approach; they do not establish a large speedup from the model alone.

Measuring Time Without Pretending the Workloads Match

The merge hook saves the diff, selection, executed and skipped schemes, skip reasons, classifier duration, and each scheme’s execution time. A successful post-merge hook marks the session as merged. Failed attempts remain distinguishable from completed merges.

For the comparison, I excluded full-suite infrastructure changes, shared-test-utility changes that selected all or almost all schemes, zero-scheme changes, explicitly skipped tests, and failed or unmerged attempts. The before period is 15–23 September, using the same underlying dependency selector. The after period starts with classifier activation on 24 September.

The raw median still went up: 2.40 minutes before, 5.97 minutes after. Median graph-selected schemes also went from two to ten. The later changes were larger even after excluding infrastructure. Those medians do not show the classifier’s effect on equivalent work.

Instead, I estimated the cost of the skipped schemes using each scheme’s median observed execution time:

estimated check without classification
    = observed check
    + estimated execution time of skipped schemes
    - classifier overhead

The timing samples cover every skipped scheme: 1,268 successful scheme runs from 72 completed classifier-applied merges, including full suites. Across the 61 eligible merges, checks took 505.1 minutes. The estimate without classification is 637.5 minutes. That gives 132.4 minutes saved, or 20.8%, after charging classifier overhead on every merge, including those with no skips.

The avoided scheme runs are measured. The time saving is estimated. Build-cache reuse and execution order can change the cost of a skipped scheme. This is also a comparison of successful checks, not total time spent on retries.

The Remaining Check

I have not rerun the skipped suites to measure missed failures. A useful next comparison is the same revisions with classification enabled and disabled, including the tests we chose to skip. That would give us both timing and false-negative evidence.

The integration also uses jev-latest. Recording the resolved model version would make future comparisons easier to reproduce.

In the text-field case, inspecting the changed symbols reduced the selection from 40 schemes to two. Jev gives us a way to judge the remaining semantic cases. Unknown cases continue through the existing test runner.