Types and Tests: Not Rivals, Just Working Different Shifts

A common source of confusion in PHP development is the relationship between static analysis and unit testing. Are they redundant? Does adding types mean fewer tests? Does a comprehensive test suite make a type checker unnecessary?

No to all three. They cover different grounds. The interesting question is where exactly the boundary lies, and what happens at the edges.

In this blog post, we’ll use a prime number checker as our canvas. Simple enough to hold in your head, rich enough to expose all three zones: what types cover, what tests cover, and the narrow overlap where they touch.

Starting naive

<?php

function isPrime($n): bool
{
      // No need to comment on this part of the code
    for ($i = 2; $i < $n; $i++) {
        if ($n % $i === 0) {
            return false;
        }
    }
    return true;
}

?>

Here, there are no type declarations. From the static code analyser mago’s perspective, the incoming argument $n is of type mixed: a string, null, an array, anything. The analyser can barely reason about it. From a testing perspective, you can still write tests for such values, but you’re also responsible for testing inputs that should never have been passed in the first place.

More importantly, both tools have to work harder to cover ground that neither should cover. And there are correctness bugs too: isPrime(1) returns true, isPrime(0) loops forever, the $i < $n bound is may be easily optimized with, at least, a sqrt().

The type system takes the first shift

Our first step is to add types. It is not just courtesy, it’s a contract:

<?php

function isPrime(int $n): bool
{
    if ($n < 2) {
        return false;
    }

    for ($i = 2; $i * $i <= $n; $i++) {
        if ($n % $i === 0) {
            return false;
        }
    }

    return true;
}

?>

Run mago on callers:

<?php

isPrime("hello");  // error: argument of type 'string' is not assignable to parameter of type 'int'
isPrime(null);     // error: argument of type 'null' is not assignable to parameter of type 'int'
isPrime(3.14);     // error: argument of type 'float' is not assignable to parameter of type 'int'

?>

Three entire categories of wrong input, caught before the test suite even runs. You do not need tests for isPrime("hello"): the type checker owns that case permanently, at zero runtime cost.

But here is where PHP’s type system hits a hard ceiling: int represents a signed integer. There is no native positive-int type in PHP 8.x. isPrime(-7) passes the type check without complaint, even though a negative number as input to a primality test is semantically nonsensical.

The type system has done everything it can. It cannot express value-level constraints like “greater than one” with the native scalar PHP types.

It is worth nothing that static analysers may have a feature for that particular case, using PHPdoc types. We’ll keep that on the side here, as it makes the explanations less clear.

Tests pick up the slack

When the type system hits its limits, it is where unit tests step in. A data provider covers the value-level cases that types cannot:

<?php

final class IsPrimeTest extends TestCase
{
    #[DataProvider('cases')]
    public function testIsPrime(int $n, bool $expected): void
    {
        $this->assertSame($expected, isPrime($n));
    }

    public static function cases(): iterable
    {
        yield 'negative'    => [-7,     false];
        yield 'zero'        => [0,      false];
        yield 'one'         => [1,      false];
        yield 'two'         => [2,      true];
        yield 'three'       => [3,      true];
        yield 'four'        => [4,      false];
        yield 'large prime' => [104729, true];
    }
}

?>

The type int already guarantees these are integers. The tests guarantee they behave correctly as values. That is a different job entirely.

two is worth calling out: it is the only even prime, and a naive loop starting at 2 immediately divides it evenly and returns false. A type checker has no way to know the algorithm is wrong for a specific value. Only a test can catch that. Any edge cases, or values that are close to limits or strange situations must be included in the tests.

The overlap zone

Look at the negative test case. When you write isPrime(-7) and assert false, you are making a statement that could have been a type constraint. If PHP had a native positive-int type, or if you add a PHPDoc annotation that mago understands:

<?php

/** @param positive-int $n */
function isPrime(int $n): bool { ... }

?>

Then passing -7 becomes a static analysis error, and the negative test case is now testing something the type system already forbids. The test is redundant from a type perspective: though you might keep it as behavioral documentation, or remove it as noise.

This is the overlap: a narrow zone where a test and a type constraint are answering the same question. It is not large, but it is where the most interesting design decisions live. When you find yourself testing a value the type system could have forbidden, ask: should this be a type constraint instead?

Large numbers: when int runs out

PHP’s int is 64 bits on modern platforms: PHP_INT_MAX = 9223372036854775807. For testing Mersenne primes or any serious number theory, that is nowhere near enough.

Pass PHP_INT_MAX + 1 to your function and PHP silently promotes to float, losing precision. The type declaration says int, but you are no longer sending integers: PHP now stops with a type error.

One option is to move from int to GMP:

<?php

function isPrime(\GMP $n): bool
{
    if (gmp_cmp($n, gmp_init(2)) < 0) {
        return false;
    }

    return gmp_prob_prime($n) !== 0;
}

?>

\GMP is a proper object type. Mago can reason about it fully. The type system is back in business, and you can now test at arbitrary scale:

<?php

yield 'M127' => [gmp_init('170141183460469231731687303715884105727'), true];

?>

But notice: \GMP is still a signed numeric type. gmp_init(-7) is perfectly valid. The negative case has not gone away: it has just moved to a different type. Tests still need to cover it. And, in particular, there is not more PHPdoc /** @param positive-gmp $n */, unless static analysis tools read this, and add it (and if they do, I’ll rewrite the article with BCmath).

Switching representation gives you back the type system. It does not eliminate value-level concerns.

int|\GMP: union types as a complexity multiplier

The natural instinct when refactoring an existing codebase is to accept both types and let callers choose:

<?php

function isPrime(int|\GMP $n): bool
{
    $value = match (true) {
        $n instanceof \GMP => $n,
        default            => gmp_init($n),
    };

    if (gmp_cmp($value, gmp_init(2)) < 0) {
        return false;
    }

    return gmp_prob_prime($value) !== 0;
}

?>

Mago tracks both branches of the union and warn you if you forget to handle one. That is real value. But look at the cost: every caller now has a decision to make. Every test case needs to decide which representation to use. The implementation carries an extra branch. And the negative case now manifests twice: once for int inputs and once for \GMP inputs.

Union types increase expressiveness but transfer complexity to every call site. Sometimes the right call is to draw a hard boundary: accept only \GMP, and let callers convert with gmp_init(). The type becomes simpler, the implementation becomes simpler, and conversion happens at one explicit point.

int|\GMP is not wrong. It is a deliberate tradeoff. The type system gained flexibility; the code gained surface area. That bargain is not always worth taking.

The shape of the boundary

Types are a permanent, zero-runtime-cost constraint on shape, or category of data. Tests are executable specifications on behavior. Neither replaces the other.

The first question is to check if the types are covering exactly the domain values. Here, int are not, and, to some extend, GMP neither. That means tests have to step in and make sure anything left over by types are under control. Usually, this boils down to values, which more general behavior goes to types.

The prime number checker was trivial, yet it turned out to be tricky. The same problem play out in every codebase. Every contraints that can be linked to the type, can be removed from the tests. Otherwise, it must go to the tests. And between tests and types, in the gray zone, sometimes, you need checks from everyone.