README.md

Native mocks versus PHPUnit mocks

Mock type	Tests duration
Native mocks	160.10 seconds
Mockery	177.01 seconds (+10%)
PHPUnit mocks	232.29 seconds (+45%)
Prophecy	3868.80 seconds (+2416%)

Benchmarked on the exact same Windows machine, with PHP 7.3.7, PHPUnit 8.2.5, inside a Docker container, testing consisting on executing the same test 400k times.

I did not show native classes here, as we only focus on PHPUnit's system.

Benchmarks:

Native anonymous class (base time)

$repo = new class() extends SubscriptionRepository {
    public $executions = [];
    public function __construct()
    {
        // Don't call parent constructor in case it needs dependencies
    }
    public function hasSimilarActiveSubscriptions(Subscription $subscription): bool
    {
        // Only mock methods that have to be used during the test
        $this->executions[] = $subscription;
        return true;
    }
};

// Tests ...

static::assertCount(1, $repo->executions);
static::assertSame($subscription, $repo->executions[0]);

Mockery (10% slower)

$repo = Mockery::mock(SubscriptionRepository::class);
$repo
    ->shouldReceive('hasSimilarActiveSubscriptions')
    ->with($subscription)
    ->once()
    ->andReturn(true)
;

// Tests ...

PHPUnit (45% slower)

$repo = $this->createMock(SubscriptionRepository::class);
$repo->expects(static::once())
    ->method('hasSimilarActiveSubscriptions')
    ->with($subscription)
    ->willReturn(true)
;

// Tests ...

Prophecy (2416% slower)

$repo = $this->prophesize(SubscriptionRepository::class);
$repo
    ->hasSimilarActiveSubscriptions($subscription)
    ->shouldBeCalledOnce()
    ->willReturn(true)
;

// Don't forget that "$repo" must be injected using "$repo->reveal()"

// Tests ...

I think it is worth mentioning that I could only run Prophecy tests with batches of 25k tests instead of 400k. More was leading to segfaults. Therefore, the time is an average (but significantly slower anyway).

@oleg-andreyev on most projects it doesn't matter. Would be good to get to the bottom of it and improve performance/memory usage though.

@jakzal you are right that in most cases it does not matter, but when you have a large set of tests it matters. As you may know recently Symfony moved from Phpunit mocks to anonymous classes were it was possible and build become a bit faster. It great that @Pierstoval gathered this information now community can help and improve it.

Well on some projects I have hundreds of tests, and if they run in 5 minutes, I'd be perfectly okay for them to be run in 3 minutes instead by giving them a perf boost just by changing the mocking system.

I'm going to open a PR on a personal project on which I have ~400 tests, change all createMock() calls to native implementations (if possible) and check the performance differences with the PHPUnit's --repeat=... option, but this will be in the future when all tests will be refactored 😛

And by the way, thanks @jakzal for providing more benchmarks that confirm the initial theory 👍