Be part of JetBrains PHPverse 2026 on June 9 – a free online event bringing PHP devs worldwide together.

mstnorris's avatar

Very slow PHPUnit tests using PHP7.2 or PHP7.1.

If I run my tests using PHP7.2 or PHP7.1 they are about 3x slower than if I run them using PHP7.0. Is there anyway to get to the bottom of why this is happening?

I'm using Laravel 5.5.20 and Laravel Homestead 7.0.1. I have 47 rather simple tests, some hitting the database, others just simple assertions; so there isn't anything that should take ages.

I installed johnkary/phpunit-speedtrap to see which tests take the longest so I could remove those but there isn't a specific test that takes a long time because if I remove the offending test, the next one will take ages (see below).

First Run                Second Run
Test A    0.2 sec        Test A    0.2 sec
Test B.   0.3 sec        Test B.   0.3 sec
Test C    0.1 sec        Test C    0.1 sec
Test D    0.1 sec        Test D    0.1 sec
Test E    9.3 sec        REMOVED Test E
Test F    0.3 sec        Test F    9.3 sec <-- Test F now takes ages
Test G    0.2 sec        Test G    0.2 sec

I am also using an in-memory SQLite3 database, with the Laravel CreatesApplication and RefreshDatabase trait as I want each test to run independently.

I do not have Xdebug installed or running. Is there something known that PHP7.1 and PHP7.2 take a long time to run PHPUnit tests? Is there something else I can install (or even run it with Xdebug) to track down what exactly it is that is causing the issue?

Setup

Laravel 5.5.20
Laravel Homestead 7.0.1 (Per-project installation)
PHPUnit 6.4.4
Vagrant 2.0.1
Virtualbox 5.2.4

Results

PHP 7.2 PHPUnit 6.4.4
Time: 12.4 seconds, Memory: 162.00MB

PHP 7.1 PHPUnit 6.4.4
Time: 12.19 seconds, Memory: 162.00MB

PHP 7.0 PHPUnit 6.4.4
Time: 4.88 seconds, Memory: 162.00MB
0 likes
33 replies
MikeHopley's avatar

What happens if you remove Test D? I'm wondering whether that test is somehow causing expensive setup for the following test.

One general suggestion:

You can split your tests into separate suites, such as unit and feature. The unit suite would be used for tests that do not hit a database, so you can dispense with database setup/teardown.

As well as making your tests faster, it can be helpful when tests fail. If your unit suite passes but your feature suite has many failures, then you immediately know it's a database issue.

mstnorris's avatar

Thanks, all my tests are split up into various different groups and suites so I can run a smaller subset. This only happens when they are run together and I as you suggested (which I have done many times) I can remove the offending test, and/or the test that's run before it, and unfortunately it makes no difference.

1 like
MikeHopley's avatar

Interesting.

Does that mean you only get the slow-down when you are running all the suites together? What about if you run each suite separately?

It's very odd. :(

mstnorris's avatar

I get the slow down when I run them (the test suites) separately too. The issue does appear to be with PHP7.1 and PHP7.2 as the tests run very fast when using PHP7.0.

If I run the tests individually however, they do run pretty quickly regardless of the version of PHP.

MikeHopley's avatar

Presumably that means you're getting slow-down even when just running your unit suite, which I assume does not involve any database setup / teardown.

If so, at least you've eliminated that possibility.

No idea why the tests would be running slower on newer PHP, sorry. :(

shez1983's avatar

what if you re-arranged the tests so the first few happen afterwards? or run that test on its own to see how long it takes in isolation?

i know pretty stupid suggestions but it is truly baffling..

ONe other trick that i always do now is have a mysql testing db - which is seeded and in my tests i use TransactionTrait so each test DOES run in isolation... without having to refresh db in each test..

mstnorris's avatar

Rearranging the tests doesn't matter as it isn't a specific test that takes the time, it is always let's say the 10th one to run. I will dig a little deeper and check that this is the case all the time and that it really doesn't matter what the order is.

I have run the tests in isolation and independently and they're very fast. The issue is only when they are run together.

I have tested this with MySQL as well as the in-memory SQLite3 database and this doesn't have much of an impact on the speed.

As I mentioned, I am using the new RefreshDatabase trait, which as I understand essentially does the same thing as the DatabaseMigrations and DatabaseTransactions did previously (I know they are still available).

Having looked at the RefreshDatabase trait

protected function usingInMemoryDatabase()
{
    return config('database.connections')[
        config('database.default')
    ]['database'] == ':memory:';
}

Could it be that it doesn't recognise that I am using an in-memory database as I am defining the config values within the phpunit.xml file itself, rather than in the database.php config file. And therefore not doing as I expect?

MikeHopley's avatar

Could it be that it doesn't recognise that I am using an in-memory database as I am defining the config values within the phpunit.xml file itself, rather than in the database.php config file. And therefore not doing as I expect?

I think you are correct there. As you can see from the code, Laravel is checking its config files and not inspecting PHPunit's config.

jplhomer's avatar

@mstnorris I'm currently experiencing the same issue! Pulling my hair out, and I've tried so many things: sqlite/mysql, withholding certain tests, trying in-memory vs in-database storage. Tests always hang on the ~25th test for me, and the hanging test takes at least 30 seconds to complete. Skipping a test just causes the next one to hang.

............................                                      28 / 28 (100%)

You should really fix these slow tests (>500ms)...
 1. 36563ms to run Tests\Unit\VacationRequestTest:test_a_request_approved_scope_works
 2. 815ms to run Tests\Feature\CreateVacationRequestTest:test_notification_is_sent_when_vacation_requested


Time: 38.85 seconds, Memory: 296.00MB

OK (28 tests, 53 assertions)

I'm running:

Laravel v5.5.21
macOS High Sierra
PHPUnit 6.5.5
PHP 7.1.3

I've noticed that my php shoots to 100% CPU usage, and that the memory used is extremely high when the tests run long.

Experiencing this same issue when running it in a Docker container with PHP 7.2

1 like
mstnorris's avatar

@jplhomer thanks for your reply. I'm glad I'm not the only one.

I also asked the question on Stack Overflow, if you wouldn't mind leaving a reply there too, giving weight to the fact that it isn't just my setup that would be great.

https://stackoverflow.com/questions/48249205/phpunit-tests-run-using-php7-2-and-7-1-are-3x-slower-than-when-run-using-php7-0

I'll write up some more notes over the coming days as I'd really like to get to the bottom of it.

1 like
shez1983's avatar

could it be something you are doing in TEstCASE that is run for evey test? that spikes up teh usage/RAM and thus slowing down the test? but then again the stats you showed didnt indicate Laravel/phpunit using more RAM..

mstnorris's avatar

@shez1983 I'm using the RefreshDatabase trait so it resets the database after each test.

I can also confirm that running the tests does cause a spike in CPU usage (over 100%), but RAM is not affected. This is however for all three versions (7.0, 7.1, and 7.2) so it doesn't appear to make a difference. PHP7.0 is still approx 3x as fast with the current tests that I have.

dmhamilt's avatar

Have you tried turning off broadcasting?

This does not explain everything, but helped me trim off seconds for me.

in .env add/change BROADCAST_DRIVER=log

shez1983's avatar

can you turn of refreshDatabase and see what happens then? can you also install valet or test this on a server to see if you get same results there?

mstnorris's avatar

@shez1983 if I turn off RefreshDatabase then my tests will fail as each is written/expected to run separately.

I haven't used Valet however if I do run the tests from my host machine - which I think would be the same as running them using Valet. (FYI macOS Sierra 10.12.6 with PHP7.1 installed) then I have the same issue.

@dmhamilt I don't see how that would help, and anyway, it is set to log by default.

arukomp's avatar

I got my tests to run 2-3 times faster by uninstalling xdebug extension

$ brew remove php71-xdebug
1 like
Cronix's avatar

@arukomp that makes sense. Composer takes a really long time when xdebug is enabled as well. Good find!

shez1983's avatar

you can use transaction trait instead - try that..

arukomp's avatar

here's something for switching xdebug on/off if you've installed PHP through Brew:

/usr/local/bin/enable-xdebug

#!/bin/sh

sed -i.default "s/^;zend_extension=/zend_extension=/" /usr/local/etc/php/7.1/conf.d/ext-xdebug.ini

launchctl unload /Library/LaunchDaemons/homebrew.mxcl.php71.plist
launchctl load /Library/LaunchDaemons/homebrew.mxcl.php71.plist

echo "xdebug enabled"

/usr/local/bin/disable-xdebug

#!/bin/sh

sed -i.default "s/^zend_extension=/;zend_extension=/" /usr/local/etc/php/7.1/conf.d/ext-xdebug.ini

launchctl unload /Library/LaunchDaemons/homebrew.mxcl.php71.plist
launchctl load /Library/LaunchDaemons/homebrew.mxcl.php71.plist

echo "xdebug disabled"

don't forget to chmod +x /usr/local/bin/*able-xdebug

$ enable-xdebug
$ disable-xdebug

if brew's PHP is running under the root user, don't forget to add sudo before these commands

credit: https://gist.github.com/hacfi/5a9e49fff3c9b67eb793

girol's avatar

Did you try to clear your config cache? Minutes ago I"ve experienced the same issue and clearing the cache solved it. I'm using PHP 7.2 and Laravel 5.5.

1 like
nexxai's avatar

For those who are running PHP 7.2, I had to do this to remove xdebug:

pecl uninstall xdebug

Then I had to edit my php.ini file and remove the two xdebug.so lines. Once I did that, the speed of my test suite went from ~25 seconds down to 8.

1 like
amadeann's avatar

In my case disabling Xdebug sped up tests by 50%. Since I am using Homestead, all I need to do it:

xoff
phpunit
xon

When I run a full test suite.

Roni's avatar

Don't remove x-debug if you need it, thats nuts, just disable it and any other extraneous php_ini add-ons while in unit test. add -n to your flags

here are some aliases to help out from my .bash_profile if you run your commands in the terminal.

alias c='clear'

#TESTING PHPUNIT
alias phpspec='c;vendor/bin/phpspec'
alias phpunit='c;vendor/bin/phpunit'
alias p='c;php -n vendor/bin/phpunit --exclude-group external'
alias pf='c;php -n vendor/bin/phpunit --exclude-group external --filter '
alias pext='c;php -n vendor/bin/phpunit'
alias pextf='c;php -n vendor/bin/phpunit --filter '


Use the advice above, I extend DBTestCase a file with RefreshDatabases Trait and a bunch of other macros and hacks I need for testing features, for unit tests just extent regular test case so you don't run migrations when you don't need to.

Add an annotation like @external to tests that can't be rushed, like talking to an external payment gateway when you aren't mocking one out or any external service you can't rush like a fax server etc.. Once you have those running, test them when you need but not when you are doing TDD.

p or pf will run 3-5x faster that phpunit.

if you are using phpstorm, in your Debug Configuration under command line / Interpreter Options: add your -n there too and it should fly again.

HTH Cheers

2 likes
amadeann's avatar

@Roni Super useful info. In my case though I'm struggling to get the tests running with -n flag.

I had to do this:

php -n -d extension=dom.so -d extension=mbstring.so -d extension=json.so -d extension=pdo.so -d extension=pdo_sqlite.so vendor/bin/phpunit

to start with, and got into this error:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /home/vagrant/code/myproject/vendor/laravel/framework/src/Illuminate/Database/Schema/Blueprint.php on line 1251

somewhere in the middle of my test suite.

Tests seem to be running faster, but it looks like a lot of my tests depend on specific extensions being enabled. But disabling XDebug for the duration of tests (with xon/xoff in Homestead is a huge improvement anyway.

Roni's avatar

What I prefer to do is add another group for example @needsPHPINI and test it separately. the -n flag removes the php_ini add ons and though you can enable them selectively, it can slow you down.

Then for my TDD cycles I just exclude the groups, unless i'm working directly on that item, and then i typically run only the one test or one class to keep it clean and fast.

I don't have my coding laptop with me but I realized I didn't leave an annotation example

here is a link from a quick google search on stack exchange

https://stackoverflow.com/questions/22180186/testing-levels-groups-in-phpunit

but basically


/** @test @needsPHPINI */
public function a_special_test() {
        // presumably doing something that has some special ini file need   
        $this->get('/')
            ->assertOk();

    }

Looking at that error it also looks like it's in a migration... I haven't done it yet, but i have a project I'm going to try it on shortly. Where instead of migrating the whole database I'd choose chunks of it, or set up a testing DB and then use a transaction based approach. Especially for an upgrade project with no tests. It may be faster, but someone with more experience should weigh in on that. But using those approaches may also vastly reduce your set up and tear down times.

1 like
tykus's avatar

I am also experiencing this same issue at the moment.

My setup is Laravel 5.6 / PHP 7.2. I mostly have feature tests which use a MySQL database and the DatabaseTransactions trait. The tests will individually run quite quickly, but immediately there are more than ~25 examples, that ~25th test will stall for about 30 seconds and CPU usage for the process jumps to 100%.

I have tried (i) different versions and combinations of PHP and PHPUnit (ii) disabling Xdebug (iii) running within a Homestead and directly on my Mac host (iv) running on a colleague's machine (v) forced the most basic test into that position, e.g. assertTrue(true).

Every attempt has the same result; still trying to find a solution...

Edit I have implemented this solution in a tearDown method on the TestCase class. It appears to be working, but I don't understand why? By my reckoning, the properties that are being unset are being taken care of in the parent::tearDown() method anyway.

Roni's avatar

@tykus,Thanks for the link, I'm going to add it to my base class tomorrow and see what happens. However, I took compsci / engineering 20 years ago so my c++/Java is significantly dated (haven't used it for 14 years for almost anything), but I do remember the same thing happening in Java years ago.

If you left garbage collection up to the virtual machine it may not choose optimal times to run. Or that was the case a LONG time ago. Now, if you implemented your own garbage collection, an anti-pattern back then it could in some situations make a pretty drastic difference.

I'd imagine under the hood in some situations, especially where you might be resource constrained in some fashion, active garbage collection might free up some ram or high speed cache at some threshold, where previously had you starting to use a low speed resource.

Just old dude ramblings but I'd be curios to find out more. Some of my test suites are make a cup of coffee actions, and I'd love to cut those down.

iraklisg's avatar

In my experience, the process of migrating the testing database is the one that takes the most of the time and also consumes most of my system's resources.

My scenario includes a long and complex database (which comprises multiple schemas and tables) that needs to be migrated in order to perform my tests. Disabling php-xdebug extension didn't help so much since the bottleneck is the database migrations ()

What I do, in order to speed up my test suite is the following:

  • I stop using DatabaseMigrations trait, since it forces my database to be refreshed each time I run a test.

  • I migrate my testing database once before making any test (the testing database will contain all necessary tables without data)

php artisan migrate:refresh
  • I created a custom clearDatabases method within Tests\TestCase as follows:
/**
 * Truncates all tables of given connections.
 * @param array $connections
 */
protected function clearDatabases(array $connections): void
{
    foreach ($connections as $connection) {
        $tables = DB::connection($connection)->getDoctrineSchemaManager()->listTableNames();
        DB::connection($connection)->statement('SET FOREIGN_KEY_CHECKS=0');
        foreach ($tables as $table) {
            if ('migrations' == $table) {
                continue;
            }
            DB::connection($connection)->table($table)->truncate();
        }
        DB::connection($connection)->statement('SET FOREIGN_KEY_CHECKS=1');
    }
}

The method accepts an array of the connections as an argument. Each connection corresponds to a different mysql schema. The method finds all tables related to that connection and truncates their data.

  • I explicitly call clearDatabases from my test'ssetUp method by passing the required connections ($schemas) required for this particular test

class ApplicationFooTest extends TestCase
{
    protected function setUp()
    {
        parent::setUp();

        $this->clearDatabases([DB_CONNECTION_FOO')]);
    }
}

That is, before running each of my tests, I use the database that I have already migrated and clear any data

The advantages of the proposed solution is that I was able to dramatically improve the time needed for my test suite to run by aprox -96%

The main disadvantage is that I have to manually refresh my testing database each time I add a new migrations (still, this is not a big problem for me, since adding new migrations is something that I don't do all the time when testing )

zanematthew's avatar

Confirming that just loading the needed drives is almost 50% faster, see below for detail:

Loading the default ini.

/var/www/html/current # vendor/bin/phpunit
PHPUnit 7.2.7 by Sebastian Bergmann and contributors.

.EEER....................EE......EFEFF.........................  63 / 105 ( 60%)
........................................RR                      105 / 105 (100%)

Time: 1.87 minutes, Memory: 46.00 MB

Loading only the PDO Mysql extension

/var/www/html/current # php -n -d extension=pdo_mysql.so vendor/bin/phpunit
PHPUnit 7.2.7 by Sebastian Bergmann and contributors.

.EEER....................EE......EFEFF.........................  63 / 105 ( 60%)
........................................RR                      105 / 105 (100%)

Time: 41.07 seconds, Memory: 40.00 MB

At some point I'll cleanup my test, but this is pretty neat.

Next

Please or to participate in this conversation.