Summer Sale! All accounts are 50% off this week.

richardhulbert's avatar

Individual tests pass but not in series

Hi I am obviously missing something can anyone help?

<?php

namespace Tests\Feature\Integration;

use App\Models\User;
use Illuminate\Foundation\Testing\DatabaseTransactions;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Illuminate\Foundation\Testing\WithFaker;
use Spatie\Permission\Models\Permission;
use Spatie\Permission\Models\Role;
use Tests\TestCase;

class ViewsRoutesTest extends TestCase
{
    use DatabaseTransactions;

    /** @test */
    function route_is_guarded_and_redirects_to_login() {
        $response = $this->get('admin/view');
        // expect a redirect to login
        $response->assertStatus(302);
        $response->assertRedirect('/login');
    }

    /** @test */
    function views_route_when_logged_in_without_admin_rights() {
        $user = User::factory()->create();
        $this->actingAs($user);
        $response = $this->get('admin/view');
        $response->assertStatus(403);
    }
    

    /** @test */
    function index_route_when_logged_in_with_admin_rights() {
        $user = User::factory()->create();
        $role = Role::create(['name' => 'test_role']);
        $permission = Permission::create(['name' => 'see_admin']);
        $role->givePermissionTo($permission);
        $user->assignRole($role);
         $this->actingAs($user);
        $response = $this->get('/admin/view');
        $response->assertStatus(200);
    }
}

anyone of these passes but run in series the second test gets a 500 error somewhere

0 likes
11 replies
Nakov's avatar

Instead of use DatabaseTransactions; have you tried with use RefreshDatabase; trait? So it starts on a clean state always?

richardhulbert's avatar

@Nakov Hi Nakov - yes I did try that apart from being much slower. It fails in the same way.

Nakov's avatar

@richardhulbert much slower? on a single file? I bet is not taking more then a second :)

btw, share some error message so someone can help. Based on the test class nothing seems obvious.

richardhulbert's avatar

@Nakov Hi yes much slower - there several table but actually it is using the schema dump rather than migration files and for some reason that is taking a long time to execute - >15 seconds

I have a feeling that in fact it has something to do with this thread. https://laracasts.com/discuss/channels/general-discussion/codeception-tests-fail-when-run-together-but-pass-individually

Which for some reason I couldn't find initially. So If I understand the thread I need to set up all the models in the SetUp method of the test and things might work

fyi the error I get is:

 Expected response status code [403] but received 500.
richardhulbert's avatar

sadly :

class ViewsRoutesTest extends TestCase
{
    use DatabaseTransactions;
    private $user;
    public function setUp(): void {
        parent::setUp();
        $this->user = User::factory()->create();
    }

    /** @test */
    function route_is_guarded_and_redirects_to_login() {
        $response = $this->get('admin/view');
        // expect a redirect to login
        $response->assertStatus(302);
        $response->assertRedirect('/login');
    }

    /** @test */
    function views_route_when_logged_in_without_admin_rights() {
        $this->be($this->user);
        $response = $this->get('admin/view');
        $response->assertStatus(403);
    }

also fails with the same 500 error

This is really bonkers If I run 'views_route_when_logged_in_without_admin_rights' the test passes if I run the class 'ViewsRoutesTest' if fails WTF?

richardhulbert's avatar

@Talinon Yes! I am why? they are in php includes

include 'contenthead/contenthead.php';

and then in contented.php:

use App\Http\Controllers\PageController;
include_once 'dashboard.php';
include_once 'users.php';
include_once 'roles.php';
include_once 'permissions.php';
include_once 'content.php';
include_once 'contentTypes.php';
include_once 'gallery.php';
include_once 'pages.php';
include_once 'styles.php';
include_once 'views.php';
include_once 'branch.php';
// public facing
Route::get('routes', [PageController::class,'routes']);
Route::get('/pages/{id}',[PageController::class,'page']);
Route::get('/{any}', [PageController::class,'index'])->where('any', '^(?!api).*$');
richardhulbert's avatar
richardhulbert
OP
Best Answer
Level 6

Ha Fixed it! I need to run each call in a separate process.

adding:

 /**
     * @runInSeparateProcess
     * @test
     */

fixed it. Many thanks to all who looked at this problem

Talinon's avatar

@richardhulbert

I suspected as much. I think it's the way you are including your routes. You should register them within the app/providers/RouteServiceProvider

   Route::middleware('web')
                ->group(base_path('routes/myroute.php'));

Give that a try and see if your test suite passes in sequence

richardhulbert's avatar

@Talinon Hi I will try that but given that the routes work fine as is, what will I gain by registering them. Also if I do php artisan route:list I can see them all there!? Anyway I will try to understand what RouteServiceProvider does

richardhulbert's avatar

@Talinon to confirm adding the route in 'app/providers/RouteServiceProvider' doesn't seem to make any difference.

 protected function mapWebRoutes()
    {
        Route::middleware('web')
             ->namespace($this->namespace)
             ->group(base_path('routes/web.php'));
        Route::middleware('web')
             ->namespace($this->namespace)
             ->group(base_path('routes/contenthead/views.php'));
    }

after all, surely as the file structure of the route files is really just includes so at a php level web.php has all the routes included. Indeed if I php artisan route:list -v they are all there.

Interestingly now with the addition of /** @runInSeparateProcess */ , php artisan test /Users/richard/PhpstormProjects/vanilla/tests/Feature/Integration/ViewsRoutesTest.php works fine and yet when I run php artisan test all tests in ViewsRoutesTest.php fail. If I run phpunit all tests pass.

smell of a bug perhaps?

Please or to participate in this conversation.