alchermd's avatar

How do I test login throttling?

Hello!

I've built a customized authentication system that isn't that much different from what the make:auth scaffolds for us, and have recently added throttling via the ThrottlesLogins trait. I have confirmed with manual usage that it does indeed throttle the login attempts, but have yet to find a way to write a concise test for it. What I currently have:

/** @test */
public function multiple_invalid_login_attempts_are_throttled()
{
    $admin = $this->createAdminUser();

    $response = null;
    for ($i = 0; $i < 100; ++$i) {
        $response = $this->post(route('admin.authenticate'), [
            'email' => $admin->email,
            'password' => 'secret',
        ]);
    }

    $response->assertStatus(429); // Also tried asserting for headers and session errors, but no luck
}

What really trips me up is that I have this code block for throttling in my controller that isn't being triggered even after that many login attempts:

public function store(Request $request)
{
    if ($this->hasTooManyLoginAttempts($request)) {
        dd('hola!'); // This dd() should be triggered when running the test, but it does not ...
        $this->fireLockoutEvent($request);
        $this->sendLockoutResponse($request);
    }

    // ...
}

Any idea where should I look at?

0 likes
13 replies
D9705996's avatar

Also you wouldn't get to the store method if the throttle middleware has prevented the login so this would never be called.

if ($this->hasTooManyLoginAttempts($request)) {
Drfraker's avatar

Let's say your requirements are to lockout after 5 attempts in a minute time period.

-First test by adding 5 separate login attempts and assert that they are successfully redirected back to the login page without the lockout and then a final login attempt and assert that it receives the error and is redirected with a message, or whatever is important to your test.

-Second test that after the 1 minute wait the user can make a successful login. Use the Carbon's setTestNow() method to set the time for the test to a known time. Like maybe set it to now()->setTestNow(now()->parse('jan 1, 2018 12:00 pm')); Follow the same steps as the first test then move test now more than a minute forwared now()->setTestNow(now()->parse('jan 1, 2018 12:02 pm')); and login one more time asserting that the user was able to successfully attempt login.

m7vm7v's avatar

What I would do is for example if you have throttled for 5 times then

//create a user with known password 
$user = factory(User::class, ['email' => '[email protected]', 'password' => 'secret']);

//then make 5 post request to the authentication route
for ($i = 0; $i < 5; ++$i) {
        $this->post(route('admin.authenticate'), [
            'email' => $user->email,
            'password' => 'wrong',
        ]);

    $this->assertStatus(422);
}

//then on the 6th you would expect to see the throtteler to stop the request
$this->post(route('admin.authenticate'), [
            'email' => $user->email,
            'password' => 'wrong',
]);

$this->assertStatus(429);

I would also recommend you to watch this https://laracasts.com/series/whats-new-in-laravel-5-5/episodes/17 and use the $this->whithoutExceptionHandling();

D9705996's avatar

I would not get into specifics of testing the implementation of the throttle middleware. I would simply test you have throttling applied to a route

$this->assertContains('throttle:5,10', Route::getRoutes()->getByName('home')->gatherMiddleware());

You just need to make sure you test the full name of your middleware and that you name your routes (there might be a way to do this without named routes but I couldn't get it to work). My routes are

Route::get('/', function () {
    return view('welcome');
})->middleware('throttle:5,10')->name('home');

The rational for this is that laravel already tests that the throttle middleware works. If I remove the middleware I get

PHPUnit 7.4.3 by Sebastian Bergmann and contributors.

F.                                                                  2 / 2 (100%)

Time: 263 ms, Memory: 14.00MB

There was 1 failure:

1) Tests\Unit\ExampleTest::testBasicTest
Failed asserting that an array contains 'throttle:5,10'.

/home/dwalker/sites/blog/tests/Unit/ExampleTest.php:17

1 like
Sergiu17's avatar

My 5 pennies

Route::get('/testing-throttle', function () {
    return response('Success', 200);
})->middleware('throttle:1,1');
/**
* @test
*/
public function testing_throttle()
{
    $response = $this->get('/testing-throttle');
    $response->assertStatus(200);

    $response = $this->get('/testing-throttle');
    $response->assertHeader('X-RATELIMIT-REMAINING', 0);
}

Laravel responds with X-RATELIMIT-REMAINING header when limit is reached

Make sure you run tests once per minute, lol, or you can set less than one minute, like 1 second or 5

->middleware('throttle:1,.05');
1 like
D9705996's avatar

@sergiu17 - IMHO You dont need to test the actual middleware implementation. This is covered by the laravel framework tests. You just need to make sre you have applied the correct middleware to the route as per my example.

There's obliviously no harm in testing like you have but if the laravel middleware changes e.g the header name changes your test will fail but the actual throttling will still work.

1 like
Sergiu17's avatar

@D9705996 - Yes, you are right, more appropriate would be to assert status code, or better as you did with assert contain. That's why I said my 5 pennies :D Thanks!

D9705996's avatar

@sergiu17 - statuscode wont help as you would need to simulate the throttle scenario. Again you can do this but assuming your throttling isnt that strict you would need to hit your route enough times to trigger the 429 response.

To be honest I think this is something that should be in the framework for testing. I feel my first PR coming on"

1 like
alchermd's avatar

@D9705996 - Hi! I don't think the ThrottlesLogins trait applies the middleware explicitly (or at all) so I can't test it that way.

alchermd's avatar

Anyway, I found a workaround and just assert to expect a ValidationException to be thrown for the sixth login attempt. I think I'm good with that for now. Thanks everyone!

1 like
agm1984's avatar

I somewhat disagree with part of the philosophy here, because, such a test is against critical business logic. It is against a behaviour that is required to prevent brute force API hitting. If it stops working, your application is more vulnerable.

To finish my point, if Laravel introduces a breaking change such as the syntax to enable the throttling, and then someone updates Laravel and forgets to update the syntax, it won't work and you will have no idea.

This means you should test the behaviour that you require. If the definition of a login throttle is to receive an error 429, then all you need to test is that you do get that when expected. You don't care what logic causes the 429 to occur.

agm1984's avatar

Sorry to bump this again, but to finish my thought, here is my working test:

// 10 attempts, 5 minute cooldown
Route::group(['middleware' => ['guest', 'throttle:10,5']], function () {});

To actually test it, you need to be aware of this:

When making multiple requests in one test, the state of your laravel application is not reset between the requests. The Auth manager is a singleton in the laravel container, and it keeps a local cache of the resolved auth guards. The resolved auth guards keep a local cache of the authed user.

So, your first request to your api/logout endpoint resolves the auth manager, which resolves the api guard, which stores a references to the authed user whose token you will be revoking.

Now, when you make your second request to /api/user, the already resolved auth manager is pulled from the container, the already resolved api guard is pulled from it's local cache, and the same already resolved user is pulled from the guard's local cache. This is why the second request passes authentication instead of failing it.

When testing auth related stuff with multiple requests in the same test, you need to reset the resolved instances between tests. Also, you can't just unset the resolved auth manager instance.

Source: https://stackoverflow.com/questions/57813795/method-illuminate-auth-requestguardlogout-does-not-exist-laravel-passport

To accomplish that, you can add this:

TestCase.php

use Illuminate\Auth\SessionGuard;

...

    protected function resetAuth(array $guards = null) : void
    {
        $guards = $guards ?: array_keys(config('auth.guards'));

        foreach ($guards as $guard) {
            $guard = $this->app['auth']->guard($guard);

            if ($guard instanceof SessionGuard) {
                $guard->logout();
            }
        }

        $protectedProperty = new \ReflectionProperty($this->app['auth'], 'guards');
        $protectedProperty->setAccessible(true);
        $protectedProperty->setValue($this->app['auth'], []);
    }

Then, you can access it via $this->resetAuth(); in any unit test. By my estimates, this is a miracle utility function for testing edge cases.

Then you can simply test throttling like this:

    /** @test */
    public function it_should_throw_error_429_when_login_attempt_is_throttled()
    {
        $this->resetAuth();

        $throttledUser = factory(User::class, 1)->create()->first();

        foreach (range(0, 9) as $attempt) {
            $this->postJson(route('login'), ['email' => $throttledUser->email, 'password' => "{TestCase::AUTH_PASSWORD}_{$attempt}"]);
        }

        $this->postJson(route('login'), ['email' => $throttledUser->email, 'password' => 'k'])
            ->assertStatus(429)
            ->assertJson(['message' => 'Too Many Attempts.']);

        $this->resetAuth();
    }

And for the record I have no real issue with that in_array middleware test shown above. I figure we can look at the meme: "why not both.gif". If any change causes anything related to the auth unit tests to fail, it probably warrants any brittleness or extra keyboard time to confirm the logic is correct.

But my key opinion is that we want to test-to-confirm the behaviour is seen.

Please or to participate in this conversation.