What/How to test

Jeffrey just published a really good mini-podcast on this topic. Check it out.

I would suggest there are a few important principles to keep in mind when you are testing:

Keep things in perspective. You do testing because you want your software to be reliable. Exactly how you decide to do testing is much less important than the fact that you do have tests.

If you find that a test is getting excessively tedious to write, then maybe you're going about it the wrong way. For example, let's say you are writing tonnes of complicated mocks to avoid touching the database. Maybe you should just write a simple test that touches the database and be done with it.

Do i write a test to see if laravel returns the correct error if, let's say, a user inputs a email that should be unique to the registration form but all other fields are correct? After that, write another test to see if i get the correct error if the email is okay, but for example the password confirmation does not match the password? Do i test registration to see if the user is correctly entered into the database? After that do i test login to see if the user can log in with those details?

I tend to "bundle" my acceptance tests, by testing the failure states first, and then finishing with the success state. So for a login test, I'll check the behaviour for "invalid username", then for "valid username, wrong password", then for a correct login (here I will check that the user is in fact logged in). That's just one way to do it; but regardless, I would recommend testing error states as well as success states.

If i do it like that, i (think) i'm already testing everything in my application, right? I test with this all the forms, all the pages, all the links. Should I (And WHY) do any other testing (Unit testing with PHPSpec)?

Testing "outside in" like that is a really good starting point. It means you've automated the tests that you used to do manually in a web browser. In other words, your QA process has become much less work (and more reliable).

However, unit tests are often beneficial because they are more precise. Imagine that you change something and one of your acceptance tests fails. Okay, the test has proved valuable because it alerted you to the failure. That's great. But now what? How do you find the problem?

If you have a good suite of unit tests too, then they will often show you exactly where the failure is. Unit tests verify the correctness of small bits of code -- typically individual methods. This is "inside out" testing.

Some people also like using unit tests to guide program design. This is more a matter of personal taste, and it also depends on the type of code you're writing. For example, I'm not convinced that "test driven design" is useful when you're doing CRUD work. Anyway, PHPspec encourages this approach. But you can also just use PHPunit to do unit testing.

Thanks for your answer, @MikeHopley - much appreciated.

That mini-podcast (released yesterday - nice!) is really good. And he kinda confirms how I thought about it. However, at the end he talks about laravel's built in testing API and - if i understood that correctly - says something along the line "... if you are not using Laravel, you can use Behat or Codeception for unit test..." which made me think he implied the Laravel Testing API are Unit Tests? But they are not, right? That API is used for functional testing?

If you find that a test is getting excessively tedious to write, then maybe you're going about it the wrong way. For example, let's say you are writing tonnes of complicated mocks to avoid touching the database. Maybe you should just write a simple test that touches the database and be done with it.

I heard about mockups today when i watched a few laracasts on testing, but this part I don't get: WHY NOT touch the database? I'm in an testing environment, there is no data in the database that is important - so what is the downside to just go for it and touch the database, resetting it after the test? I totally get how mocking is good for parts like sending an email.

Yeah, even though I didn't do tests until now, I think I kinda have (and want) to test both fail and test states. - Or else I can just scratch testing all together.

Testing "outside in" like that is a really good starting point. It means you've automated the tests that you used to do manually in a web browser. In other words, your QA process has become much less work (and more reliable).

Good to hear that I was on the right track with this. I just thought "ugh, that's a LOT of stuff to write, that can't be right".

However, unit tests are often beneficial because they are more precise. Imagine that you change something and one of your acceptance tests fails. Okay, the test has proved valuable because it alerted you to the failure. That's great. But now what? How do you find the problem?

If you have a good suite of unit tests too, then they will often show you exactly where the failure is. Unit tests verify the correctness of small bits of code -- typically individual methods. This is "inside out" testing.

Some people also like using unit tests to guide program design. This is more a matter of personal taste, and it also depends on the type of code you're writing. For example, I'm not convinced that "test driven design" is useful when you're doing CRUD work. Anyway, PHPspec encourages this approach. But you can also just use PHPunit to do unit testing.

Okay, I gotta take a closer look at Unit Testing it seems - it does seem to be useful in addition to the functional tests. I think I will stay for the Unit tests with PHPunit, it seems to be relatively easy to learn - and is also suggested in the Podcast.

To expand upon what MikeHopley touched on in his last paragraph, there are essentially two ways to go about testing:

• Test After (Test Last)
• Test First

Test After is used to verify the work you have done, so that you have a acceptance / regression tests that you can use to make sure that what you have done is correct / functionality does not break when you refactor / change code. This seems to be the approach taken by most in the Laravel community. Personally I do not find much value in it, as it does nothing to assist with the design of the program.

Test First on the other hand, can be used to TDD your code and guide program design. As MikeHopley suggests, it isn't too valuable an approach when designing CRUD systems, especially with a decent framework like Laravel. Where I find this vastly more valuable is when designing other program logic.

Referring to the types of testing in your OP, which are end to end tests with requests from the front end, into the application and responses back to the browser, I have been doing quite a bit of TDD of this type of thing in a Laravel app recently. However, it feels quite like wading through treacle a lot of the time, compared to TDDing non-front-end system objects with PHPUnit directly. The big problem I have with this approach is that by testing front end components using the Laravel Testing tools, you are coupling your tests to the front end design, which can cause trouble further down the line. For instance, here is an example test I have written:

 public function testUsDollarCurrencySelectionISelectedInTheProductSearch()
    {
        $this->visit('/search?minimum_price=1000')
            ->seeIsSelected('currency', 'USD')
            ->see('Min Price ($)')
            ->see('Max Price ($)');
    }

To make this test pass, I've written the relevant code in the views to ensure that the correct currency is selected. I've also ensured that the strings "Min Price ($)" and "Max Price ($)" are displayed correctly, and so the tests passes. However, as soon the client wishes to change this design, I have to change the test in order to do so, even though the behaviour of the system has not changed at all. Worse still, if the designers just go ahead and make the changes without notifying me, then my test will fail the next time I run it, giving me more work to find and fix...

if you are not using Laravel, you can use Behat or Codeception for unit test..." which made me think he implied the Laravel Testing API are Unit Tests? But they are not, right? That API is used for functional testing?

Correct. I've not listened to the podcast, but IMO the Laravel Testing is more BDD, like Behat and Codeception are. PHPUnit is Unit Testing.

I heard about mockups today when i watched a few laracasts on testing, but this part I don't get: WHY NOT touch the database? I'm in an testing environment, there is no data in the database that is important - so what is the downside to just go for it and touch the database, resetting it after the test? I totally get how mocking is good for parts like sending an email.

Correct again. No reason not to touch the database if you are using the DatabaseMigrations() trait.

Good to hear that I was on the right track with this. I just thought "ugh, that's a LOT of stuff to write, that can't be right".

I find that I write approximately 4x as many LOC in my tests than the code they are testing. There is a lot to write, but it is right :)

Okay, I gotta take a closer look at Unit Testing it seems - it does seem to be useful in addition to the functional tests. I think I will stay for the Unit tests with PHPunit, it seems to be relatively easy to learn - and is also suggested in the Podcast.

I would recommend starting with some Unit Testing, to learn the inside-out approach. Starting with small units is easier to learn about testing. Then when you are happier with that, you can move the levels into functional testing, then end-to-end testing. Then when you are confident with this, you can do your testing outside-in, which is generally the best approach for system design and TDD/BDD.

The big problem I have with this approach is that by testing front end components using the Laravel Testing tools, you are coupling your tests to the front end design, which can cause trouble further down the line. For instance, here is an example test I have written:

This is EXACTLY what i thought too when thinking about TDD and Laravel integrated tests.

Correct. I've not listened to the podcast, but IMO the Laravel Testing is more BDD, like Behat and Codeception are. PHPUnit is Unit Testing.

Laracasts sadly has not Unit Testing with PHPUnit lesson yet ( In the Podcast, @JeffreyWay said he will do a lesson on it soon ... i'm waiting, Jeffrey! ;) ). And till now i haven't found any nice resource for it either - Most stuff that pops up on google is about Laravel Integration testing .. and they just call it Unit Tests.

I would recommend starting with some Unit Testing, to learn the inside-out approach. Starting with small units is easier to learn about testing. Then when you are happier with that, you can move the levels into functional testing, then end-to-end testing. Then when you are confident with this, you can do your testing outside-in, which is generally the best approach for system design and TDD/BDD.

To be honest, the Outside-in tests with Laravel Integrated tests are pretty straightforward and easy, but no doubt i got to learn about unit tests too.

Yes calling the Laravel Integration tests Unit Tests is a misnomer. I don't like to pay too much attention to naming of the types of tests, as the lines are very blurred between them (contrary to what tutorials may tell you!), but in this case, there is a such a big difference that they should be named correctly.

When learning how to write unit tests as I was learning about TDD, I found most of the tutorials too contrived to be of real use. Things like "write a calculator", "FizzBuzz" and other things which are of little use in actual projects I was doing. Following the tutorial was easy, but as soon as I tried to implement in projects, my understanding fell apart and couldn't do it. What was missing from the tutorials was the real-world context, that you get with a live project. Missing were the requirements, the nuances, the feedback from others etc. which make the difference between a tutorial and a project.

I have also had a couple of TDD mentors over the last couple of years and they have helped me no end in my understanding. Unfortunately I do not have the time at present to be able to do any one-to-one mentoring, as my time at the keyboard is rather sporadic. However, I can offer you some TDD training via a Github repo, kind of like a correspondence course. I should be able to push to the repo at least once or twice a day. The format would be along the lines of:

• You tell me the component (Class) that you wish to TDD (so we use something which has context for you). Some examples of things I have written recently include; a currency converter; an email enquiry delivery service; a REST Api client; data object formatters for data returned from said REST client etc.

• We discuss the requirements for the component (via email / Skype etc.)

• I make a start TDDing the component; using the red-green-refactor cycle.

• I commit after each red-green-refactor step.

• I push when I have completed some cycles.

• You can review the TDD process via the commit log; this will give you an understanding of my approach to TDD and how I can make it work for me.

• You branch my code and try to continue the TDD cycle for the next requirements of the component.

• I continue in my way on the same next bit of the component, so we are both coding the same thing alongside each other.

• we both push at the end of this next component and then review the code and commit log to see how things are progressing.

How does something like the above sound? I've set up a base L5.1 repo to do this from: https://github.com/ifpingram/php-tdd-laravel - if you want to give it a try, let me know you github handle and I will add you as a collaborator in the repo...

I heard about mockups today when i watched a few laracasts on testing, but this part I don't get: WHY NOT touch the database? I'm in an testing environment, there is no data in the database that is important - so what is the downside to just go for it and touch the database, resetting it after the test?

In many cases, that's absolutely the right thing to do. But not always.

One advantage is that the test will run faster. Often the speed difference is small enough that it's irrelevant, but sometimes the speed difference is dramatic. Another advantage is that mocking may involve easier setup.

For example, I have a StripeWebhook class, which receives webhooks from Stripe and converts them into a standardised "internal" message for my application. So basically, it takes an array of data, analyses it, and outputs another array. Due to how the logic works, in one place I need to check the database to see whether a subscription already exists.

Now, I could go set up "matching" subscriptions in the database. I have other tests that rely on this kind of dummy data. But really, that is one line of code in a 200 line class (and 300 lines of test code).

In this case, I prefer to mock that method in my tests. You see, the class being tested isn't really meant to interact with the database. It just has this one little database dependency. So the easiest thing in this case, I feel, is to mock the call.

In the same class, I also have a dependency on the Stripe API: when I receive a webhook, I may need to fetch some additional data from Stripe. But this is very slow compared to mocking the response. I have lots of tests to run, and if every one is making an API call to Stripe, then my test could go from ~ 1 second to ~ 30 seconds.

And again, this class is not really concerned with the correctness of my Stripe API code. I have a separate StripeApi class for that. So instead, I swap out the real StripeApi for my StripeMockApi class, which provides some default responses.

Test After is used to verify the work you have done, so that you have a acceptance / regression tests that you can use to make sure that what you have done is correct / functionality does not break when you refactor / change code. This seems to be the approach taken by most in the Laravel community. Personally I do not find much value in it, as it does nothing to assist with the design of the program.

I would say the main value in tests is ensuring that software works; and that when something breaks you are aware that it has broken, and can easily find the part of your code where it broke.

TDD can be a great way to write code. I quite like using TDD. But testing your code is much more important than adhering to TDD.

I do not necessarily need TDD to help me design good code. But I do need tests to make sure my stuff actually works.

I would say the main value in tests is ensuring that software works; and that when something breaks you are aware that it has broken, and can easily find the part of your code where it broke.

For me that is a fantastic bonus of writing tests in a TDD fashion :)

I do not necessarily need TDD to help me design good code.

And that's probably why there is a divergence on our opinions, I need the help that TDD gives me to design good code!