kfirba's avatar
Level 50

Behat suites and contexts

Hello!

By watching laracasts lessons I really grew to love Behat. However, there is one thing I can't get right.

As Jeffrey says, we should write our features in a more general way so we don't explicitly bind our feature to the UI (Acceptance) or to just one testing suite. In some videos I can see that he has the following folders:

  • Acceptance
  • Functional
  • Integration
  • spec

I believe spec belongs to PHPSpec, no problem there.

I saw that he stores some of his .feature files in the Acceptance folder. Does it mean he has it cloned in the Functional and Integration folders so he can run the same .feature against a different suite?

In general I think I'm a little bit confused about the contexts and suites, can anyone please arrange the mess in my mind?

How can I even create multiple suites and contexts? Does a context belong to a suite? If I have xContext in Acceptance/bootstrap does it mean I need another xContext in Functional/bootstrap for example?

Thanks in advance!

0 likes
13 replies
oes's avatar

The cool thing with Behat is you can create multiple suites but they can test against multiple context files. So for example let say you write a feature to perform a search on your site. You would expect the page to render x results. But you are able to test the results without hitting the ui ie by testing the response of a service or command handler by testing with the domain suite first.

So as an example I would call these suites domain & ui and then have two context files ie DomainContext, UserInterfaceContext. I would write my behat.yml file like so. Note how the paths for these features are the same. This is how I can create one feature but test against multiple context. One touching the UI the other just testing the domain.

default:
    formatters:
        pretty: true
    suites:
        domain:
            paths: [ %paths.base%/features/joint ]
            contexts:
                - DomainContext
        ui:
            paths: [ %paths.base%/features/joint ]
            contexts:
                - UserInterfaceContext
    extensions:
        Laracasts\Behat: ~
        Behat\MinkExtension:
            default_session: laravel
            base_url: http://demo.app
            laravel: ~

So when creating your site you would want to test the domain before the UI, from console you would type "behat -s domain" This means only test the domain context. You get the idea. Now the cool thing you should get your domain all working and with right results. All you need to do then is create your ui layer and test against it.

Does this make more sense?

kfirba's avatar
Level 50

@oes Thanks for dedicating time to write this wonderful answer.

Few questions:

  • what is the paths: attribute in your behat.yml? (What does it point at and how)
  • How would your folder structure look?
  • Where do you store the .feature file?
  • By domain suite do you mean the Functional tests?
  • By ui suite do you mean the Acceptance?
  • What is the point using different contexts names if anyways I'm going to have 1 context file per suite? Why not just name them FeatureContext?

I think most of my questions will be solved if I could see the file structure for your example.

oes's avatar
oes
Best Answer
Level 3

@kfirba Here goes.

  • Its the paths to where your feature files are kept. See docs: http://behat.readthedocs.org/en/v3.0/guides/5.suites.html
  • Like this: http://d.pr/i/1jqzs/4iqA3Yk5 (new project being setup)
  • See screenshot above.
  • Domain can be both functional and acceptance. You can always tag a scenario as acceptance so to skip them or just run them. See Jeffery latest video on Mailtrap to see how
  • UI tests are fully outside in and most likely all if them would be acceptance tests.
  • As per my first reply by having one feature for multiple contexts (ui, domain) give's you the ability to test in two ways. Let me share simple code below.

ie, Let say we have a feature where a person can search for a product on your site.

Feature would be something like. (product_search.feature)

Feature: Search For a Product
  In order to find a product
  As a customer
  I need to be able to perform a product search 

  Scenario: Perform a search to find the product I need
    Given there is a product named "Basket Ball"
    When I perform a search for the term "basket"
    Then I should see a product called "Basket Ball"

So this works well as I can do this by domain first. Here is a very brief example

DomainContext.php

use .........
use PHPUnit_Framework_Assert as PHPUnit;
use Laracasts\TestDummy\Factory;

/**
 * Defines application features from the specific context.
 */
class DomainContext implements Context, SnippetAcceptingContext
{
    use DispatchesCommands, Migrator, DatabaseTransactions;

    protected $product;
    protected $result;

    public function __construct(){}

    /**
     * @Given a product named :arg1
     */
    public function aProductNamed($name)
    {
        $this->product = Factory::create('App\Product', ['name' => $name]);
    }

    /**
     * @When I perform a search for the term "term"
     */
    public function iPerformASearchForTheTerm($term)
    {
        $request = Request::create('/', 'POST', ['term' => $term]);
        $this->result = $this->dispatchFrom(SearchProductCommand::class, $request); // Using Commands :-)
    }


    /**
     * @Then I should see a product called :arg1
     */
    public function iShouldSeeAProductCalled($called)
    {
        PHPUnit::assertEquals($called, $this->result->product->name);
    }
}   

UserInterfaceContext.php


use ......... use PHPUnit_Framework_Assert as PHPUnit; use Laracasts\TestDummy\Factory; /** * Defines application features from the specific context. */ class DomainContext extends MinkContext implements Context, SnippetAcceptingContext { use DispatchesCommands, Migrator, DatabaseTransactions; public function __construct(){} /** * @Given a product named :arg1 */ public function aProductNamed($name) { Factory::create('App\Product', ['name' => $name]); } /** * @When I perform a search for the term "term" */ public function iPerformASearchForTheTerm($term) { $this->visitPath('search'); $this->fillField('term', $term); $this->pressButton('Search'); } /** * @Then I should see a product called :arg1 */ public function iShouldSeeAProductCalled($name) { $this->assertResponseContains($name); } }

See how one feature can test multiple contexts. These are of course not full versions as you would be adding more assertions etc but hopefully give you an idea of how this works.

Hope this helps. You can mark as answered if you like :-)

4 likes
kfirba's avatar
Level 50

@oes Briliant! Thanks! Much appreciated.

Just a few more question to summarize things:

  1. I saw you have joint folder and web folder for features, why is that?
  2. If I want to separate my tests by folders e.g. Acceptance folder, Functional folder and so on, how would I represent that in my behat.yml file?
  • I'm not sure now that I fully understand the terms of Acceptance, Functional, and Integrarion tests as well as testing my "Domain" layer etc. As it is irrelevant to this discussion I will open a new one to clarify things about that.
oes's avatar

@kfirba Response Below

  • I have some tests which do not require both domain and ui. They are just web stuff (just ui( so I put them in there.
  • Please use the link in last post. You can create what folders you like.

  • Acceptance = Fully testing ui layer outside in. (Behat)

  • Functional = Unit tests (Phpspec)
  • Integration = Testing database layer on your code. Like an integration test but touching the database, ie testing repository (Phpunit)

I use all three tools describe above.

Hope this helps clear it up.

kfirba's avatar
Level 50

@oes Thanks again.

Functional tests are solely unit tests? If that so, where do I test my code without mocking objects?

Also, can't you use Behat for Integration Testing?

oes's avatar

@kfirba

Functional are unit tests yes, I use phpsepec for them. You may have to mock some objects but its important the if you are going to mock a response to make sure that is what you will get by a functional type of test. This has been all the talk recently with DHH etc.

Yea use behat for integration tests. These are what I call Domain as per my examples above as they are hitting the database. But the Domain test is testing a flow. ie To get a product from the store it will follow this type of sequence. But its also important you create separate integration tests for testing different outcomes per their response.

The biggest thing is to test, and don't feel you need 100% coverage as thats just overkill. Testing things like validation are a wait of time as Taylor has already unit tested them. Only create a test for example on a custom validation and code YOU have created and not the framework.

Hope this helps !

kfirba's avatar
Level 50

@oes Thanks.

In your first paragraph you said:

Functional are unit tests yes, I use phpsepec for them. You may have to mock some objects but its important the if you are going to mock a response to make sure that is what you will get by a functional type of test.

So you are saying that functional tests are unit tests but you also mention that I have to make sure that my expectation are backed up by (functional) tests. So Functional Tests are either unit tests and not unit tests?

oes's avatar

@kfirba

Functional test's are Unit tests. Your testing a function (method) or unit of code. :-)

For example a good use-case for a functional test is a calculator. You dont need to integrate with a database as the data required is not coming from a database and can be supplied to the class, method your testing.

kfirba's avatar
Level 50

@oes

I understand. Taking the example you used of searching a product, you showed 2 ways to test that, one from the outside in - User Interface, and one from the inside out.

The outside in testing, I believe you would call them acceptance tests. What about the other inside out test? Would you also refer to them as Acceptance tests even tho they have nothing to with the UI but to your internal implementation or as a Functional tests because you are testing the functionality of your application? (Maybe you would refer to that as an Integration test?)

Follow-up question:

What kind of test is it to test a controller? There are many things involved. You can have a Repository, Commands etc.

oes's avatar

@kfirba

See this quick mockup: http://d.pr/i/1fHpV/4SDPsdJd

  • The grey band is the User Interface. So testing what's on screen submitting buttons etc.
  • Domain level is testing the code works. Like performing the product search. Does the command handler return the correct results etc... Its toucing many levels of your application to get the end result.
  • Unit as we have discussed.

So in theory you test the Domain level first. It goes without saying to do the Domain level you end up creating Unit, Integration tests. Hence they go hand in hand.

I work this way now and do the UI layer last. Or at least until the Domain Feature is working as expected.

I don't test controller's. I dont see what benefit they have. I already test what's coming into the controller and the UI layer will test what the controller is outputting. Other people will disagree just my way of looking at it.

kfirba's avatar
Level 50

@oes Thanks seems like I'm able to understand it more now.

On that note, to which type of testing would you relate the Domain layer?

oes's avatar

@kfirba

Not sure what you mean by relate but again. A Domain layer is testing a full feature, ie perform a search, enroll a user, add a product... Its a feature of your site.

Please or to participate in this conversation.