Web development testing: Cucumber & Capybara

It's taken days to get my head around these web testing tools and more to get the tools working, but I finally got cucumber and capybara to work under both Mac and Windows.

I originally thought that cucumber seemed too high level but now that I've got it running, it's actually kind of fun and nice. Here's how it works.

Cucumber (and its backengine(s)) is an integration testing tool -- that is it tests the whole chain with a simulated web browser using the capybara gem as the back-engine interface to the simulated web browser. This is in contrast to rspec which is opening the models and controllers as actual ruby objects without the browser -- thus rspec is more limited in scope but way faster. You can apparently make cucumber/capybara do ajaxy things with special flag but I haven't tried that yet -- so far I've just filled in a form and pressed buttons without ajax.

Before running cucumber there's a bunch of little details you have to get right -- gems, dev kits, and then generate cucmber blah blah blah. Too much to document here. It was a multiday pain and required that I get a Gemfile from Sarah who had versions that worked. Also, I only got it working under rails 3 so I had to install ruby version managers and other stuff. It took tons of googling to get it all working.

It works like this. The cucumber generator creates a directory under the project root called "features" and under that folders called "step_definitions" and "support".

Into the features folder you put ".feature" files. Here is an actual one I just wrote (excuse the line spacing, this blog doesn't deal with "pre" tag very well):

Feature: Create account

As a new user

In order that I be able to store programs and data permanently

I want to be able to create an account



Scenario: Create a valid account

  Given I am on /users/new

  When I fill in the following:

    | Name          | zack     |

    | Password      | password |

    | Confirmation  | password |

  And I press "Create account"

  Then I should be on /users/account_created

  And there should exist a user named "zack"



Scenario: Create a invalid account due to mismatching passwords

  Given I am on /users/new

  When I fill in the following:

    | Name          | zack     |

    | Password      | password |

    | Confirmation  | passwo   |

  And I press "Create account"

  Then I should be on /users

  And I should see "Errors"

  And there should not exist a user named "zack"

The first four lines are pure documentation, cucumber ignores them. But the madlib style is standard and quite nice.

Each scenario is what's actually parsed. Think of each line under the scenario as a function call with various parameters.

The "function call" of the line is actually a regular expression that is put into the step_definitions folder. Cucumber installs a bunch of default "steps" (think functions) that are very handy and are in the step_definitions/web_step.rb file. I'm using a bunch of them here. Here's a sampling of one:

Given /^(?:|I )am on (.+)$/ do |page_name|

    visit path_to(page_name)

end

That is a "function" definition that says: when you see the pattern "Given I am on XXX" then run the command "visit" which is a capybara command that loads that page in the simulated web browser.

You can either use their existing steps or you create your own. If the step isn't found it emits a handy regular expression that you can copy and paste into your own step file.

So the long and short of it is that cucumber is a list of actions in very readable language that drives "steps" (i.e. test functions) in the "steps_definitions" folder that calls capybara (and maybe other gems?) to do the heavy lifting of running the simulated web browser.

The output of cucumber is a fairly well formatted color output that highlights each step in green if it passed, red if it failed, and yellow if it is pending (not yet implemented).

As Sarah said to me, it's actually kind of fun. You have a sense of pseudo-programming gathering your thoughts together as you edit the feature file and obviously the feature files are easy to read by non-technical types as they have no magical code incantations.

Thanks to @fablednet for help!