Richard Lawrence » BDD

Cucumber Tip: Key-Value Tables

Richard Lawrence — Thu, 27 Oct 2011 21:38:28 +0000

You may not realize this: Tables in Cucumber steps don’t have to have a header row. Sometimes it can work really well to use a headerless table of key-value pairs.

Let’s look at an example. Suppose we have a scenario that fills out an advanced search form to search for medical providers matching certain criteria. A mockup of the form looks something like this:

If we were using the recently deprecated web steps generated by cucumber-rails, we might write steps to perform a search like these:

1
2
3
4
5
6

Given I'm on the advanced search page
And I select "Endocrinology" from "Specialty"
And I choose "Yes" within "Accepts Insurance"
And I fill in "ZIP Code" with "90010"
And I select "5 miles" from "Search Radius"
When I press "Search"

But we know better than to do that, right? After all, we’re trying to describe how the search logic should work, not how the form should look. So we try to write our own steps to do these same thing without talking about the implementation so much, but it’s really hard to get away from the implementation. Even if we drive out words like select, fill in, and click, we still have a step per form element.

Suppose we try to combine them into a single step for the domain concept of performing a search. We might get something like:

1	When I search for an endocrinologist within 5 miles of 90010 who accepts insurance

The implementation details are gone, but it’s long and hard to read. Enter the key-value table:

1
2
3
4
5
6

In the step definition, we can use the rows_hash method on the table object to convert that key-value table to a single hash. There’s no need to iterate through the rows ourselves. For example, assuming a helper method called provider_search that actually performs the search,

1
2
3
4
5
6
7
8

When /^I search for a provider with the criteria:$/ do |table|
criteria = table.rows_hash
provider_search :provider_type => criteria['Provider Type'],
:specialty => criteria['Specialty'],
:accepts_insurance => criteria['Accepts Insurance'].to_bool,
:zip_code => criteria['ZIP'],
:radius => criteria['Search Radius'].to_i
end

You may have noticed the call to to_bool. This uses a helper method like the following to allow us to use yes and no instead of true and false to make more readable scenarios. Put this somewhere in the support directory.

1
2
3
4
5

class String
def to_bool
!!(self =~ /^yes|y|true|t|1$/i)
end
end

The call to to_i on the search radius takes advantage of Ruby’s string to integer conversion behavior—if a string starts with an integer but has other text after the integer, the other text is stripped off during the conversion. Try it: Fire up irb (the interactive Ruby console) in a command window and run "5 miles".to_i.

These key-value tables can get long for a complex entity. Not all the details will matter each time. We can make the step read better by introducing a concept of a default provider search. Maybe we have to supply a ZIP code and search radius, but the scenario we’re working on is really about specialties and insurance. We could say,

1
2
3
4

In the step definition, we’ll declare the default values in a hash and merge it with the supplied values.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

When /^I search for a provider with the default criteria and:$/ do |table|
default_criteria = {
'Provider Type' => 'Doctor',
'Specialty' => 'General',
'Accepts Insurance' => 'Yes',
'ZIP' => 90010,
'Search Radius' => '5 miles'
}

criteria = default_criteria.merge(table.rows_hash)
provider_search :provider_type => criteria['Provider Type'],
:specialty => criteria['Specialty'],
:accepts_insurance => criteria['Accepts Insurance'].to_bool,
:zip_code => criteria['ZIP'],
:radius => criteria['Search Radius'].to_i
end

If the defaults are well known, this is enough. If they need to be documented, one way I like to do it is as executable documentation with a scenario like this:

1
2
3
4
5
6
7
8

If you do this, you’ll want to move the default criteria hash into a helper method or constant to make sure it’s the same hash used in the real scenario and the documentation scenario.

We could make the step definition read better by separating the key and value conversions from the use of the data. Cucumber’s table object has map_headers! and map_column!. Unfortunately, those don’t work for rows_hash at this time. I recently submitted a pull request to make it work. Hopefully, we’ll see that in a Cucumber release soon.

The Future of Cucumber on .NET

Richard Lawrence — Fri, 21 Oct 2011 17:56:14 +0000

Beginning just over two years ago, I worked with some great developers to create Cuke4Nuke as a way to bring Cucumber to .NET. Shortly after the first releases of Cuke4Nuke, TechTalk released SpecFlow, a native .NET tool inspired by Cucumber. As a pure .NET solution without Ruby dependencies, SpecFlow seemed to be easier for Microsoft shops to adopt. But Cuke4Nuke had (to me) enough advantages that I kept it going.

By this summer, however, it was clear that SpecFlow had made up for most of Cuke4Nuke’s advantages and had more momentum in the .NET community and was under much more active development. There were just a few gaps SpecFlow needed to overcome (in my opinion), mostly related to better harmonization with the other Cucumber projects. (With the pure Java Cucumber-JVM under active development as a Cuke4Duke replacement, there was no longer as compelling an argument for shared code between Cucumber versions, Cuke4Nuke’s biggest remaining advantage.)

At AA-FTT and later in the week at Agile 2011, my colleague Paul Rayner and I met with Christian Hassa and Gaspar Nagy from the SpecFlow team. We discussed the few things that needed to change in SpecFlow to make it behave more like the other Cucumbers.

This week, most of those changes were released as part of SpecFlow 1.8. With that release, I’m letting Cuke4Nuke die. I won’t be maintaining or extending Cuke4Nuke. My Cucumber classes and book will cover SpecFlow rather than Cuke4Nuke.

I’ve had a great time getting to know the SpecFlow guys, and I look forward to collaborating with them in the future to help bring Cucumber to more and more teams in the Microsoft world.

Just Enough Regular Expressions for Cucumber

Richard Lawrence — Tue, 20 Jul 2010 21:38:50 +0000

Jon Archer wrote last week about how Cucumber makes knowledge of regular expressions important. He’s right: Regular expressions are the key to Cucumber’s flexibility. Well-crafted regular expressions let you reuse step definitions, avoiding duplication and keeping your tests maintainable. But even experienced developers find them mysterious and overwhelming.

Fortunately, you don’t need regular expressions like this one to wield the power of Cucumber:

(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])

In fact, if you use regular expressions like this in your step definitions, you’ve gone too far. (This regular expression, in case you’re wondering, matches the official spec for valid email addresses.)

As with most things, the 80/20 rule applies. There are a handful of useful patterns that are sufficient to make you a Cucumber power user.

Anchors

The regular expression I'm logged in matches I'm logged in and I'm logged in as an admin. To avoid ambiguous matches, use ^I'm logged in$.

The caret at the beginning anchors to the beginning of the string. The dollar at the end does the same with the end of the string. Use these with all your step definitions and you won’t have surprise matches.

Wildcards and quantifiers

Matching specific words is fine. But you often want flexibility to match a variety of strings. Here are some common patterns for non-exact matches.

`.*`	matches anything (or nothing), literally “any character (except a newline) 0 or more times”
`.+`	matches at least one of anything
`[0-9]` or `\d`	matches a series of digits (or nothing)
`[0-9]+` or `\d+`	matches one or more digits
`"[^\"]*"`	matches something (or nothing) in double quotes
`an?`	matches a or an (the question mark makes the n optional)

Capturing and not capturing

When you put part of a regular expression in parentheses, whatever it matches gets captured for use later. This is known as a “capture group.” In Cucumber, captured strings become step definition parameters. Typically, if you’re using a wildcard, you probably want to capture the matching value for use in your step definition. Here’s a Cuke4Nuke example,

1
2
3
4
5

[Given(@"^I'm logged in as an? (.*)$")]
public void ImLoggedInAsA(string role)
{
// log in as the given role
}

If your step is Given I'm logged in as an admin, then the step definition gets passed "admin" for role.

Cuke4Nuke converts captured strings to the step definition parameter type, which is handy for step definitions like this:

1
2
3
4
5

[Given(@"^I have (\d+) cukes$")]
public void IHaveNCukes(int cukeCount)
{
// set up the given number of cukes
}

The step Given I have 42 cukes means the step definition gets called with 42 (as an integer) for cukeCount.

Sometimes, you have to use parentheses to get a regular expression to work, but you don’t want to capture the match.

For example, suppose I want to be able to match both When I log in as an admin and Given I'm logged in as an admin. After all, both step definitions do the same thing. There’s no reason to have duplicated automation code in my step definitions simply because one is a Given and one is a When.

I might write something like this:

1
2
3
4
5

[When(@"^(I'm logged|I log) in as an? (.*)$")]
public void LogInAs(string role)
{
// log in as the given role
}

The parentheses and pipe indicate a logical OR, just what I need to match two different strings.

This will fail to run, though. My regular expression is capturing two strings, but my step definition method only takes one. I need to designate the first group as non-capturing like this:

1
2
3
4
5

[When(@"^(?:I'm logged|I log) in as an? (.*)$")]
public void LogInAs(string role)
{
// log in as the given role
}

Now, with the addition of ?: at the beginning of the group, it will perform as I expect.

By the way: You may be wondering how the attribute can be When and still match Given I'm logged in as an admin. It turns out that in Cuke4Nuke, just like in Cucumber for Ruby, it doesn’t matter whether you use Given, When, or Then to define a step definition. They’re all step definitions and are interchangeable. It’s fairly common for today’s When to be tomorrow’s Given, so this is a nice feature.

Just enough

This is only the tip of the regular expression iceberg. Here’s a book and website if you’re interested in going deeper. But for day-to-day work with Cucumber, anchors, simple wildcards and quantifiers, and capturing and non-capturing groups are all you need.

The Latest on Cuke4Nuke

Richard Lawrence — Wed, 30 Dec 2009 17:42:38 +0000

This morning, I released version 0.3.0 of Cuke4Nuke. With this release, Cuke4Nuke supports almost everything you can do with Cucumber in Ruby or Java, making C# a first class language for Cucumber. (The only missing features are small things like tags on Before and After hooks and a richer Table object.) Check out the Cuke4Nuke wiki for instructions to install and get started with Cucumber in .NET. To see it in action, check out my screencast on Cuke4Nuke and WatiN.

Screencast: Testing Web Applications in .NET with Cuke4Nuke and WatiN

Richard Lawrence — Thu, 03 Dec 2009 15:53:41 +0000

Yesterday, I released Cuke4Nuke 0.2.2, which added WatiN compatibility and an example of how to use the two tools together. Here’s a short screencast in which I walk through the example:

Resources from the video:

The Cuke4Nuke Wiki with installation instructions
WatiN
Source code from the example
The new Ruby installer

Cuke4Nuke: Cucumber for .NET Teams

Richard Lawrence — Sun, 20 Sep 2009 00:16:38 +0000

Update: If you’ve just landed here, you could get the impression from this post that Cuke4Nuke doesn’t exist yet. It does. Check out this screencast showing what you can do with it as of early December 2009.

If you’ve read this blog for a while or talked with me about functional test tools, you’ve heard me talk about Cucumber. It’s my favorite ATDD tool because it’s so good at mapping stories and acceptance criteria to automated functional tests. Product Owners and BAs write acceptance criteria in natural language. Developers and testers unobtrusively automate tests for them. Anyone on the team can run the tests and see the current state of the system.

Here’s a simple example:

1
2
3
4
5
6
7
8
9
10
11
12
13

Feature: Google search
In order to find things on the web
As a user
I want to search for web pages containing specific text

Scenario: Load search page
When I go to the search page
Then I should be on the search page

Scenario: Search
Given I'm on the search page
When I search for "richard lawrence"
Then I should see "www.richardlawrence.info" in the results

This reads almost exactly as I teach Product Owners to specify acceptance criteria. But it’s not just text. It’s a potentially automated test.

A developer or tester can come along and automate this test like so in Ruby:

1
2
3
4
5
6
7
8
9
10
11

# assuming @google is an instance of a test DSL that wraps Watir, Selenium, etc.

When /^I search for "(.*)"$/ do |query|
@google.search_for query
end

Then /^I should see "(.*)" in the results$/ do |expected_text|
assert { @google.results_contain? expected_text }
end

# etc...

Each of the Given/When/Then calls is a step definition. When there’s a matching line in a Cucumber test, the step definition gets executed.

Recently, support was added for step definitions in Java via a project called Cuke4Duke:

1
2
3
4
5
6
7
8
9
10
11

@When("I search for \"(*)\"")
public void search(String query) throws Exception
{
google.searchFor(query);
}

@Then("I should see \"(*)\" in the results")
public void checkResults(String expectedUrl)
{
assertThat(google.containsResult(expectedUrl), is(true));
}

Now, a Java team can use Cucumber without ever writing a line of Ruby.

Unfortunately, there’s nothing like this for .NET teams. Until now (or soon, anyway)…

AA-FTT and the Birth of Cuke4Nuke

Last month, I attended the Agile Alliance Functional Test Tool conference (AA-FTT for short). AA-FTT is an open space conference. I came with one goal: to get together with other people who want Cucumber for .NET and start making it happen. I wasn’t sure anyone else would be interested, so I was thrilled with the reaction.

Aslak Hellesøy introduced Cucumber for those in the group who hadn’t used it and then talked through the multiple language support he’d recently added to the tool. Then, we discussed ways to build .NET support. The obvious solution was to use IronRuby and Cucumber’s language support to handle C# step definitions. Matt Wynne downloaded the latest IronRuby and installed Cucumber on it. He kicked off a simple Cucumber example, and we waited. And waited. Some two minutes later, we had results from tests that take less than two seconds to run under the standard Ruby interpreter. In a process that values frequent, fast test runs, IronRuby was a non-starter. (If you want to try Cucumber under IronRuby, here are some instructions.)

So we discussed other options and settled on using a simple wire protocol for Cucumber to communicate to .NET out-of-process, similar to Slim in FitNesse. And here’s the best part: we started building it. Matt and I paired right there to start fleshing out the wire protocol (with Cucumber tests, naturally). Later in the week at Agile 2009, Matt and Aslak paired to build the Ruby side of the wire protocol, and Matt and I tackled the first bits of the .NET side. Last week, I got the skeleton of the .NET side working and up on GitHub. You can define simple steps in C#. Cucumber can ask the .NET wire server to tell it about the steps it has and to invoke them and return the pass/fail results.

About a week ago, Aslak announced the project on the Cucumber mailing list and recruited more contributors. Declan Whelan, Scott Ford, Åsmund Eldhuset, Anders Hammervold, Chris Kooken, and Steve Eley have already stepped up with ideas and code.

Getting Involed

How can you get involved? I’m glad you asked.

Join the mailing list.
Fork the GitHub repository and work on one of the features in the backlog.
Post a message to the mailing list to let us know what you’re working on. I’ll tag the ticket in the backlog with your GitHub username so we don’t duplicate effort. Prefix your message subject with [Cuke4Nuke].
Comment on tickets in the backlog.
Try using Cuke4Nuke as we develop it and give us feedback via the mailing list.
Shout encouragements in the comments here and on Twitter to let us know you care, even if you can’t contribute.

Web Testing for .NET Teams: WatiN or Watir?

Richard Lawrence — Tue, 20 Jan 2009 03:49:04 +0000

I’ve noticed a pattern with several of my .NET clients who want to get into automated acceptance testing for web applications. They like the idea of WatiN because it would let them write tests in the same language as their production code. But then they notice that there’s much more documentation and apparently a much more active community around Watir. And that Ruby language looks interesting too. What to do?

I think there are good arguments for both. Here are the major pros and cons from my perspective…

WatiN

Pros	Cons
Code and test in the same language (e.g. C#) No need to learn a new language (i.e. Ruby) No need to support another language No need to hire people who know another language Did I mention that you can write tests in C#?	IE only (though there’s a CTP for FireFox support) Very limited documentation More syntactic noise (more due to .NET than to WatiN itself; e.g. new Regex(“txtFoo$”) vs. /txtFoo$/)

Watir

Pros	Cons
Very active community Active ongoing development on Watir and related projects Good support on the discussion list Concise, clear syntax (more due to Ruby than to Watir itself) Good documentation (though finding the current RDoc online can be difficult) Works with Cucumber, my current favorite acceptance test tool Supports multiple browsers (IE, Firefox, Safari, and soon Chrome and Opera)	Uses a language unfamiliar to .NET teams (that’s who we’re talking about here, remember) Doesn’t directly work with .NET test frameworks (of course, you can wrap the Watir tests for use by MSTest or NUnit)

The Bottom Line

On balance, I think Watir is the better tool, has the better documentation, the most active community, and the best long-term prospects. But I understand the desire to standardize on C# or VB.NET for both production and test code. The long-term cost of adding another language to a product is not something to ignore.

Personally, I’ll continue to use and teach both tools with clients. When it comes to my own projects, however, I’ll be using and extending Watir, for all the reasons above and simply because I enjoy working with Ruby.

Which do you use and why?

A Common, but Bad, Idea

Richard Lawrence — Fri, 14 Nov 2008 04:44:09 +0000

Please don’t do this:

It will hurt. Not right away, but around sprint 4 when the bugs found in sprint 3 for the stories built in sprint 2 need to be mixed in with the stories planned in sprint 3 for sprint 4. And then it’ll hurt more in sprint 5 and later when you have to start regression testing more and more code each sprint.

Instead, just do as much work as you can get really DONE in each sprint. Plan it, build it, test it, and deliver it. It’ll be hard to ensure that testing doesn’t get squeezed out the back of the sprint. It will. When that happens, try moving testing to the front of each story. Create your test cases as specifications and then do just enough development to make them pass. (Consider a tool like Cucumber for this, using it to drive Watir if you’re building a web app. Or look at Fit/Fitnesse if that’s more your style.)