Richard Lawrence » .NET

The Future of Cucumber on .NET

Richard Lawrence — Fri, 21 Oct 2011 17:56:14 +0000

Beginning just over two years ago, I worked with some great developers to create Cuke4Nuke as a way to bring Cucumber to .NET. Shortly after the first releases of Cuke4Nuke, TechTalk released SpecFlow, a native .NET tool inspired by Cucumber. As a pure .NET solution without Ruby dependencies, SpecFlow seemed to be easier for Microsoft shops to adopt. But Cuke4Nuke had (to me) enough advantages that I kept it going.

By this summer, however, it was clear that SpecFlow had made up for most of Cuke4Nuke’s advantages and had more momentum in the .NET community and was under much more active development. There were just a few gaps SpecFlow needed to overcome (in my opinion), mostly related to better harmonization with the other Cucumber projects. (With the pure Java Cucumber-JVM under active development as a Cuke4Duke replacement, there was no longer as compelling an argument for shared code between Cucumber versions, Cuke4Nuke’s biggest remaining advantage.)

At AA-FTT and later in the week at Agile 2011, my colleague Paul Rayner and I met with Christian Hassa and Gaspar Nagy from the SpecFlow team. We discussed the few things that needed to change in SpecFlow to make it behave more like the other Cucumbers.

This week, most of those changes were released as part of SpecFlow 1.8. With that release, I’m letting Cuke4Nuke die. I won’t be maintaining or extending Cuke4Nuke. My Cucumber classes and book will cover SpecFlow rather than Cuke4Nuke.

I’ve had a great time getting to know the SpecFlow guys, and I look forward to collaborating with them in the future to help bring Cucumber to more and more teams in the Microsoft world.

Cuke4Nuke Needs Your Help

Richard Lawrence — Sat, 05 Mar 2011 18:04:45 +0000

Almost 2 years ago, at the 2009 AA-FTT conference, I started Cuke4Nuke to bring Cucumber to the .NET world. Since then, thanks to some fantastic help from Aslak Hellesøy, Matt Wynne, Declan Whelan, Åsmund Eldhuset, and others, Cuke4Nuke supports almost all the features of Cucumber.

But there are still a few key features it needs, and I have too many projects going to keep up with it. Specifically, I’d most like to see:

Support for VS2010 (.NET 4) and Mono
Step definition config files
Tagged hooks
An all-in-one installer like RailsInstaller
Better UTF-8 support
Culture support to go with the current language support (e.g. for string to decimal or date conversions)
Debugging for step definitions in Visual Studio
A completely automated build and release process for Cuke4Nuke itself
Updates to cuke4vs, which isn’t my project but which brings valuable Visual Studio integration to Cuke4Nuke

Are you a .NET and/or Ruby developer? Would you like to contribute? I’d love the help. Check out the instructions for contributing and jump in!

P.S. – Since someone always asks…Yes, I know about SpecFlow. I think it’s a great project. But I still believe there’s value in a .NET version of Cucumber that shares as much code and behavior as possible with the main Cucumber project.

Debugging Cuke4Nuke Step Definitions

Richard Lawrence — Wed, 13 Jan 2010 02:52:51 +0000

At a client today, we were doing some tricky automation with WatiN (against Telerik controls) in Cuke4Nuke. We wanted to dig into what WatiN was finding in the browser. The obvious move was to fire up the debugger. But with Cuke4Nuke, this is trickier than you might think. The Cuke4Nuke gem includes a release build—no debug symbols. Plus, the process isn’t around long enough to attach to if you use the cuke4nuke executable. We probably could have figured it out, but it wasn’t worth the trouble.

Since we weren’t debugging the Cucumber side of things, we didn’t need all the Cucumber and Cuke4Nuke plumbing. We just needed to run the code using WatiN.

NUnit and Resharper to the rescue…Cuke4Nuke cares about Given, When, Then, Before, and After attributes on your methods. But that doesn’t mean those are the only attributes allowed.

We added a TestFixture attribute to the class. We put SetUp and TearDown attributes on the Before and After methods. We created a new method with a Test attribute that called the relevant step definition methods. Set a breakpoint, click the Resharper test icon and choose Debug, and next thing we knew we were debugging our way through the step definition code.

Next time you need to debug a tricky step definition, don’t bother trying to get Cuke4Nuke in the debugger. Just use your unit test framework to call your step definitions. It’s faster to set up and faster to run.

The Latest on Cuke4Nuke

Richard Lawrence — Wed, 30 Dec 2009 17:42:38 +0000

This morning, I released version 0.3.0 of Cuke4Nuke. With this release, Cuke4Nuke supports almost everything you can do with Cucumber in Ruby or Java, making C# a first class language for Cucumber. (The only missing features are small things like tags on Before and After hooks and a richer Table object.) Check out the Cuke4Nuke wiki for instructions to install and get started with Cucumber in .NET. To see it in action, check out my screencast on Cuke4Nuke and WatiN.

WatiN Patterns #3: Don’t Over-specify

Richard Lawrence — Fri, 30 Oct 2009 15:20:31 +0000

After a long hiatus, I’m resuming the WatiN Patterns series. Pattern #1 covered why and how your tests should clean up after themselves. Pattern #2 covered how you should name your tests and why they should only assert one thing.

Pattern #3 is about keeping your tests maintainable by specifying just enough in your element selectors.

If you’re using WatiN, you’re probably also using ASP.NET Web Forms, which means your controls get IDs like this: ctl1001_Content_Content_txtUsername. Though you gave your username text box the ID “txtUsername”, ASP.NET prepends the IDs of each of the container controls to your text box’s ID in the HTML that goes to the browser. This is good for preventing name collisions between controls in different user controls on the same page. But it can lead you to write fragile tests. Consider this example:

1
2
3
4
5
6
7
8
9
10
11
12

[Test]
public void ShouldAuthenticateUsersSuccessfully()
{
using (IE ie = new IE())
{
ie.GoTo("http://mytestapp/");
ie.TextField("ctl1001_Content_Content_txtUsername").TypeText("richard");
ie.TextField("ctl1001_Content_Content_txtPassword").TypeText("password");
ie.Button("ctl1001_Content_Content_btnLogin").Click();
Assert.IsTrue(ie.ContainsText("Welcome, Richard!"));
}
}

Suppose someone moves the login widgets outside one of those nested “Content” controls. Now our test fails. Did login stop working? Probably not. But, thanks to ASP.NET’s “helpful” ID scheme, we can no longer find the login widgets—their IDs have changed. We could update the test to the new IDs:

1
2
3
4
5
6
7
8
9
10
11
12

[Test]
public void ShouldAuthenticateUsersSuccessfully()
{
using (IE ie = new IE())
{
ie.GoTo("http://mytestapp/");
ie.TextField("ctl1001_Content_txtUsername").TypeText("richard");
ie.TextField("ctl1001_Content_txtPassword").TypeText("password");
ie.Button("ctl1001_Content_btnLogin").Click();
Assert.IsTrue(ie.ContainsText("Welcome, Richard!"));
}
}

Or we could solve the root problem of fragile tests:

1
2
3
4
5
6
7
8
9
10
11
12

[Test]
public void ShouldAuthenticateUsersSuccessfully()
{
using (IE ie = new IE())
{
ie.GoTo("http://mytestapp/");
ie.TextField(new Regex("txtUsername$")).TypeText("richard");
ie.TextField(new Regex("txtPassword$")).TypeText("password");
ie.Button(new Regex("btnLogin$")).Click();
Assert.IsTrue(ie.ContainsText("Welcome, Richard!"));
}
}

WatiN accommodates regular expressions instead of strings for IDs. Now we’ve specified, “a text field with an ID ending in txtUsername” (note the ending dollar sign), instead of, “a text field with exactly the ID ctl1001_Content_Content_txtUsername.” Much more flexible.

To keep your tests from being fragile, when selecting elements on the page, use the least specific method and value necessary to find the element you want.

Cuke4Nuke: Cucumber for .NET Teams

Richard Lawrence — Sun, 20 Sep 2009 00:16:38 +0000

Update: If you’ve just landed here, you could get the impression from this post that Cuke4Nuke doesn’t exist yet. It does. Check out this screencast showing what you can do with it as of early December 2009.

If you’ve read this blog for a while or talked with me about functional test tools, you’ve heard me talk about Cucumber. It’s my favorite ATDD tool because it’s so good at mapping stories and acceptance criteria to automated functional tests. Product Owners and BAs write acceptance criteria in natural language. Developers and testers unobtrusively automate tests for them. Anyone on the team can run the tests and see the current state of the system.

Here’s a simple example:

1
2
3
4
5
6
7
8
9
10
11
12
13

Feature: Google search
In order to find things on the web
As a user
I want to search for web pages containing specific text

Scenario: Load search page
When I go to the search page
Then I should be on the search page

Scenario: Search
Given I'm on the search page
When I search for "richard lawrence"
Then I should see "www.richardlawrence.info" in the results

This reads almost exactly as I teach Product Owners to specify acceptance criteria. But it’s not just text. It’s a potentially automated test.

A developer or tester can come along and automate this test like so in Ruby:

1
2
3
4
5
6
7
8
9
10
11

# assuming @google is an instance of a test DSL that wraps Watir, Selenium, etc.

When /^I search for "(.*)"$/ do |query|
@google.search_for query
end

Then /^I should see "(.*)" in the results$/ do |expected_text|
assert { @google.results_contain? expected_text }
end

# etc...

Each of the Given/When/Then calls is a step definition. When there’s a matching line in a Cucumber test, the step definition gets executed.

Recently, support was added for step definitions in Java via a project called Cuke4Duke:

1
2
3
4
5
6
7
8
9
10
11

@When("I search for \"(*)\"")
public void search(String query) throws Exception
{
google.searchFor(query);
}

@Then("I should see \"(*)\" in the results")
public void checkResults(String expectedUrl)
{
assertThat(google.containsResult(expectedUrl), is(true));
}

Now, a Java team can use Cucumber without ever writing a line of Ruby.

Unfortunately, there’s nothing like this for .NET teams. Until now (or soon, anyway)…

AA-FTT and the Birth of Cuke4Nuke

Last month, I attended the Agile Alliance Functional Test Tool conference (AA-FTT for short). AA-FTT is an open space conference. I came with one goal: to get together with other people who want Cucumber for .NET and start making it happen. I wasn’t sure anyone else would be interested, so I was thrilled with the reaction.

Aslak Hellesøy introduced Cucumber for those in the group who hadn’t used it and then talked through the multiple language support he’d recently added to the tool. Then, we discussed ways to build .NET support. The obvious solution was to use IronRuby and Cucumber’s language support to handle C# step definitions. Matt Wynne downloaded the latest IronRuby and installed Cucumber on it. He kicked off a simple Cucumber example, and we waited. And waited. Some two minutes later, we had results from tests that take less than two seconds to run under the standard Ruby interpreter. In a process that values frequent, fast test runs, IronRuby was a non-starter. (If you want to try Cucumber under IronRuby, here are some instructions.)

So we discussed other options and settled on using a simple wire protocol for Cucumber to communicate to .NET out-of-process, similar to Slim in FitNesse. And here’s the best part: we started building it. Matt and I paired right there to start fleshing out the wire protocol (with Cucumber tests, naturally). Later in the week at Agile 2009, Matt and Aslak paired to build the Ruby side of the wire protocol, and Matt and I tackled the first bits of the .NET side. Last week, I got the skeleton of the .NET side working and up on GitHub. You can define simple steps in C#. Cucumber can ask the .NET wire server to tell it about the steps it has and to invoke them and return the pass/fail results.

About a week ago, Aslak announced the project on the Cucumber mailing list and recruited more contributors. Declan Whelan, Scott Ford, Åsmund Eldhuset, Anders Hammervold, Chris Kooken, and Steve Eley have already stepped up with ideas and code.

Getting Involed

How can you get involved? I’m glad you asked.

Join the mailing list.
Fork the GitHub repository and work on one of the features in the backlog.
Post a message to the mailing list to let us know what you’re working on. I’ll tag the ticket in the backlog with your GitHub username so we don’t duplicate effort. Prefix your message subject with [Cuke4Nuke].
Comment on tickets in the backlog.
Try using Cuke4Nuke as we develop it and give us feedback via the mailing list.
Shout encouragements in the comments here and on Twitter to let us know you care, even if you can’t contribute.

WatiN Patterns #2: One Assertion and a Name to Match

Richard Lawrence — Wed, 11 Feb 2009 23:46:19 +0000

One way to keep your WatiN tests maintainable is to keep them small and focused. WatiN Pattern #2, then, is a way to do just that.

Your tests should assert one thing. Just one. And the name of the test should describe whatever it is you’re asserting. Looking at our example from last time, we see that we’ve got the one assertion part right, but the name doesn’t match.

1
2
3
4
5
6
7
8
9
10
11

[TestMethod]
public void SearchPageTest()
{
using (IE ie = new IE())
{
ie.GoTo("http://www.google.com/");
ie.TextField(Find.ByName("q")).TypeText("Richard Lawrence");
ie.Button(Find.ByName("btnG")).Click();
Assert.IsTrue(ie.ContainsText("www.richardlawrence.info"));
}
}

In a nod to the BDD approach (more BDD to show up in future patterns), I like to name the test class for the thing we’re testing and use test names starting with “Should” that specify what the thing we’re testing ought to do. So, we could change our example to something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

// using statements, namespace declaration, etc.
// ...
[TestClass]
public class GoogleSearchPage
{

// ...

[TestMethod]
public void ShouldReturnSearchResultsSuccessfully()
{
using (IE ie = new IE())
{
ie.GoTo("http://www.google.com/");
ie.TextField(Find.ByName("q")).TypeText("Richard Lawrence");
ie.Button(Find.ByName("btnG")).Click();
Assert.IsTrue(ie.ContainsText("www.richardlawrence.info"));
}
}
}

Now we have one assertion, and our test name indicates that it’s asserting that the Google search page should return search results successfully. If that test fails, we know what’s gone wrong.

It’s not yet transparent that the assertion on line 17 does what the test name indicates, but we’ll deal with that in future patterns.

WatiN Patterns #1: No Browser Left Behind

Richard Lawrence — Wed, 04 Feb 2009 23:35:44 +0000

In my previous posts on WatiN, I lamented the shortage of online documentation and resolved to do something about it by documenting the patterns I’ve found for good WatiN tests. This is the first in a series in which I’ll take an example of the typical beginner WatiN test I see and refactor it to use the patterns I recommend.

Consider this test:

1
2
3
4
5
6
7
8
9
10

[TestMethod]
public void SearchPageTest()
{
IE ie = new IE();
ie.GoTo("http://www.google.com/");
ie.TextField(Find.ByName("q")).TypeText("Richard Lawrence");
ie.Button(Find.ByName("btnG")).Click();
Assert.IsTrue(ie.ContainsText("www.richardlawrence.info"));
ie.Close();
}

It has a few problems, but I want to highlight one in this post: This test doesn’t properly clean up after itself. If the code on lines 66-70 throws an exception or if the assertion fails, line 71 will never be executed. When you’re testing on your own machine, this can be annoying—you have to close the IE window manually. But if the test runs unattended (e.g. as part of continuous integration) it can be much more than annoying.

I’ve sometimes seen this attempt to work around the problem:

1
2
3
4
5
6
7
8
9
10
11

[TestMethod]
public void SearchPageTest()
{
IE ie = new IE();
ie.GoTo("http://www.google.com/");
ie.TextField(Find.ByName("q")).TypeText("Richard Lawrence");
ie.Button(Find.ByName("btnG")).Click();
bool resultsFound = ie.ContainsText("www.richardlawrence.info");
ie.Close();
Assert.IsTrue(resultsFound);
}

Introducing a local boolean to indicate success and closing the browser before the assertion ensures that the browser gets closed whether the assertion fails or succeeds. However, it won’t necessarily close the browser in case of an exception.

Fortunately, WatiN.Core.IE implements IDisposable, so you can do this:

1
2
3
4
5
6
7
8
9
10
11

In case you’re not familar with the “using” statement, it’s a shortcut for this:

1
2
3
4
5
6
7
8
9
10
11
12
13

IE ie;
try
{
ie = new IE();
// whatever you're going to do with ie
}
finally
{
if (ie != null)
{
ie.Dispose();
}
}

At the end of the using block (or when an exception is thrown) ie.Dispose() is called, which closes the browser window.

Use this pattern in your WatiN tests to ensure that your tests clean up after themselves and leave no browser window behind.

Another Look at WatiN

Richard Lawrence — Sat, 24 Jan 2009 16:29:50 +0000

At my current client, we’ve decided to use WatiN, largely for the C# vs. Ruby reason I discussed earlier this week. After spending a week working with WatiN (following a year of rarely using it), I’m impressed. Ruby and the active Watir community still have their advantages. But WatiN has really come into its own with C# 3.0 features like lambdas. I’m pleased with the test code we’re producing in terms of readability, speed, flexibility, and maintainability. I’ve proposed an Agile 2009 tutorial session on the patterns I’m using to get those results, and I’ll post more on that topic here soon. (Which will hopefully help with one of WatiN’s shortcomings—documentation.)

Web Testing for .NET Teams: WatiN or Watir?