Richard Lawrence » WatiN

Debugging Cuke4Nuke Step Definitions

Richard Lawrence — Wed, 13 Jan 2010 02:52:51 +0000

At a client today, we were doing some tricky automation with WatiN (against Telerik controls) in Cuke4Nuke. We wanted to dig into what WatiN was finding in the browser. The obvious move was to fire up the debugger. But with Cuke4Nuke, this is trickier than you might think. The Cuke4Nuke gem includes a release build—no debug symbols. Plus, the process isn’t around long enough to attach to if you use the cuke4nuke executable. We probably could have figured it out, but it wasn’t worth the trouble.

Since we weren’t debugging the Cucumber side of things, we didn’t need all the Cucumber and Cuke4Nuke plumbing. We just needed to run the code using WatiN.

NUnit and Resharper to the rescue…Cuke4Nuke cares about Given, When, Then, Before, and After attributes on your methods. But that doesn’t mean those are the only attributes allowed.

We added a TestFixture attribute to the class. We put SetUp and TearDown attributes on the Before and After methods. We created a new method with a Test attribute that called the relevant step definition methods. Set a breakpoint, click the Resharper test icon and choose Debug, and next thing we knew we were debugging our way through the step definition code.

Next time you need to debug a tricky step definition, don’t bother trying to get Cuke4Nuke in the debugger. Just use your unit test framework to call your step definitions. It’s faster to set up and faster to run.

How to Remove Duplication in Cucumber Tests Using Scenario Outlines

Richard Lawrence — Mon, 04 Jan 2010 17:09:36 +0000

Gojko Adzic has a new blog post demonstrating the new table parameter support in Cuke4Nuke. Table parameters are an important part of Cucumber. They’re great for setting up data and asserting that lists are what you expect. But I use them much less often than the other kind of table in Cucumber, scenario outlines.

Scenario outlines are the solution to repetitive Given-When-Then scenarios. Extending the WatiN example in my previous screencast, we might find ourselves writing scenarios like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

Scenario: Search - richard lawrence
Given I'm on the search page
When I search for "richard lawrence"
Then I should see "www.richardlawrence.info" in the results

Scenario: Search - pickaxe
Given I'm on the search page
When I search for "pickaxe"
Then I should see "www.ruby-doc.org/docs/ProgrammingRuby/" in the results

Scenario: Search - cuke4nuke
Given I'm on the search page
When I search for "cuke4nuke"
Then I should see "github.com/richardlawrence/Cuke4Nuke" in the results

This kind of repetition gets old fast, both to write and to read. Scenario outlines allow us to separate the structure of the test, which doesn’t change, from the data, which does. Here are the same scenarios refactored:

1
2
3
4
5
6
7
8
9
10

Cucumber turns each example—each table row—into a concrete scenario before looking for matching step definitions in Cuke4Nuke, so nothing needs to change on the Cuke4Nuke side.

In most cases, I recommend starting with concrete scenarios and refactoring to scenario outlines once you have a few scenarios working with the correct step definitions.

The Latest on Cuke4Nuke

Richard Lawrence — Wed, 30 Dec 2009 17:42:38 +0000

This morning, I released version 0.3.0 of Cuke4Nuke. With this release, Cuke4Nuke supports almost everything you can do with Cucumber in Ruby or Java, making C# a first class language for Cucumber. (The only missing features are small things like tags on Before and After hooks and a richer Table object.) Check out the Cuke4Nuke wiki for instructions to install and get started with Cucumber in .NET. To see it in action, check out my screencast on Cuke4Nuke and WatiN.

Screencast: Testing Web Applications in .NET with Cuke4Nuke and WatiN

Richard Lawrence — Thu, 03 Dec 2009 15:53:41 +0000

Yesterday, I released Cuke4Nuke 0.2.2, which added WatiN compatibility and an example of how to use the two tools together. Here’s a short screencast in which I walk through the example:

Resources from the video:

The Cuke4Nuke Wiki with installation instructions
WatiN
Source code from the example
The new Ruby installer

WatiN Patterns #3: Don’t Over-specify

Richard Lawrence — Fri, 30 Oct 2009 15:20:31 +0000

After a long hiatus, I’m resuming the WatiN Patterns series. Pattern #1 covered why and how your tests should clean up after themselves. Pattern #2 covered how you should name your tests and why they should only assert one thing.

Pattern #3 is about keeping your tests maintainable by specifying just enough in your element selectors.

If you’re using WatiN, you’re probably also using ASP.NET Web Forms, which means your controls get IDs like this: ctl1001_Content_Content_txtUsername. Though you gave your username text box the ID “txtUsername”, ASP.NET prepends the IDs of each of the container controls to your text box’s ID in the HTML that goes to the browser. This is good for preventing name collisions between controls in different user controls on the same page. But it can lead you to write fragile tests. Consider this example:

1
2
3
4
5
6
7
8
9
10
11
12

[Test]
public void ShouldAuthenticateUsersSuccessfully()
{
using (IE ie = new IE())
{
ie.GoTo("http://mytestapp/");
ie.TextField("ctl1001_Content_Content_txtUsername").TypeText("richard");
ie.TextField("ctl1001_Content_Content_txtPassword").TypeText("password");
ie.Button("ctl1001_Content_Content_btnLogin").Click();
Assert.IsTrue(ie.ContainsText("Welcome, Richard!"));
}
}

Suppose someone moves the login widgets outside one of those nested “Content” controls. Now our test fails. Did login stop working? Probably not. But, thanks to ASP.NET’s “helpful” ID scheme, we can no longer find the login widgets—their IDs have changed. We could update the test to the new IDs:

1
2
3
4
5
6
7
8
9
10
11
12

[Test]
public void ShouldAuthenticateUsersSuccessfully()
{
using (IE ie = new IE())
{
ie.GoTo("http://mytestapp/");
ie.TextField("ctl1001_Content_txtUsername").TypeText("richard");
ie.TextField("ctl1001_Content_txtPassword").TypeText("password");
ie.Button("ctl1001_Content_btnLogin").Click();
Assert.IsTrue(ie.ContainsText("Welcome, Richard!"));
}
}

Or we could solve the root problem of fragile tests:

1
2
3
4
5
6
7
8
9
10
11
12

[Test]
public void ShouldAuthenticateUsersSuccessfully()
{
using (IE ie = new IE())
{
ie.GoTo("http://mytestapp/");
ie.TextField(new Regex("txtUsername$")).TypeText("richard");
ie.TextField(new Regex("txtPassword$")).TypeText("password");
ie.Button(new Regex("btnLogin$")).Click();
Assert.IsTrue(ie.ContainsText("Welcome, Richard!"));
}
}

WatiN accommodates regular expressions instead of strings for IDs. Now we’ve specified, “a text field with an ID ending in txtUsername” (note the ending dollar sign), instead of, “a text field with exactly the ID ctl1001_Content_Content_txtUsername.” Much more flexible.

To keep your tests from being fragile, when selecting elements on the page, use the least specific method and value necessary to find the element you want.

WatiN Patterns #2: One Assertion and a Name to Match

Richard Lawrence — Wed, 11 Feb 2009 23:46:19 +0000

One way to keep your WatiN tests maintainable is to keep them small and focused. WatiN Pattern #2, then, is a way to do just that.

Your tests should assert one thing. Just one. And the name of the test should describe whatever it is you’re asserting. Looking at our example from last time, we see that we’ve got the one assertion part right, but the name doesn’t match.

1
2
3
4
5
6
7
8
9
10
11

[TestMethod]
public void SearchPageTest()
{
using (IE ie = new IE())
{
ie.GoTo("http://www.google.com/");
ie.TextField(Find.ByName("q")).TypeText("Richard Lawrence");
ie.Button(Find.ByName("btnG")).Click();
Assert.IsTrue(ie.ContainsText("www.richardlawrence.info"));
}
}

In a nod to the BDD approach (more BDD to show up in future patterns), I like to name the test class for the thing we’re testing and use test names starting with “Should” that specify what the thing we’re testing ought to do. So, we could change our example to something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

// using statements, namespace declaration, etc.
// ...
[TestClass]
public class GoogleSearchPage
{

// ...

[TestMethod]
public void ShouldReturnSearchResultsSuccessfully()
{
using (IE ie = new IE())
{
ie.GoTo("http://www.google.com/");
ie.TextField(Find.ByName("q")).TypeText("Richard Lawrence");
ie.Button(Find.ByName("btnG")).Click();
Assert.IsTrue(ie.ContainsText("www.richardlawrence.info"));
}
}
}

Now we have one assertion, and our test name indicates that it’s asserting that the Google search page should return search results successfully. If that test fails, we know what’s gone wrong.

It’s not yet transparent that the assertion on line 17 does what the test name indicates, but we’ll deal with that in future patterns.

WatiN Patterns #1: No Browser Left Behind

Richard Lawrence — Wed, 04 Feb 2009 23:35:44 +0000

In my previous posts on WatiN, I lamented the shortage of online documentation and resolved to do something about it by documenting the patterns I’ve found for good WatiN tests. This is the first in a series in which I’ll take an example of the typical beginner WatiN test I see and refactor it to use the patterns I recommend.

Consider this test:

1
2
3
4
5
6
7
8
9
10

[TestMethod]
public void SearchPageTest()
{
IE ie = new IE();
ie.GoTo("http://www.google.com/");
ie.TextField(Find.ByName("q")).TypeText("Richard Lawrence");
ie.Button(Find.ByName("btnG")).Click();
Assert.IsTrue(ie.ContainsText("www.richardlawrence.info"));
ie.Close();
}

It has a few problems, but I want to highlight one in this post: This test doesn’t properly clean up after itself. If the code on lines 66-70 throws an exception or if the assertion fails, line 71 will never be executed. When you’re testing on your own machine, this can be annoying—you have to close the IE window manually. But if the test runs unattended (e.g. as part of continuous integration) it can be much more than annoying.

I’ve sometimes seen this attempt to work around the problem:

1
2
3
4
5
6
7
8
9
10
11

[TestMethod]
public void SearchPageTest()
{
IE ie = new IE();
ie.GoTo("http://www.google.com/");
ie.TextField(Find.ByName("q")).TypeText("Richard Lawrence");
ie.Button(Find.ByName("btnG")).Click();
bool resultsFound = ie.ContainsText("www.richardlawrence.info");
ie.Close();
Assert.IsTrue(resultsFound);
}

Introducing a local boolean to indicate success and closing the browser before the assertion ensures that the browser gets closed whether the assertion fails or succeeds. However, it won’t necessarily close the browser in case of an exception.

Fortunately, WatiN.Core.IE implements IDisposable, so you can do this:

1
2
3
4
5
6
7
8
9
10
11

In case you’re not familar with the “using” statement, it’s a shortcut for this:

1
2
3
4
5
6
7
8
9
10
11
12
13

IE ie;
try
{
ie = new IE();
// whatever you're going to do with ie
}
finally
{
if (ie != null)
{
ie.Dispose();
}
}

At the end of the using block (or when an exception is thrown) ie.Dispose() is called, which closes the browser window.

Use this pattern in your WatiN tests to ensure that your tests clean up after themselves and leave no browser window behind.

Another Look at WatiN

Richard Lawrence — Sat, 24 Jan 2009 16:29:50 +0000

At my current client, we’ve decided to use WatiN, largely for the C# vs. Ruby reason I discussed earlier this week. After spending a week working with WatiN (following a year of rarely using it), I’m impressed. Ruby and the active Watir community still have their advantages. But WatiN has really come into its own with C# 3.0 features like lambdas. I’m pleased with the test code we’re producing in terms of readability, speed, flexibility, and maintainability. I’ve proposed an Agile 2009 tutorial session on the patterns I’m using to get those results, and I’ll post more on that topic here soon. (Which will hopefully help with one of WatiN’s shortcomings—documentation.)

Web Testing for .NET Teams: WatiN or Watir?

Web Testing for .NET Teams: WatiN or Watir?

Richard Lawrence — Tue, 20 Jan 2009 03:49:04 +0000

I’ve noticed a pattern with several of my .NET clients who want to get into automated acceptance testing for web applications. They like the idea of WatiN because it would let them write tests in the same language as their production code. But then they notice that there’s much more documentation and apparently a much more active community around Watir. And that Ruby language looks interesting too. What to do?

I think there are good arguments for both. Here are the major pros and cons from my perspective…

WatiN

Pros	Cons
Code and test in the same language (e.g. C#) No need to learn a new language (i.e. Ruby) No need to support another language No need to hire people who know another language Did I mention that you can write tests in C#?	IE only (though there’s a CTP for FireFox support) Very limited documentation More syntactic noise (more due to .NET than to WatiN itself; e.g. new Regex(“txtFoo$”) vs. /txtFoo$/)

Watir

Pros	Cons
Very active community Active ongoing development on Watir and related projects Good support on the discussion list Concise, clear syntax (more due to Ruby than to Watir itself) Good documentation (though finding the current RDoc online can be difficult) Works with Cucumber, my current favorite acceptance test tool Supports multiple browsers (IE, Firefox, Safari, and soon Chrome and Opera)	Uses a language unfamiliar to .NET teams (that’s who we’re talking about here, remember) Doesn’t directly work with .NET test frameworks (of course, you can wrap the Watir tests for use by MSTest or NUnit)

The Bottom Line

On balance, I think Watir is the better tool, has the better documentation, the most active community, and the best long-term prospects. But I understand the desire to standardize on C# or VB.NET for both production and test code. The long-term cost of adding another language to a product is not something to ignore.

Personally, I’ll continue to use and teach both tools with clients. When it comes to my own projects, however, I’ll be using and extending Watir, for all the reasons above and simply because I enjoy working with Ruby.

Which do you use and why?