XP Testing a GUI with FIT, Fitnesse, and Abbot

Joseph Bergin
Pace University

In Extreme Programming, one of the core processes is test first programming. There are frameworks, like JUnit (Erich Gamma and Kent Beck, http://junit.org), that let the developers on a team build unit tests as they go to get confidence in their build. There is also a new acceptance testing framework called FIT (Framework for Integrated Test), developed by Ward Cunningham, that can be used by the customer to build tests easily using spreadsheet tables saved as html. FIT parses html pages looking for tables and when it finds them looks for commands there that it knows about and executes the tests described there.

FIT requires some simple glue code (fixtures) be written that stand between an html table and the production code being developed (the "build"). Both JUnit and FIT have the property that the code for testing is not part of the build, but in separate Java (say) classes. Actually, these tools can be used for development in languages other than Java. There are "units" for most languages, and FIT is not language dependent, though built in Java. For other frameworks see http://www.xprogramming.com/software.htm

JUnit and FIT have been enhanced in some interesting ways. Bob Martin (Object Mentor) has built a web based wrapper around FIT that provides a wiki for easy acceptance test development and is also executes FIT. Even more interesting is that you can use it to build a full suite of tests on several hierarchical html/wiki pages that are run by pushing a single button. You get a very graphical report as to which tests pass and which do not. It is important to note that the tests can be written before any code is built in the application.

JUnit has also be enhanced with a tool called Abbot (A Better Bot) that can be used to automate GUI testing. It can be used in either one of two modes. A user can directly manipulate an existing GUI and Abbot will build a script that can be replayed later. It can also be used in test-first mode to build tests for GUI functionality not yet built. It is, however, mostly intended for developer use. I believe this project is headed by Timothy Wall.

Here we will put FIT, Fitnesse, and Abbot together to see test-first customer level testing done for GUI elements of an application

I assume you want to write tests first, before you write the GUI code, or even the application (model) code.

Getting Set Up

Get FIT from http://fit.c2.com. You need at least fit.jar from this package. Add this jar to your project. Actually, the fit.jar is included in the fitnesse piece shown below, but you will want to read the FIT documentation in any case. In your reading focus on the

ActionFixture

Get fitnesse from http://fitnesse.org. You will need to run its server somewhere. I run it locally at a convenient port. It will give you a wiki with an integrated FIT framework.There is a startup file for the server that you can edit by adding a port option unless you are happy with the default port 80. Once it is running open your browser and point it to (say) http://localhost:80. You can then use it to learn about itself. It is a wiki that contains its own documentation. In your reading, pay attention to the following points:

The simple syntax for adding a table to a page (using | between elements) OR using paste from another application.
How to name sub pages of an existing page (prefix with ^)
How to set up your ClassPath (using the !path function)
General wiki editing rules.
Page properties, especially the test and suite properties

Get Abbot from http://abbot.sourceforge.net. This includes a script generator that we don't use here, but you might find helpful. You want at least abbot.jar and jdom-1.0b8.jar from this package. Other jars might be necessary depending on what you do. Add these to your project also.

Planning

When you write your GUI code (after writing tests, of course) plan on giving every component that you might want to test a name with setName(String). This is necessary. If you use really good variable names for the components then the name you set for the component can be the same as the name you give the variable that points to it. These names should be unique to each component. The customer won't see these, however, so they don't need to be truly "external" names.

Here we will describe only how to use FIT and Fitnesse with Abbot. Each of these tools has much more functionality than we use here, but our goal is to provide an easy to use acceptance test framework for the GUI elements, that a customer can use.

Here the scenario is a very simple calculator built with separation between the model and the GUI. The GUI is called Calculator.java. Its listeners make calls to the underlying model (CalculatorModel.java). For the test we do here we need at least buttons for two, three, and five, as well as plus, minus, and equals keys. We also need a display value for the result. Assume you have stories for some of the functionality, and that the user has drawn you a sketch of the GUI. We focus on the sketch here.

Starting

Using some tool that can write html pages (word, excel, dreamweaver, fitnesse, ...) write a test using a FIT ActionFixture that will press your buttons etc. in the GUI. Below is a fitnesse test copied from the page edit form. (It doesn't test everything.) Note that in the fitnesse wiki !- and -! will prevent the wiki from treating the enclosed as wiki words. This is necessary here since the class names are "bumpy."

|!-fit.ActionFixture-! |
|start | !-calculator2003.CalculatorGuiFixture-!|
|enter | delay| 500 |
|check | value | 0 |
|press| five||
|press|three||
|press| plus||
|press| five||
|press|equals||
|check|value|58|
|press| minus||
|press|two||
|press|equals||
|check|value|56|

And here is the equivalent html captured from the browser using "Show Source":

<table border="1" cellspacing="0">
< tr><td colspan="3">fit.ActionFixture</td></tr>
< tr><td>start</td><td colspan="2">calculator2003.CalculatorGuiFixture</td></tr>
<tr><td>enter</td><td>delay</td><td>500</td></tr>
< tr><td>check</td><td>value</td><td>0</td></tr>
< tr><td>press</td><td>five</td><td></td></tr>
< tr><td>press</td><td>three</td><td></td></tr>
< tr><td>press</td><td>plus</td><td></td></tr>
< tr><td>press</td><td>five</td><td></td></tr>
< tr><td>press</td><td>equals</td><td></td></tr>
< tr><td>check</td><td>value</td><td>58</td></tr>
< tr><td>press</td><td>minus</td><td></td></tr>
< tr><td>press</td><td>two</td><td></td></tr>
< tr><td>press</td><td>equals</td><td></td></tr>
< tr><td>check</td><td>value</td><td>56</td></tr>
< /table>

And here is what it looks like rendered by a browser.

fit.ActionFixture

start calculator2003.CalculatorGuiFixture

enter delay 500

check value 0

press five

press three

press plus

press five

press equals

check value 58

press minus

press two

press equals

check value 56

The first row tells FIT what kind of table this is and fit.ActionFixture is from fit.jar. It exists. The start command on the second row tells which class is used as "glue" between the html page that contains this test and the build. Here it is calculator2003.CalculatorGuiFixture. This need not exist yet, though if you run the test before it does you will get exceptions. The exceptions will be shown in a table like the above where the exception is in the cell of the element that generated the exception. As usual in XP testing you are encouraged to run every test when you write it even before you build the functionality it tests. Fitnesse provides a test button to run tests, though you need to enable it for each page via the Fitnesse properties link. When you click the test button, the page is given to FIT and the results displayed.

The enter, check, and press functions (along with start) are part of an ActionFixture. Use enter to call a void method of one argument. Use press to call a void function of no arguments, and check to call a method that returns a value. The methods called are defined in the fixture as we see below. The second column contains names of these functions from CalculatorGuiFixture. These may not exist yet, but you are committing to function names for them here. But note that the CalculatorGuiFixture is not part of the build, so you are not constraining the production code in any way. Here five is a void function with no arguments and value is a function with no arguments that returns something.

FIT knows how to translate quite a lot of things into and out of a form suitable for the table, but occasionally you need to write a parser along with your fixture so that you can input real objects from the string representation in the table. Also, everything is open source so it can be extended, of course.

Conceptually enter is similar to entering information into fields of the GUI, press is similar to pushing buttons and keys, and check verifies that fields, labels, and the like have the correct values. We want to make this "metaphor" real.

If you run the test it will fail of course, since you haven't written any code yet.

This is what you get if you test now when nothing exists but the test itself:

fit.ActionFixture
start java.lang.ClassNotFoundException: calculator2003.CalculatorGuiRobotFixture at java.net.URLClassLoader$1.run(URLClassLoader.java:198) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:186) at java.lang.ClassLoader.loadClass(ClassLoader.java:299) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:265) at java.lang.ClassLoader.loadClass(ClassLoader.java:255) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:315) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:140) at fit.ActionFixture.start(ActionFixture.java:28) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	calculator2003.CalculatorGuiRobotFixture
enter java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.enter(ActionFixture.java:32) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	delay	500
check java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.check(ActionFixture.java:44) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	value	0
press java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.press(ActionFixture.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	five
press java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.press(ActionFixture.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	three
press java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.press(ActionFixture.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	plus
press java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.press(ActionFixture.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	five
press java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.press(ActionFixture.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	equals
check java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.check(ActionFixture.java:44) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	value	58
press java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.press(ActionFixture.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	minus
press java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.press(ActionFixture.java:40) at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	two
press java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.press(ActionFixture.java:40) at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	equals
check java.lang.NullPointerException at fit.ActionFixture.method(ActionFixture.java:55) at fit.ActionFixture.method(ActionFixture.java:51) at fit.ActionFixture.check(ActionFixture.java:44) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at fit.ActionFixture.doCells(ActionFixture.java:19) at fit.Fixture.doRow(Fixture.java:94) at fit.Fixture.doRows(Fixture.java:88) at fit.Fixture.doTable(Fixture.java:82) at fit.Fixture.doTables(Fixture.java:72) at fitnesse.FitFilter.process(Unknown Source) at fitnesse.FitFilter.run(Unknown Source) at fitnesse.FitFilter.main(Unknown Source)	value	56

See examples of successful and unsuccessful tests at the bottom of this page. But until you have enough infrastructure in place you will get some variation of the above.

First Some Background

Both FIT and Abbot use Java's Reflection API to look at a running application and obtain information from it. Abbot, for example looks around the running application to find components with the names we give them with setName. This even works with private GUI fields. Usually the objects in the GUI we test were declared private. However, public, private, etc. is a property of the variables not the objects that the reference variables point to. In any program it is possible to have both public and private fields point to the same objects. Once the objects exist, and have names (with setName), the Java reflection API can look around for objects with the names we gave them (ComponentReference does this). If it finds such an object it can point to it from a variable declared in the fixture. This doesn't exactly break encapsulation (though it is close), because you can't use this to change the values of the private variables in the GUI, or even see them if you don't already know the variable names. But you can get access to the objects they point to and send them whatever messages they know. If we know the names of the variables in the GUI we can use the same names in the fixture if we want, though that isn't necessary.

Likewise FIT uses reflection to match the names we give in the second column with methods in the fixture we write. You can ask retrieve a class at runtime if you know its name, and you can retrieve the methods from the class object.

Building the Fixture

Next, write a fixture class corresponding to this sample test. For this example it is CalculatorGuiFixture.java. It is the glue between your test and your application's GUI. Note that FIT is used for the test glue and Abbot is used only for the robot generating reflection code it provides. The second column of the table defines what methods you need to write in this class along with some infrastructure.

Note that fixture classes are not encapsulated. Everything is public, making it easy to connect up the parts using Java reflection. The names in the press and check slots of the table are matched by functions in the fixture. An enter will call a method with a single argument, press calls one with no argument, and check calls one with no argument that returns a value. Here we assume all arguments and return values are strings. If this is not the case you can provide parsing methods for input arguments and use toString for return values. FIT has some infrastructure to parse Java's built in types such as int, but for your own you need to write the parse "glue." This isn't likely to be needed in GUI testing as the Java Components deal mostly with String data. But it might be needed for other kinds of customer driven FIT testing.

 
package calculator2003;

import java.awt.*;
import fit.Fixture;
import junit.extensions.abbot.*;
import abbot.script.ComponentReference;
import abbot.tester.ComponentTester;

public class CalculatorGuiFixture extends Fixture // from FIT
{
	public Calculator calc = new Calculator(); // The GUI to be tested

	Button button5 = null; // References to objects in the GUI
	Button button2 = null;
	Button button3 = null;
	Button buttonEquals = null;
	Button buttonPlus = null;
	Button buttonMinus = null;
	TextField display = null;
	ComponentTester testBasic = null;
	
	public CalculatorGuiRobotFixture() throws Exception
	{	GuiTest test = new GuiTest("Calculator");
		test.setUp();
	}
	
	class GuiTest extends ComponentTestFixture // From Abbot's JUnit extensions
	{	public GuiTest(String name)
		{	super(name);
		}
		
		public void setUp() throws Exception
		{
			ComponentReference ref = new ComponentReference("twoButton", Button.class, "twoButton", "2");
			button2 = (Button)getFinder().findComponent(ref);
			testBasic = new ComponentTester();
			
			ref = new ComponentReference("threeButton", Button.class, "threeButton", "3");
			button3 = (Button)getFinder().findComponent(ref);
			
			ref = new ComponentReference("fiveButton", Button.class, "fiveButton", "5");
			button5 = (Button)getFinder().findComponent(ref);
			
			ref = new ComponentReference("equalsButton", Button.class, "equalsButton", "equals");
			buttonEquals = (Button)getFinder().findComponent(ref);
			
			ref = new ComponentReference("plusButton", Button.class, "plusButton", "plus");
			buttonPlus = (Button)getFinder().findComponent(ref);
			
			ref = new ComponentReference("minusButton", Button.class, "minusButton", "minus");
			buttonMinus = (Button)getFinder().findComponent(ref);
			
			ref = new ComponentReference("display", TextField.class);
			display = (TextField)getFinder().findComponent(ref);			
		}
	}

	private int delay = 0;

	public void delay(int d) // Control the speed of the "robot". 
	{	delay = d;
	}
	
	private void click(Button button)
	{	testBasic.actionClick(button);	// robot clicks the button, firing action events
		testBasic.actionDelay(delay);
	}

	public void five() throws Exception  // The "glue" methods referenced from the table. 
	{	click(button5);
	}
	public void three() throws Exception
	{	click(button3);
	}
	public void two() throws Exception
	{	click(button2);
	}
	public void equals() throws Exception
	{	click(buttonEquals);
	}
	public void plus() throws Exception
	{	click(buttonPlus);
	}
	public void minus() throws Exception
	{	click(buttonMinus);
	}
	public String value() throws Exception
	{	testBasic.actionDelay(delay);
		return display.getText();
	}

}

Note that the Abbot stuff is in an inner class within the fixture. It's job is to find components of the GUI that you want to test and then builds robots (Testers) that can manipulate them. Note that a ComponentTester has a very thin interface, but it works ok for buttons and TextFields and the like. Abbot provides a much more complete interface to Swing AND AWT components and you can also create tester objects with a rich interface if the infrastructure is there. For example, a ChoiceTester is used to test a java.awt.Choice. If you write your own component classes you will need to extend this framework as well.

The outer class creates an instance of the GUI to be tested and declares variables for both the GUI components to be exercised AND the testers that will be used to exercise them. Note that a Button in Java has no push method, but if we want to really exercise it, we need to be able to fire events just as if it is pushed. This is the purpose of the robot testers. Here we need only one robot (testBasic) for it will be able to exercise all the buttons (and text fields as well if we need that, though we don't here.) For more specialized components, such as java.awt.Choice you will want the more specialized robot. See the Abbot documentation.

Note that many of the methods and constructors here throw Exception. This is intended and not just sloppy programming. You don't want to handle exceptions in your tests, but rather pass them through to the test frameworks so that you can see them.

While the outer class declares components and robots, it is the inner class that gives them values in its setUp method. The outer class creates an instance of the inner one in its constructor and fires the setUp method to make this happen. It is important to do this in a "no argument" constructor, since this will constructor be called from the FIT framework, via the start function in the table.

For each component, we first need a ComponentReference and we then need to use the reference to find the actual component. This search for components uses the names we will define in the setName messages we will build in to the GUI. So you are making a committment here to component names, though not to the variable names that will be used in the GUI. But you are not constraining anything about layout or behavior. The second string in

ref = new ComponentReference("plusButton",   Button.class, "plusButton", "plus");

is the name you assign to the button with setName in the GUI. The other strings don't have any impact on this, but that one is essential. There is a simpler form you can use if you have only one component of a given class but with multiple buttons (or whatever) you will get an exception if you don't name them and point to them by name.

We also create one ComponentTester (the most general kind) that will exercise our buttons. Note that you don't need a tester for each component, though you may need more than one if you have some more specialized kinds of components in your GUI.

We used an inner class for the Abbot derived code to give us direct access to the component variables defined in the outer Fixture. However, this makes it harder to reuse this class for the purpose for which Abbot was built. It is, in fact, a JUnit TestCase and defines various useful assertions about the GUI that we don't use here.

You now have access to the components themselves and can send them any messages. You can also use the robot (testBasic) to push the buttons and fire corresponding events. Here we manipulate the TextFields directly (getText). On the Macintosh a delay is necessary and it needs to be fairly big. The advantage is that it seems like an invisible person is pushing buttons, but it slows testing accordingly. Without the delay (try it) the test fails on the Mac and the GUI doesn't perform correctly as the messages get lost. There seems to be a race condition somewhere that isn't handled properly. Each robot message runs in a new thread. They are supposed to sync but something is happening (at least on the Mac).It seems to work elsewhere without the delay. We have put a call to the delay in the table, though this is a property of the test and not of the GUI. Putting it in the table lets you manipulate it like any other test data.

Note that once all the parts are in place, including the build, the GUI will actually appear on the screen when you run a test and with a delay you will be able to follow the "mouse" as it pushes the buttons.

Once you have a Tester and references to the GUI components, you can write the methods of the outer Fixture that are fired by FIT. These are the ones named in the second column of our test table: five, plus, etc.

Now that you have the fixture you can run the test, but it will still fail until you write the GUI itself. The result will be like the one above, except that the exception in the first box will be something like

Unresolved compilation problem: 
 Calculator cannot be resolved or is not a type

Now you can write your GUI. Don't forget to name every component with setName using the same name you used above in the glue code.

After you build your GUI and its model you can (hopefully) run the test successfully. And once you have one test running successfully, you can write lots of others using the same framework. Note that except for the requirement to name your components with setName in the GUI, there is no test code in the GUI class or the underlying model system. It is all in the fixture.

When you push the test button in fitnesse, the GUI will pop up and the test will begin as laid out in the table above. Here is an example. Note that the GUI is not yet complete, but of course we test as we go, so that is expected. We also assume enough of the model has been built to support the keys in the GUI.

Simple Calc Gui picture

If the test runs successfully we will see the table below in a new html page. Green indicates a successful test, just as it does with JUnit.

fit.ActionFixture

start calculator2003.CalculatorGuiRobotFixture

enter delay 500

check value 0

press five

press three

press plus

press five

press equals

check value 58

press minus

press two

press equals

check value 56

But if you have an error, you will see something like the following. Here the minus functionality is broken.

Get the Code

The code is available for this Polymorphic Calculator along with the fixture that is discussed here, the GUI and the model.

Final Notes

The customers can use FIT/fitnesse to write stories and tests for the build. They can work far in advance of the developers and in parallel with the developers. They need someone to build fixtures for them, of course. If the stories/tests are arranged in a suite, you can get a quick picture of where you are with respect to acceptance at any time, by pushing one button. As things get built correctly you get more and more green. The developers can also look at the tests to refine their ideas about the meaning of the stories. Just as the developers aren't done until they can't think of anything else to test with JUnit, the customers aren't done until they can't think of anything else they would want before accepting the build. As always, boundary conditions need special attention.

Fitnesse has a simple syntax for creating html tables as shown above, but it also has a way to paste tables directly from MS/Word and Excel. Since most customers know these tools, it can be very easy for them to create tests once they have fixtures.

FIT doesn't actually need Fitnesse, but the latter adds value in its ability to run a suite represented by a hierarchical set of wiki pages all at once. You can use FIT on the command line, however, or directly from your IDE. FIT takes an html page as an argument and produces tables like those shown above. It parses the page looking for tables, but preserving everything else so the page it returns looks like the original but for the updated tables. You can also define a simple cgi script that will run FIT with the contents of the page that contains a link to the script. This assumes, of course, that you run a web server suitable for cgi scripts (Apache, for example). So if the page with the tables also contains a link to runfit.cgi, for example, and the script has the right glue, the script will process that page and return it to the browser. See http://fit.c2.com for details.

It is interesting to write the customer stories in fitnesse and have the customer acceptance tests for the story just below. You can have a page for each story if you like, arranged in a hierarchy. Fitnesse even has a catalog function (!contents) so that you can easily see the sub pages from any point in your hierarchy. But make sure you don't have one version of a story on a 3 by 5 card and a different version in the test page.

Caveats

There are a few things to consider as you build the GUI code and fixtures. First is that if you have several GUIs to test, your "setName" names need to be distinct over the application, not just over each class. Otherwise Abbot will get naming conflicts. Second, when you enter something into a field in the fixture, you may also want to click on that field so that it gets focus, as it would when a user manipulates it. If you use keystroke actions in the fixture, note that you need to first give focus to the component that you want to take the strokes. RequestFocus is not implmented the same (or correctly, I think) in all VMs. ActionClicking in the component may be more successful than requestFocus.

More important, perhaps, is that if you want to test lots of GUIs in a Fitnesse suite, then its success depends to a great extent on the thread model of the underlying OS. It may be that all of the GUIs are thrown up at once in separate threads. If they also overlay one another, then mouse clicks intended for one will be taken by the one in front instead. This will lead to failed tests. I've not yet found a solution to this.

Fitnesse and Abbot also seem to have an interesting interaction on Windows/XP at least. When a GUI comes up it may not come up as the front window (behind the browser window). Fitnesse seems to loop in this case, bringing up the window again and again seeking to get a successful run. If you manually click on the window (it is only there momentarily) you will get a successful run as it will be brought to the front. I don't have a really good solution to this either, except that putting a mouse click in the fitnesse table seems to help.

Last Updated: November 24, 2003

fit.ActionFixture
start	calculator2003.CalculatorGuiFixture
enter	delay	500
check	value	0
press	five
press	three
press	plus
press	five
press	equals
check	value	58
press	minus
press	two
press	equals
check	value	56