Automated GUI testing with Dogtail

by Len DiMaggio

Coming in from the cold--GUI testing on Linux

If you've ever had to create automated tests for your applications' GUI, then you've generally been in luck, as there are several fine test packages to choose from--provided, of course, that you're working on Windows® and have lots of money to throw at the problem. If you're working on Linux, however, and don't have the budget of a Washington, D.C. lobbying firm, and you don't want to get locked into a closed, proprietary test architecture, then you've generally been out of luck.

That sad situation is changing, thanks to some folks (Zack Cerza, Ed Rousseau, and David Malcolm) at the Westford (Massachusetts, USA) Red Hat office and the open source automated test framework that they've initiated. The framework is called "Dogtail." This article describes Dogtail and walks you through creating and executing a simple Dogtail test script. (We'd also like to acknowledge the contributions of former Red Hat employee Chris Lee to the development of the initial Dogtail code, and the contributions of Red Hat employees Máirín Duffy and Diana Fong to the design of the Dogtail graphics and web site.)

What is Dogtail?

The Dogtail website says it all:

"...Dogtail is a GUI test tool and automation framework written in Python. It uses Accessibility (A11Y1)) technologies to communicate with desktop applications. Dogtail scripts are written in Python and executed like any other Python program..."

Let's stop for a minute and dissect that little paragraph, as it actually says quite a bit:

• Dogtail is a GUI test tool and automation framework written in Python. Dogtail itself doesn't test your application. It provides an easy-to-use framework in which you can build your tests.
• It uses Accessibility (A11Y) technologies to communicate with desktop applications. This is a key aspect of Dogtail's design. Unlike some other GUI test automation frameworks, Dogtail doesn't scrape information from the visual representation of the the application under the test's GUI into a proprietary data-store. Instead, it makes use of the accessibility-related metadata to create an in-memory model of the application's GUI elements. (We'll discuss this in more detail in a minute.)
• Dogtail scripts are written in Python and executed like any other Python program. One of my first questions when I was learning how to use Dogtail was, "After I install it, how do I use it? What do I run?" The answer was, "You write and run your test scripts." It's just that easy. (We'll walk through an example script later on in this article.)
• And although the paragraph doesn't mention it, it's important to note that Dogtail is an open source project. The code is there for you to use, learn from, and contribute to.

How does Dogtail work?

One of the main functions of any GUI testing framework is the identification of the elements in the GUI. It's possible to locate and identify GUI elements by their X and Y matrix coordinates, but this approach is limited by the requirement that the GUI layout not change. The better approach is to locate and identify the GUI elements as discrete objects that the test scripts can manipulate. Dogtail achieves this identification through the application GUI's accessibility information metadata.

Dogtail makes use of the Assistive Technology Service Provider Interface (AT-SPI1) accessibility framework. This is a part of the Gnome Accessibility Project (GAP2). AT-SPI provides a Service Provider Interface for the Assistive Technologies available on the GNOME platform, and a library against which applications can be linked.

Dogtail currently supports GNOME applications and many other GTK+ applications. Dogtail uses pyspi (a module written in Pyrex to give Dogtail access to AT-SPI's C API).

The Dogtail APIs--Accessibility API and procedural API

Dogtail currently supports two APIs for test script development: the procedural API and the object-oriented API. HappyDoc API documentation is available for both APIs at the Dogtail website.

The procedural API has been designed to be used for functional testing of desktop applications. The design goal of this API was to keep it simple enough for use by script authors with basic Python experience while still being powerful and flexible enough for general desktop GUI automated testing. If you are new to python or need to perform functional testing where you don't have to have fine-grain control of GUI objects, this is the API to use. The main module to study is dogtail.procedural.

The object-oriented API is designed to enable fine-grain control over interaction with GUI "accessibles" and easy customization through subclassing. Application developers may find it useful to use this API to drive their applications to specific states for testing and debugging. Application wrapper modules can be written to easily roll up functionality for debugging purposes. A good module to study for this approach is dogtail.tree.

Don't panic. The best way to learn the APIs is to use them. We'll walk through short example test scripts written in each API in a later section of this article.

Using Dogtail - Step by step

OK, enough talk. Let's put all this information to use and write and execute a test script. There are (5) steps to this process.

Step 1 - Install Dogtail

First, verify that you have the packages that Dogtail requires installed:

The following packages are required to use Dogtail:

• AT-SPI-enabled desktop (GNOME at this point in time)
• Python 2.3 or higher (available through your Red Hat Enterprise Linux distribution)
• ImageMagick 6.2 or higher (available through your Red Hat Enterprise Linux distribution)
• rpm-python or python-apt (available through your Red Hat Enterprise Linux distribution)
• ElementTree for Python (available through your Red Hat Enterprise Linux distribution)
• pyspi - Python AT-SPI bindings

Then, install the current Dogtail rpm.

Step 2 - Set up accessibility

In order for Dogtail to be able to make use of accessibility technology information, you have to enable accessibility in the GNOME desktop. To do this through the GNOME desktop GUI, access this menu: applications->preferences->accessibility->assistive technologies preferences and select the "enable assistive technologies" checkbox.

And, if your target application is written in Java, you also have to set this environment variable:

export GTK_MODULES="gail:atk-bridge"

If you start your target application from a shell prompt, you should also see this text displayed:

GTK Accessibility Module initialized

Step 3 - Identify the GUI elements in your application

OK, now it gets interesting. In order to write a test script to verify the operation of your application, you have to first identify the GUI element that you want to manipulate. For our sample scripts, we'll create and run a test for the gedit text editor.

Let's take a look at an example of how to view an application GUI's accessibility information metadata. There are a couple of approaches that you can follow.

Approach #1: The "sniff" utility - The Dogtail framework includes a standalone utility named sniff. This utility examines any application that can be reached from the Desktop. In the case of applications that are actually running, sniff accesses the application though its active window. Here's an example of sniff's output for the gedit text editor. Note how sniff displays the names of all applications' GUI elements. Sniff even recognizes that the file being edited by gedit is in the modified (but not yet saved) state.

Approach #2: Directly call the introspection methods - You can access the same information, in a structured (and very verbose) format, by invoking Dogtail's introspection methods directly. For example, to see a full set of the GUI elements that are included in gedit, just start up Python, and enter these statements:

>>>from dogtail.tree import root
>>>f = root.application('gedit')
>>>f.dump()

And then watch the output scroll by. Like I said, it's verbose. Here's a fragment:

Node roleName='menu item' name='New' description='' text='New'
 click
Node roleName='menu item' name='Open...' description='' text='Open...'
 click
Node roleName='menu item' name='Open Location...' description='' text='Open Location...'
 click
Node roleName='separator' name='' description='' text=''
 click
Node roleName='menu item' name='Save' description='' text='Save'
click
Node roleName='menu item' name='Save As...' description='' text='Save As...'
click
Node roleName='menu item' name='Revert' description='' text='Revert'
click
Node roleName='separator' name='' description='' text=''
click
Node roleName='menu item' name='Page Setup' description='' text='Page Setup'
 click
Node roleName='menu item' name='Print Preview...' description='' text='Print Preview...'
 click

What's with all the references to "click"? These are the actions that the GUI elements support. These are the actions that you will want your test scripts to execute. We'll talk about these in more detail later on in the article.

Step 4 - Write and run your test scripts

So far so good. Now, let's write some code. This section of the article walks through a simple test of the gedit GNOME text editor using each of the Dogtail APIs. The test simply reads in and saves a file and then compares the saved file against a known good ("golden") file. We'll examine the two gedit test scripts that are available for download from the Dogtail website.

Using the procedural API

Let's start with the procedural API sample script. Here's the code:

     1  #!/usr/bin/env python
     2  # Dogtail demo script using procedural API
     3  # FIXME: Use TC.
     4  __author__ = 'Zack Cerza <zcerza@redhat.com'
     5
     6  import dogtail.tc
     7  from dogtail.procedural import *
     8  from dogtail.utils import screenshot
     9  from os import environ, path, remove
    10
    11  # Load our persistent Dogtail objects
    12  TestString = dogtail.tc.TCString()
    13
    14  # Remove the output file, if it's still there from a previous run
    15  if path.isfile(path.join(path.expandvars("$HOME"), "Desktop", "UTF8demo.txt")):
    16          remove(path.join(path.expandvars("$HOME"), "Desktop", "UTF8demo.txt"))
    17
    18  # Start gedit.
    19  run('gedit')
    20 
    21  # Set focus on gedit
    22  focus.application('gedit')
    23
    24  # Focus gedit's text buffer.
    25  focus.text()
    26
    27  # Load the UTF-8 demo file. Use codecs.open() instead of open().
    28  from codecs import open
    29  from sys import path
    30  utfdemo = open(path[0] + '/data/UTF-8-demo.txt')
    31
    32  # Load the UTF-8 demo file into the text buffer.
    33  focus.widget.text = utfdemo.read()
    34
    35  # Click gedit's Save button.
    36  click('Save')
    37
    38  # Focus gedit's Save As... dialog
    39  focus.dialog('Save as...')
    40
    41  # click the Browse for other folders widget
    42  activate('Browse for other folders')
    43
    44  # Click the Desktop widget
    45  activate('Desktop', roleName = 'table cell')
    46
    47  # We want to save to the file name 'UTF8demo.txt'.
    48  focus.text()
    49  focus.widget.text = 'UTF8demo.txt'
    50
    51  # Click the Save button.
    52  click('Save')
    53
    54  # Let's quit now.
    55  click('Quit')
    56
    57  # We have driven gedit now lets check to see if the saved file is the same as 
    58  # the baseline file
    59
    60  # Read in the "gold" file
    61  import codecs
    62  try:
    63          # When reading the file, we have to make sure and tell codecs.open() which 
    64          # encoding we're using, otherwise python gets confused later.
    65          gold = open(path[0] + '/data/UTF-8-demo.txt', encoding='utf-8').readlines()
    66  except IOError:
    67          print "File open failed"
    68
    69  # Read the test file for comparison
    70  filepath = environ['HOME'] + '/Desktop/UTF8demo.txt'
    71  # When reading the file, we have to make sure and tell codecs.open() which 
    72  # encoding we're using, otherwise python gets confused later.
    73  testfile = open(filepath, encoding='utf-8').readlines()
    74
    75  # We now have the original and saved files as lists. Let's compare them line
    76  # by line to see if they are the same
    77  i = 0
    78  for baseline in gold:
    79          label = "line test " + str(i + 1)
    80          TestString.compare(label, baseline, testfile[i], encoding='utf-8')
    81          i = i + 1

Let's examine the script line by line:

Using the accessibility/object-oriented API

And now let's look at the accessibility, object-oriented version of the same test script.

     1  #!/usr/bin/env python
     2  # Dogtail demo script using tree.py
     3  # FIXME: Use TC.
     4  __author__ = 'Zack Cerza <zcerza@redhat.com'
     5
     6  from dogtail import tree
     7  from dogtail.utils import run
     8  from time import sleep
     9  from os import environ, path, remove
    10  environ['LANG']='en_US.UTF-8'
    11
    12  # Remove the output file, if it's still there from a previous run
    13  if path.isfile(path.join(path.expandvars("$HOME"), "Desktop", "UTF8demo.txt")):
    14          remove(path.join(path.expandvars("$HOME"), "Desktop", "UTF8demo.txt"))
    15
    16  # Start gedit.
    17  run("gedit")
    18
    19  # Get a handle to gedit's application object.
    20  gedit = tree.root.application('gedit')
    21
    22  # Get a handle to gedit's text object.
    23  textbuffer = gedit.child(roleName = 'text')
    24
    25  # Load the UTF-8 demo file.
    26  from sys import path
    27  utfdemo = file(path[0] + '/data/UTF-8-demo.txt')
    28
    29  # Load the UTF-8 demo file into gedit's text buffer.
    30  textbuffer.text = utfdemo.read()
    31
    32  # Get a handle to gedit's File menu.
    33  filemenu = gedit.menu('File')
    34
    35  # Get a handle to gedit's Save button.
    36  savebutton = gedit.button('Save')
    37
    38  # Click the button
    39  savebutton.click()
    40
    41  # Get a handle to gedit's Save As... dialog.
    42  saveas = gedit.dialog('Save as...')
    43
    44  # We want to save to the file name 'UTF8demo.txt'.
    45  saveas.child(roleName = 'text').text = 'UTF8demo.txt'
    46
    47  # Save the file on the Desktop
    48
    49  # Don't make the mistake of only searching by name, there are multiple
    50  # "Desktop" entires in the Save As dialog - you have to query for the
    51  # roleName too - see the online help for the Dogtail "tree" class for
    52  # details
    53  desktop = saveas.child('Desktop', roleName='table cell')
    54  desktop.actions['activate'].do()
    55
    56  #  Click the Save button.
    57  saveas.button('Save').click()
    58
    59  # Let's quit now.
    60  filemenu.menuItem('Quit').click() 

The differences in the code between this script and the procedural test script are:

The Python advantage (make that two...)

We discussed earlier in the article that building Dogtail on Python made writing test scripts easy and supported Dogtail's object-oriented design. Another advantage of having Dogtail built on Python is that you can debug your test scripts with the Python interpreter. Using the interpreter is a great way to try things out quickly as you're learning Dogtail, as any statements that you can run in a Dogtail scripts can also be run by typing them into the interpreter.

Here's a sample taken from the example test scripts we just discussed. After you define the savebutton variable, you can get a directory of all the methods that are supported for it by just typing in the dir statement.

>>> # Get a handle to gedit's Save button.
... savebutton = gedit.button('Save')
>>> dir (savebutton)
['_Node__accessible', '_Node__action', '_Node__component', '_Node__hideChildren', 
'_Node__nodeIsIdentifiable', '_Node__text', '__doc__', '__getattr__', '__init__', 
'__module__', '__setattr__', '__str__', 'addSelection', 'blink', 'button', 'child', 
'childLabelled', 'childNamed', 'click', 'contained', 'debugName', 'doAction', 'dump', 
'findAncestor', 'findChild', 'findChildren', 'getAbsoluteSearchPath', 'getLogString', 
'getNSelections', 'getRelativeSearch', 'getUserVisibleStrings', 'grabFocus', 'menu', 
'menuItem', 'rawClick', 'rawType', 'removeSelection', 'satisfies', 'setSelection', 'tab', 
'textentry', 'typeText']

And, you can also access the documentation included in the Python modules that support Dogtail. For example, here's a fragment of the help for the class:

>>> help (tree)

NAME
    dogtail.tree - Makes some sense of the AT-SPI API

FILE
    /usr/share/doc/dogtail-0.5.0/examples/dogtail/tree.py

DESCRIPTION
    The tree API handles various things for you:
    - fixes most timing issues
    - can automatically generate (hopefully) highly-readable logs of what the
    script is doing
    - traps various UI malfunctions, raising exceptions for them (again,
    hopefully improving the logs)
    
    The most important class is Node.  Each Node is an element of the desktop UI.
    There is a tree of nodes, starting at 'root', with applications as its
    children, with the top-level windows and dialogs as their children.  The various
    widgets that make up the UI appear as descendents in this tree.  All of these
    elements (root, the applications, the windows, and the widgets) are represented
    as instances of Node in a tree (provided that the program of interest is
    correctly exporting its user-interface to the accessibility system).  The Node
    class is a wrapper around Accessible and the various Accessible interfaces.

(For complete details, see the actual Dogtail help)

What's next? Dogtail's future

What's next for Dogtail?

As I'm writing this article, the Dogtail development team is hard at work implementing a recording and playback mechanism.

We'll explore that mechanism and other GUI automation topics such as logging, screen capture and analysis, and a deeper exploration of the Dogtail modules and classes next month.

What's next for you? Download Dogtail and take it for a walk. Arf. Arf.do.

More information

• Dogtail Project: http://people.redhat.com/zcerza/dogtail/index.html
• Assistive Technologies: http://developer.gnome.org/projects/gap/AT/at-types.html
• AT-SPI: http://directory.fsf.org/at-spi.html
• Linux Desktop Testing Project: http://ldtp.freedesktop.org

About the author

Len DiMaggio is not the creator of Dogtail--just one of its growing number of happy users. Len is a QE engineer at Red Hat in Westford, MA (USA) and has published articles on software testing in Dr. Dobbs Journal, Software Development Magazine, IBM Developerworks, STQE, and other journals. The credit for the creation and development of Dogtail goes to the team of Red Hat associates who first brought it to life and the community that is making contributions toward its future.