October 12, 2006

Rate this page del.icio.us  Digg slashdot StumbleUpon

Dogtail's object oriented tree API (and how to use it)

by Len DiMaggio


This is the fourth in a series of articles on the Dogtail automated GUI test framework. Last time , we showed how Dogtail is being used by the Frysk project team to create automated GUI tests. In this article, we'll take a closer look at Dogtail's objected oriented tree API and walk through examples of how you can use this API to develop automated GUI tests.

In the second installment in this series, we took a look at all of the Python modules that comprise the Dogtail framework. This month, we'll dive deeper into the object-oriented or tree API.

Let's start by reviewing why Dogtail provides both a procedural and an object-oriented tree API for writing automated test scrips. The answers to the question, "why two APIs?," and the more important question, "which API should I use?" really depends on where you're coming from, and where you're trying to go.

If you have experience in Python programming and with test automation, and you want to maintain fine grained control over the test, then what makes the most sense is for you to use Pyunit as a test framework for your Dogtail tests, use the Pyunit assert functions wherever possible, and use the Object Oriented API to write the tests.

If, however, you're just starting out with Python and/or test automation, and you want to put together a basic set of tests, then what makes the most sense is for you to start by using the tc module's compare functions, and the Procedural API. Using these tools will get you up and running quicker, and you can always convert your tests to use Pyunit and the Object Oriented API when you acquire more experience.

In short, Dogtail was intentionally designed to support use by both people with varying levels of experience with the Python language and with test automation frameworks and test automation in general.

No login required. Want to see your comments in print? Send a letter to the editor.

The tree API actions and nodes

The Action class represents an action that the accessibility layer exports as performable on a specific node, such as clicking on it. This class is a wrapper around the AT-SPI AccessibleAction object. As of this writing, Dogtail supports performing these actions on GUI nodes through the Action class:

  • click - This action signifies that a GUI node has been pressed and released.
  • press - To press (i.e., press a mouse button) on a GUI node.
  • release - And release a node that is pressed.
  • activate - This usually indicates that a GUI node has been clicked.
  • jump - This is used to follow a hyperlink in a web browser.
  • check - To select a check box.
  • dock - To move an application into the GNOME panel.
  • undock - And to remove it.
  • open - This generally happens by default when you access a GUI node.
  • menu - To open a menu (e.g., in a GNOME application in the panel)

The Node class represents the elements in the GNOME GUI desktop. This is the primary class in the tree module.

Each element in the GNOME GUI Desktop, including your application under test, is a node. The nodes are organized into a tree, starting with a root at the top. Every GUI element for which an application correctly exports accessibility information is represented as a node in this tree. The applications are the children of this top-level root node. The windows and dialogs that comprise each application are the children of the application nodes.

You can see an illustration of this tree of GNOME GUI Desktop elements in the "sniff" AT-SPI Browser utility that is part of the Dogtail distribution. For example, the following sniff display shows the applications, and windows and dialogs within each application on a typical Fedora5 system:

Fig 1. Sniff display.

The Node class implements functions to enable Dogtail scripts to traverse and interact with the GUI nodes in an application by functioning as a wrapper around AT-SPI layer objects (i.e., Accessibles) to make the accessibility information available. Your Dogtail scripts invoke these functions indirectly by reading and writing the following attributes:

Attribute Name Attribute Type Attribute Description
name read-only string Wraps Accessible_getName on the Node's underlying Accessible data
roleName read-only string Wraps Accessible_getRoleName on the Node's underlying Accessible data
description read-only string Wraps Accessible_getDescription on the Node's underlying Accessible data
child read-only Node instance A child of this node, wrapping getChild
children read-only list of Node instances The children of this node, wrapping getChildCount and getChildAtIndex
parent read-only Node instance A Node instance wrapping the parent, or None. Wraps Accessible_getParent
text
passwordText
string
write-only string
For instances wrapping AccessibleText, this is the text and is read-only, unless the instance wraps an AccessibleEditableText. In this case, you can write values to the attribute. This will get logged in the debug log, and a delay will be added. After the delay, the content of the node will be checked to ensure that it has the expected value. If it does not, an exception will be raised.

This does not work for password dialogs (since all we get back are * characters). In this case, set the passwordText attribute instead.

combovalue write-only string For combo-boxes. You write to this attribute to set the combo-box to the given value, with appropriate delays and logging.
relations read-only list of AT-SPI Relation instances Wraps Accessible_getRelationSet
labellee read-only list of Node instances The node(s) for which this node is a label. This list is generated from "relations."
labeller read-only list of Node instances The node(s) that is/are a label for this node. This list is also generated from "relations." The most common example of a labellee and its labeller counterpart are the labels that are associated with fields in a form. For example, a "First Name" labeller would correspond to the field that contains the firstName variable value.
sensitive read-only boolean Indicated whether the node is sensitive (i.e., not greyed-out in the GUI display). This is generated from stateSet and is based on presence of the AT-SPI SPI_STATE_SENSITIVE attribute. Note that not all applications set this up correctly.
showing read-only boolean This is generated from stateSet based on presence of the AT-SPI SPI_STATE_SHOWING attribute and reflects whether a GUI node is visible.
actions read-only list of Action instances This list is generated from the AT-SPI Accessible_getAction and AccessibleAction_getNActions functions. For each action that is supported by a specific node, an action function is enabled. The actions defined for a node will be specific to that node.
extents read-only Python tuple For instances wrapping a Component, the (x,y,w,h) screen extents of the component.
position read-only Python tuple For instances wrapping a Component, the (x,y) screen position of the component.
size read-only Python tuple For instances wrapping a Component, the (w,h) screen size of the component.
grabFocus For instances wrapping a Component, attempt to set the keyboard input focus to that Node.
toolkit read-only string For instances wrapping an application, the name of the toolkit/application.
version read-only string For instances wrapping an application, the application version.
ID read-only int For instances wrapping an application, the application ID #, as defined by the AT-SPI layer.

In other words, to make use of the tree API, your Dogtail test scripts "get" and "set" these attributes for GUI nodes.

For the rest of this article, we'll take a close look at the tree API through examples of how Dogtail scripts manipulate nodes in a GUI by reading and writing these attributes and the functions that manipulate them.

Let's start with the "name" attribute.

The "name" attribute

The name attribute is a read-only string. This attribute wraps the Accessible object _getName on the GUI node's underlying accessible data. You'll make extensive use of the name attribute in your Dogtail test scripts as part of the challenge of writing automated GUI tests is to locate the correct GUI nodes. Here's an example code fragment of a script that uses the name attribute to locate a specific GUI node:

theObserverActionsTable = observerPanel.child (name = 'observerActionsTable')

And, how can you view the value of the name attribute? Here's how:

print 'the name attribute = ' + theObserverActionsTable.name
>>>the name attribute = observerActionsTable

Note the lack of parenthesis on the name attribute. This is because it's an attribute named "name" not a function invoked with "name()"

Also note that you can even skip specifying the name attribute in child searches such as these. The following line of code is functionally equivalent to the above line:

theObserverActionsTable = observerPanel.child ('observerActionsTable')

Now, the whole reason that GUI node names are important at all to writing Dogtail test scripts is that we have to have some way of uniquely identifying each GUI node. But, what do we do if multiple GUI nodes have the same name? How could this ever happen? Shouldn't the application developers always ensure that each GUI node has a uniquely identifiable name that can be referenced from a test script? Ideally, yes, but sometimes, this is overlooked when an application is developed.

Also, sometimes, this is not possible. For example, if part of a GUI is generated dynamically at run-time, the names of the GUI nodes (and the number of GUI nodes) won't be known before the application is run. What's needed to resolve these situations is a mechanism to provide out test scripts with more fine-grained control that the name attribute alone. The mechanism is to combine name, roleName, and description attribute values to create a unique combination which in turn can identify a unique GUI Node.

The "roleName" attribute

The roleName attribute is another read-only string. Unlike the name attribute, roleName indicates the type of the GUI node in question. The supported types are:

  • application
  • check menu item
  • dialog
  • frame
  • icon
  • menu
  • menu item
  • page tab
  • panel
  • push button
  • radio menu item
  • scroll pane
  • table
  • table cell
  • text

Here's an example code fragment of a script that uses the roleName attribute to locate a specific GUI node:

customScrollPane = customObservers.child( roleName='scroll pane' )

And, here's another example code fragment that uses the name and roleName attributes together:

observerPanel = observerDetails.child( name='table1', roleName='panel' )

Again, how can you view the value of the roleName attribute? Here's how:

print 'the roleName attribute = ' + observerPanel.roleName
>>>the roleName attribute = panel

The "description" attribute

The description attribute is another readonly string. The value of the string is the corresponding description field in the Glade file. (See illustration below.)

Fig 2. Description field in the Glade file.

Here's an example code fragment of a script that uses the roleName and description attributes to locate a specific GUI node:

namedObserver = observerPanel.child( roleName='text', description='Enter a name for the observer' )

And, one more time, how can you view the value of the description attribute? Here's how:

print 'the description attribute = ' + namedObserver.description
>>>the description attribute = Enter a name for the observer

This example highlights what can be a common problem area in GUI test development; situations where the AT information as defined in the Glade framework is incomplete. In this specific case, the name attribute was not set. (It is now!)

Before we move on to discussing other attributes, it's worth reviewing that the name and description attributes get their values from the AT information Name and Description fields. The roleName attribute is also based on the application design defined in Glade, but the value of the attribute is the GUI node type as defined in Glade.

The "child" function and predicates too

In order to traverse the tree of applications, GUIs, and GUI nodes, we need to have a way to move down the tree starting from the root at its top. The mechanism by which you can do this is the child attribute.

Unlike the name, roleName, and description attributes, using the child attribute does not return a readonly string, it returns a GUI node, that being the child node of the node on which access the child attribute. So, by accessing the child node of other GUI nodes, you can "walk down" the application tree.

We've actually used the child attribute in the example code we used to illustrate the name, roleName, and description attributes.

But wait, there's more. To access a button in a dialog box, you might use a statement such as this:

theButton = parentNode.child(name='Quit', roleName='button', description='press to quit')

Now, this is functional, but it's also wordy and a pain in the neck to type. But, don't panic. The predicate module makes it possible for you to achieve the same results with this much simpler, and more readable, statement:

theButton = parentNode.button('Quit')

The predicate module supports searches for these types of GUI nodes:

  • applications
  • windows
  • dialogs
  • menus
  • menu items
  • text entries
  • buttons
  • tabbed panels

So far so good. But, what if you have to access a group of child GUI nodes all at once? You use the children attribute.

The "children" Attribute

After our discussion of the child attribute, the basic use of the children attribute is simply a logical extension. Where the child attribute returns a single child GUI node of he target node, the children attribute returns a Python List of all the child attributes of the target node. Easy, right?

Well, you might end up getting more than you had planned. Suppose your target GUI node is the parent of multiple different types of GUI nodes, and you're only interested in accessing one type of node. With the children attribute, your Dogtail script will have to include logic to examine each child GUI node to determine if the node is of the type you want to access.

Fortunately, the predicate module provides an easy way to filter the type of child node that you retrieve from a target node. You do this my means of the findChildren() function. You control the type of child node in the Python List returned by this function by passing the function a parameter that specifies a"generic predicate." Here's an example that returns a List of all the GUI nodes of type combo box (the second parameter is a boolean that controls whether the search is recursive):

theComboBoxes = dataEntryDialog.findChildren(predicate.GenericPredicate(roleName='combo box'), False)

The "parent" Attribute

The child and children attributes enable our Dogtail scripts to traverse the tree of applications, dialogs, GUI nodes, etc. from the root down through its branches. But, what if we need to move back up the tree? This is where we can use the parent attribute. The use of the parent attribute is the mirror image of the child attribute in that it returns a single GUI Node. For example:

aboutFrame = myApp.dialog('About')
print 'the dialog=' + str(aboutFrame)
print "the parent=' + str(aboutFrame.parent)
>>>the dialog={"About myApp" dialog}
>>>the parent=Node roleName='application' name='myApp'

The "text" and "passwordText" Attributes

Things start to get more interesting with the text and passwordText attributes. So far, all the attributes we've discussed in the article have been read-only. The text attributes, however, enable our Dogtail scripts to not only retrieve data, but also to manipulate it. The data in question takes the form of the text displayed by and entered into GUI nodes that take the form of text fields.

Let's look at the text attribute first. For examining the current content of a GUI Node's text attribute, the usage is very simple:

lastNameValue = lastNameGUINode.text

The usage for setting the attribute value is also simple:

lastNameGUINode.text = 'Flintstone'

But, Dogtail is actually doing some work behind the scenes here. The act of writing a value to the attribute is written into the debug log, and a delay will be added. After the delay, the content of the node will be checked to ensure that it has the expected value (the value that you specified). If it does not, an exception will be raised.

But, this value checking will always fail for password dialogs since all we get back are * characters. For password GUI Nodes, our Dogtail scripts have to set the passwordText attribute instead:

passwordGUINode.passwordText = '!@d234$_w'

The "combovalue" Attribute

In contrast to most of the other attributes we've discussed so far in this article, the combovalue attribute is a write-only attribute. Your Dogtail scripts write to this attribute to set the combo-box to the given value, with appropriate delays and logging.

Here's an example of how this attribute can make your life easier, and your Dogtail scripts' code easier to read and maintain. The following code is valid and will enable a Dogtail script to select an item from a combobox:

theItem = theComboBox.menuItem (theItemName)
theItem.click()

But, the following code accomplishes the same result by making use of the combovalue attribute, and reduces two lines of code into one:

theComboBox.combovalue = theItemName

The "relations", "labellee" and "labeller" Attributes

One way to improve the testability of a GUI application is to build in relationships between related fields in the same screen display. Dogtail can make writing automated test scripts that access these fields easier with the readonly attributes relations, labellee, and labeller. The most common example of a labellee and its labeller counterpart are the labels that are associated with fields in a form. For example, a"First Name" labellee would correspond to the field that contains the First Name variable value.

The relations attribute returns a Python List of relation instances. Each relation instance includes a labellee (the GUI node that represents the label field) and a labeller (the GUI node that represents the field being labeled). Let's look at an example:

theRelationsList = theDialog.relations
for x in theRelationsList:
    thelabellee = x.labellee
    thelabeller = x.labeller
    print 'labelee=' + thelabellee.name
    print 'labeller=' + thelabeller.name

The "sensitive" and"showing" Attributes

As I'm writing this article, I'm dealing with three separate projects that are all in"crunch mode." Believe me, at this point in these projects, everyone feels"sensitive."

In GUI testing, the term sensitive refers to whether a GUI node is active, and is not displayed in a greyed-out mode. For example, in a multi-part form, a button to advance from one part #1 to part #2 may not be sensitive until the user fills in certain fields such as name, address, etc.

Your Dogtail test scripts make use of the read-only boolean sensitive attribute to determine if a GUI node is sensitive when it should be. For example:

lastNameField.text = theTextValue
if streetAddressField.sensitive:
    streetAddressField.text = theOtherTextValue
elif:
    print 'error -- streetAddressField should be sensitive'

The showing attribute is similar in that it also returns a boolean value. This value is based on whether the GUI node in question is visible at all.

(Come to think of it, the way people working on my projects are trying hard to not been seen or assigned more work, maybe the GUI automation meaning of"sensitive" is accurate.)

The "actions" Attribute

The actions attribute returns a Python List of the actions supported by a GUI node. It's through these actions that your Dogtail scripts exercise much of the application under test. The specific actions supported by a node are unique to that node. How can you determine the actions supported by a node? Here's how to view the actions supported by a GUI node, in this case, a button:

editButton = customObservers.button( 'Edit' )
theActions = editButton.actions
for x in theActions:
    print 'action = ' + x
>>>action = press
>>>action = release
>>>action = click

You can also see the same listing of actions in the sniff utility. in this case for an"OK" button:

Fig 3. List of actions for an"OK" button in the sniff utility

There's one more point that I want to cover before we move on. There's an action that you'll need to use for text fields. The action is"activate."

Here's that action highlighted by sniff:

Fig 4. The"activate" action, highlighted by sniff.

The "extents", "position," and "size" Attributes

Some of the basic goals of any GUI automation effort are to make the tests portable and maintainable. This is why we write tests that access GUI nodes based on their name, roleName, etc. and not on their location on the X and Y axises of any specific screen display. There are times, however, when you do need to keep track of the size and or location of a GUI node, for example, if you're testing an application's ability to correctly resize a GUI node. For these tests, you can use the readonly extents, position and size attributes.

Using these attributes is straightforward as each returns a readonly Python tuple:

  • The extents attribute returns the X, Y coordinates and the width and height extents of the GUI node, measured in pixels.
  • The position attribute returns the X, Y coordinates of the GUI node, measured in pixels.
  • The size attribute returns the width and height extents of the GUI node, measured in pixels.

Here's an example of access the extents attribute:

okButton = observerDetails.button( 'OK' )
theExtents = okButton.extents
print 'X, Y axis coordinates = ' + str(theExtents[0]) + ',' + str(theExtents[1])
print 'width = ' + str(theExtents[2])
print 'height = ' + str(theExtents[3])
>>>X, Y axis coordinates = 905,644
>>>width = 85
>>>height = 32

The "grabFocus" Function

Before you can use the typeText() and keyCombo() functions access some GUI nodes, you have to set the keyboard input focus to that node. For example, a dialog that lists the name of processes to monitor. The use of this attribute is simple:

targetProcessName = theProcesslist.child ( name = 'tcsh')
targetProcessName.grabFocus

The "toolkit", "version", and "ID" Attributes

These three readonly attributes return information related to the libraries used to support the GUI and implementation of accessibility information built into the application that you're testing. You probably won't need to use these attributes in your Dogtail scripts, but they may be useful in debugging problems in that they can tell you the exact version of the toolkit with which the application under test is running.

The toolkit and version attributes return information on the AT toolkit used to develop the application. The ID attribute returns the integer value assigned to the application under test by the AT-SPI layer. Here's an example:

run ( 'myApplication', appName='testApplication;')
theApplicaitonObject = tree.root.application ( 'testApplication' )
print 'toolkit = ' + theApplicaitonObject.toolkit
print 'version = ' + theApplicaitonObject.version
print 'ID = ' + str(theApplicaitonObject.ID)
>>>toolkit = GAIL
>>>version = 1.8.11
>>>ID = 17

What's Next?

This next article in the series will be our last one (at least for a while -- remember that Dogtail is very much a work in progress. In the next article we'll walk through using the Dogtail script recorder and how to run Dogtail scripts in a"headless" (i.e., no display needed) mode.

References

Acknowledgments

The credit for the creation and development of Dogtail goes to the team of Red Hat associates (Zack Cerza, Ed Rousseau, and David Malcolm) who first brought it to life and the community that is making contributions toward its future.

Rate this page del.icio.us  Digg slashdot StumbleUpon

About the author

Len DiMaggio is a QE Engineer at Red Hat in Westford, MA (USA) and has published articles on software testing in Dr. Dobbs Journal, Software Development Magazine, IBM Developerworks, STQE, and other journals.