Handling such applications may require a deeper understanding of the application and its components to dynamically traverse the object tree and find a required element by its properties. This article provides and demonstrates a set of utility functions that can be used to automate these applications.
Consider a sample application with a ListBox control as shown below:
Suppose we have Learned just a single item (listBoxItem4). And now our goal is to click any other items in the same list box.
First, let’s look at the locator part of the learned item:
It looks as shown below:
tabControl1/tabItem1/listBox1/listBoxItem4/listBoxItem4
This path matches the tree structure that we may see from the UI Automation Spy:
So if we need to click a specific item while having object pointing to listBoxItem4 then we need to do the following:
- Climb 2 levels up to the whole [List].
- Search through all the items matching specific text (i.e. listBoxItem5 or so).
Modern Method
function Test()
{
Global.DoLaunch("AUTWPF\\AUTWPF.exe", null, true, "AUTWPF");
// We have object listBoxItem4, let's click it
SeS('listBoxItem4').DoClick();
// Example 1: We learned item #4. But now we want to click item #2 by overriding locator
// Please, note the object 'listBoxItem4' has 'Ignore Object Name' property set to 'true'
SeS('listBoxItem4', { "location": "tabControl1/tabItem1/listBox1/listBoxItem2/listBoxItem2" }).DoClick();
// Now we want to click items by text or name.
// So we want to start from 'listBoxItem4'
// Then climb 2 levels up to whole listbox
// and then find for specific item by its name or text.
// So here we introduce several utility functions:
// UIAParentObj, UIAGetChildCount, UIAGetChildAt, UIAFindByText
// You may find them in the WPFCalc.user.js file and use in your own projects.
// Here is how it works:
// Climb 2 levels (2 because we can see from the locator:
// ....tabItem1/listBox1/listBoxItem4/listBoxItem4
// that 2 last chunks correspond to an item and we need to climb up to
// the ...tabItem1/listBox1 ):
var par = SeS('listBoxItem4').GetParent().GetParent();
// Now par is a listBox1. Search for item named listBoxItem5 recursively:
var i5 = par.DoFindByText("listBoxItem5");
// And click it:
i5.DoClick();
}
See UIAObject.DoFindByText for more details.
Legacy Method
The attached Rapise sample contains a number of functions and a working sample demonstrating how it can be achieved.
In addition, here is the list of utility functions (you may find them in the attached sample):
//#region UIA utility functions
/**
* UIAInstance - used internally by other functions
*/
function UIAInstance(obj)
{
var instance = obj;
try
{
if (obj && obj.instance)
{
instance = obj.instance;
}
} catch (e) { }
return instance;
}
/**
* Find parent object (or n-th parent)
* @param obj root object
* @param lvl {=1} number of levels to climb
* @return found object or null
*/
function UIAParentObj(obj, lvl)
{
if (typeof (obj) == "string")
{
obj = SeS(obj);
}
lvl = lvl || 1;
var instance = UIAInstance(obj);
var par = SeSGetUIAutomationParent(instance);
if (lvl && lvl > 1)
{
return UIAParentObj(par, lvl - 1);
}
return SeSTryMatch(par, SeSUIAObjectRule);
}
/**
* Find number of children inside this object
* @param obj root object
* @return number of children, 0 - no children
*/
function UIAGetChildCount(obj)
{
var instance = UIAInstance(obj);
return SeSGetUIAutomationChildrenCount(instance);
}
/**
* Find number of children inside this object
* @param obj root object
* @param ind ind's child
* @return found child object
*/
function UIAGetChildAt(obj, ind)
{
var instance = UIAInstance(obj);
var chld = SeSGetUIAutomationChildAt(instance, ind);
return SeSTryMatch(chld, SeSUIAObjectRule);
}
/**
* Recursively search for a child having specified name or text
* @param obj start object
* @param findTxt string to match object text or name
* @param [fndClsName] optional class name to match (sometimes we need Text object, not ListBoxItem, so className should help to filter)
* @return found object
*/
function UIAFindByText(obj, findTxt, findClsName)
{
var instance = UIAInstance(obj);
var txt = SeSGetUIAutomationText(instance);
var cls = SeSGetUIAutomationClass(instance);
var name = SeSGetUIAutomationName(instance);
if (SeSCheckString(findTxt, name) || SeSCheckString(findTxt, txt))
{
if (findClsName)
{
if (SeSCheckString(findClsName, cls))
{
return SeSTryMatch(instance, SeSUIAObjectRule);
}
} else
{
return SeSTryMatch(instance, SeSUIAObjectRule);
}
}
// No match, check children
var cnt = UIAGetChildCount(instance);
for (var i = 0; i < cnt; i++)
{
var chld = SeSGetUIAutomationChildAt(instance, i);
var res = UIAFindByText(chld, findTxt, findClsName);
if (res) return res;
}
return null;
}
//#endregion UIA utility functions
The test working with these functions is following:
function Test()
{
Global.DoLaunch("AUTWPF\\AUTWPF.exe", null, true, "AUTWPF");
// We have object listBoxItem4, let's click it
SeS('listBoxItem4').DoClick();
// Example 1: We learned item #4. But now we want to click item #2 by overriding locator
// Please, note the object 'listBoxItem4' has 'Ignore Object Name' property set to 'true'
SeS('listBoxItem4', { "location": "tabControl1/tabItem1/listBox1/listBoxItem2/listBoxItem2" }).DoClick();
// Now we want to click items by text or name.
// So we want to start from 'listBoxItem4'
// Then climb 2 levels up to whole listbox
// and then find for specific item by its name or text.
// So here we introduce several utility functions:
// UIAParentObj, UIAGetChildCount, UIAGetChildAt, UIAFindByText
// You may find them in the WPFCalc.user.js file and use in your own projects.
// Here is how it works:
// Climb 2 levels (2 because we can see from the locator:
// ....tabItem1/listBox1/listBoxItem4/listBoxItem4
// that 2 last chunks correspond to an item and we need to climb up to
// the ...tabItem1/listBox1 ):
var par = UIAParentObj('listBoxItem4', 2);
// Now par is a listBox1. Search for item named listBoxItem5 recursively:
var i5 = UIAFindByText(par, "listBoxItem5");
// And click it:
i5.DoClick();
}