Knowledge Base Article

Home Page > Knowledge Base > Rapise

Home Page > Knowledge Base > Rapise > Rapise Desktop

Article Working with WPF and UIAutomation Structures

by Sarah T. on Friday, April 1, 2016

It is a common situation that desktop applications written using Microsoft Windows Presentation Framework (WPF) will have complex layouts including tabs, splitters and panels having custom grids and tables inside. This article provides help in dealing with such situations, including how to dynamically find items.

Handling such applications may require a deeper understanding of the application and its components to dynamically traverse the object tree and find a required element by its properties. This article provides and demonstrates a set of utility functions that can be used to automate these applications.

Consider a sample application with a ListBox control as shown below:

Suppose we have Learned just a single item (listBoxItem4). And now our goal is to click any other items in the same list box.

First, let’s look at the locator part of the learned item:

It looks as shown below:

tabControl1/tabItem1/listBox1/listBoxItem4/listBoxItem4

This path matches the tree structure that we may see from the UI Automation Spy:

So if we need to click a specific item while having object pointing to listBoxItem4 then we need to do the following:

  1. Climb 2 levels up to the whole [List].
  2. Search through all the items matching specific text (i.e. listBoxItem5 or so).

The attached Rapise sample contains a number of functions and a working sample demonstrating how it can be achieved. 

In addition, here is the list of utility functions (you may find them in the attached sample):

            //#region UIA utility functions
             
            /**
             * UIAInstance - used internally by other functions
             */
            function UIAInstance(obj)
            {
                   var instance = obj;
                   try
                   {
                          if(obj&&obj.instance)
                          {
                                 instance = obj.instance;
                          }
                   }catch(e){}
                   return instance;
            }
             
            /**
             * Find parent object (or n-th parent)
             * @param obj root object
             * @param lvl {=1} number of levels to climb
             * @return found object or null
             */
            function UIAParentObj(obj, lvl)
            {
                   if(typeof(obj)=="string")
                   {
                          obj = SeS(obj);
                   }
                      
                   lvl = lvl||1;
                   var instance = UIAInstance(obj);
                   var par = SeSGetUIAutomationParent(instance);
                   if(lvl&&lvl>1)
                   {
                          return UIAParentObj(par, lvl-1);
                   }
                   return SeSTryMatch(par, SeSUIAObjectRule);
            }
             
            /**
             * Find number of children inside this object
             * @param obj root object
             * @return number of children, 0 - no children
             */
            function UIAGetChildCount(obj)
            {
                   var instance = UIAInstance(obj);
                   return SeSGetUIAutomationChildrenCount(instance);
            }
             
            /**
             * Find number of children inside this object
             * @param obj root object
             * @param ind ind's child
             * @return found child object
             */
            function UIAGetChildAt(obj,ind)
            {
                   var instance = UIAInstance(obj);
                   var chld= SeSGetUIAutomationChildAt(instance,ind);
                   return SeSTryMatch(chld, SeSUIAObjectRule);
            }
             
            /**
             * Recursively search for a child having specified name or text
             * @param obj start object
             * @param findTxt string to match object text or name
             * @param [fndClsName] optional class name to match (sometimes we need Text object, not ListBoxItem, so className should help to filter)
             * @return found object
             */
            function UIAFindByText(obj, findTxt, findClsName)
            {
                   var instance = UIAInstance(obj);
                   var txt = SeSGetUIAutomationText(instance);
                   var cls = SeSGetUIAutomationClass(instance);
                   var name = SeSGetUIAutomationName(instance);
                   if(SeSCheckString(findTxt, name)||SeSCheckString(findTxt, txt))
                   {
                          if(findClsName)
                          {
                                 if(SeSCheckString(findClsName, cls))
                                 {
                                       return SeSTryMatch(instance, SeSUIAObjectRule);
                                 }
                          } else {
                                 return SeSTryMatch(instance, SeSUIAObjectRule);
                          }
                   }
                   // No match, check children
             
                   var cnt = UIAGetChildCount(instance);
                   for(var i=0;i<cnt;i++)
                   {
                          var chld= SeSGetUIAutomationChildAt(instance,i);
                          var res = UIAFindByText(chld, findTxt, findClsName);
                          if(res) return res;
                   }
                   return null;
            }
            //#endregion UIA utility functions
             
            The test working with these functions is following:
            function Test()
            {
                   Global.DoLaunch("AUTWPF\\AUTWPF.exe", null, true, "AUTWPF");
             
             
                   // We have object listBoxItem4, let's click it
                   SeS('listBoxItem4').DoClick();
                      
                   // Example 1: We learned item #4. But now we want to click item #2 by overriding locator
                   // Please, note the object 'listBoxItem4' has 'Ignore Object Name' property set to 'true'
                   SeS('listBoxItem4', {"location": "tabControl1/tabItem1/listBox1/listBoxItem2/listBoxItem2"}).DoClick();
                      
                   // Now we want to click items by text or name.    
                   // So we want to start from 'listBoxItem4'
                   // Then climb 2 levels up to whole listbox
                   // and then find for specific item by its name or text.
                   // So here we introduce several utility functions:    
                   // UIAParentObj, UIAGetChildCount, UIAGetChildAt, UIAFindByText
                   // You may find them in the WPFCalc.user.js file and use in your own projects.
                   // Here is how it works:
                   // Climb 2 levels (2 because we can see from the locator:    
                   // ....tabItem1/listBox1/listBoxItem4/listBoxItem4
                   // that 2 last chunks correspond to an item and we need to climb up to    
                   // the ...tabItem1/listBox1 ):
                   var par = UIAParentObj('listBoxItem4', 2);
                   // Now par is a listBox1. Search for item named listBoxItem5 recursively:
                   var i5 = UIAFindByText(par, "listBoxItem5");
                   // And click it:
                   i5.DoClick();             
            }
Attachments
Article Info
Last Updated: 4/1/2016
Article ID: KB204
# Views: 346
Powered by KronoDesk v1.1.0.15 | © Copyright Inflectra Corporation 2011-2016 | Licensed to Inflectra Corporation.