Object Hierarchy
• SilkTest supports a true object-oriented hierarchy of parent-child-grandchild-etc.relationships between windows and objects within windows. In this model an object such asa menu is the child of its enclosing window and a parent to its menu item objects.
• WinRunner, with some rare exceptions [often nested tables on web pages], has a flat objecthierarchy where child objects exist in parent windows. Note that web page frames are treatedas windows, and not child objects of the enclosing window on web pages that are constructedusing frames.
Object Recognition
Both of these tools use a lookup table mechanism to isolate the variable name used to referencean object in a test script from the description used by the operating system to access that object atruntime:
• SilkTest normally places an application’s GUI declarations in a test frame file. There isgenerally one GUI declaration for each window and each object in a window. A GUI declaration consists of an object identifier—the variable used in a test script—and its class and object tag definition used by the operating system to access that object at runtime.SilkTest provides the following capabilities to define an object tag:
(1) a string, which caninclude wildcards;
(2) an array reference which resolves to a string which can includewildcards;
(3) a function or method call that returns a string, which can include wildcards,
(4) an object class and class relative index number; and
(5) multiple tags [multi-tags] eachoptionally conditioned with
(6) an OS/GUI/browser specifier [a qualification label].
• WinRunner normally places an application’s logical name/physical descriptor definitions in a GUI Map file. There is generally one logical name/physical descriptor definition for eachwindow and each object in a window. The logical name is used to reference the object in atest script, while the physical descriptor is used by the operating system to access that objectat runtime.WinRunner provides the following capabilities to define a physical descriptor:
(1) a variablenumber of comma delimited strings which can include wildcards, where each stringidentifies one property of the object. [While there is only a single method of defining a physical descriptor, this definition can include a wide range and variable number ofobligatory, optional, and selector properties on an object by object basis].The notion behind this lookup table mechanism is to permit changes to an object tag [SilkTest]or a physical descriptor [WinRunner] definition without the need to change the associatedidentifier [SilkTest] or logical name [WinRunner] used in the testcase.In general the object tag [SilkTest] or physical descriptor [WinRunner] resolve to one or moreproperty definitions which uniquely identify the object in the context of its enclosing parentwindow at runtime.It is also possible with both tools to dynamically construct and use object tags [SilkTest] orphysical descriptors [WinRunner] at runtime to reference objects in test scripts.
Object Verification
Both tools provide a variety of built-in library functions permitting a user to hand code simple verification of a single object property [i.e. is/is not focused, is/is not enabled, has/does not have an expected text value, etc.]. Complex multiple properties in a single object and multiple object verifications are supported using visual recorders:
• SilkTest provides a Verify Window recorder which allows any combination of objects and object properties in the currently displayed window to be selected and captured. Using this tool results in the creation, within the testcase, of a VerifyProperties() method call against the captured window.
• WinRunner provides several GUI Checkpoint recorders to validate
(1) a single object property,
(2) multiple properties in a single object, and
(3) multiple properties of multiple objects in a window. The single property recorder places a verification statement directly inthe test code while the multiple property recorders create unique checklist [*.ckl] files inthe /chklists subdirectory [which describe the objects and properties to capture], as wellas an associated expected results [*.chk] file in the /exp subdirectory [which contains theexpected value of each object and property defined in the checklist file].Both tools offer advanced features to define new verification properties [using mapping techniques and/or built-in functions and/or external DLL functions] and/or to customize howexisting properties are captured for standard objects.
Custom Objects
Note: The description of this feature, more than any other in this report, is limited in its scope and coverage. An entire white paper could be dedicated to exploring and describing how each ofthese tools deal with custom objects. Therefore dedicate several days to evaluating how eachof these tools accommodate custom objects in your specific applications.To deal with a custom object [i.e. an object that does not map to standard class] both toolssupport the use of class mapping [i.e. mapping a custom class to a standard class with like functionality], along with a variety of X:Y pixel coordinate clicking techniques [some screen absolute, some object relative] to deal with bitmap objects, as well as the ability to use external DLL functions to assist in object identification and verification. Beyond these shared capabilitieseach tool has the following unique custom object capabilities:
• SilkTest has a feature to overlay a logical grid of X rows by Y columns on a graphic that has evenly spaced “hot spots”[this grid definition is then used to define logical GUI declarationsfor each hot spot]. These X:Y row/column coordinates are resolution independent [i.e. thelogical reference says “give me 4th column thing in the 2nd row”, where that grid expands orcontracts depending on screen resolution].
• WinRunner has a built-in text recognition engine which works with most standard fonts.This capability can often be used to extract visible text from custom objects, position thecursor over a custom object, etc. This engine can also be taught non-standard font typeswhich is does understand out of the box.Both tools offer support for testing non-graphical controls through the advanced use of customDLLs [developed by the user], or the Extension Kit [SilkTest, which may have to be purchased atan addition cost] and the Mercury API Function Library [WinRunner].
Internationalization (Language Localization)
• SilkTest supports the single-byte IBM extended ASCII character set, and its TechnicalSupport has also indicated “that Segue has no commitment for unicode”. The user guidechapter titled “Supporting Internationalized Applications” shows a straightforward techniquefor supporting X number of [single-byte IBM extended ASCII character set] languages in asingle test frame of GUI declarations.
• WinRunner provides no documentation on how to use the product to test language localizedapplications. Technical Support has indicated that
(1) “WinRunner supports multi-bytecharacter sets for language localized testing…”,
(2) “there is currently no commitment forthe unicode character set…”, and
(3) “it is possible to convert a US English GUI Map toanother language using a [user developed] phrase dictionary and various gui_* built-infunctions”.Check into the aspects of this feature very carefully if it is important to your testing effort.
Database Interfaces
Both tools provide a variety of built-in functions to perform Structure Query Language (SQL)queries to control, alter, and extract information from any database which supports the OpenDatabase Connectivity (ODBC) interface.
Database Verification
Both tools provide a variety of built-in functions to make SQL queries to extract informationfrom an ODBC compliant database and save it to a variable [or if you wish, an external file].Verification at this level is done with hand coding.WinRunner also provides a visual recorder to create a Database Checkpoint used to validate theselected contents of an ODBC compliant database within a testcase. This recorder creates sidefiles similar to GUI Checkpoints and has a built-in interface to
(1) the Microsoft Query application [which can be installed as part of Microsoft Office], and
(2) to the Data Junctionapplication [which may have to be purchased at an addition cost], to assist in constructing andsaving complex database queries.
Data Driven Testing
Both tools support the notion of data-driven tests, but implement this feature differently:
• SilkTest’s implementation is built around an array of user defined records. A record is adata structure defining an arbitrary number of data items which are populated with valueswhen the array is initialized [statically or at runtime]. Non-programmers can think of anarray of records as a memory resident spreadsheet of X number of rows which contain Ynumber columns where each row/column intersection contains a data item.The test code, as well as the array itself, must be hand coded. It is also possible to populatethe array each time the test is run by extracting the array’s data values from an ODBC compliant database, using a series of built-in SQL function calls. The test then iteratesthrough the array such that each iteration of the test uses the data items from the next recordin the array to drive the test or provide expected data values.
• WinRunner’s implementation is built around an Excel compatible spreadsheet file of Xnumber of rows which contain Y number of columns where each row/column intersectioncontains a data item. This spreadsheet is referred to as a Data Table.The test code, as well as the Data Table itself, can be created with hand coding or the use ofthe DataDriver visual recorder. It is also possible to populate a Data Table file each time thetest is run by extracting the table’s data values from an ODBC compliant database using aWinRunner wizard interfaced to the Microsoft Query application.. The test then iteratesthrough the Data Table such that each iteration of the test uses the data items from the nextrow in the table to drive the test or provide expected data values.Both tools also support the capability to pass data values into a testcase for a more modestapproach to data driving a test.
Restoring an Application’s Initial State
• SilkTest provides a built-in recovery system which restores the application to a stable state,referred to as the basestate, when the test or application under test fails ungracefully. Thedefault basestate is defined to be: (1) the application under test is running;
(2) the applicationis not minimized; and
(3) only the application’s main window is open and active. There aremany built-in functions and features which allow the test engineer to modify, extend, andcustomize the recovery system to meet the needs of each application under test.
• WinRunner does not provide a built-in recovery system. You need to code routines to returnthe application under test to its basestate—and dismiss all orphaned dialogs—when a testfails ungracefully.
