Scripting Language
Both tools provide proprietary, interpreted, scripting languages. Each language provide the usual flow control constructs, arithmetic and logical operators, and a variety of built-in libraryfunctions to perform such activities as string manipulation, [limited] regular expression support,standard input and output, etc. But that is where the similarity ends:
• SilkTest provides a strongly typed, object-oriented programming language called 4Test.Variables and constants may be one of 19 built-in data types, along with a user definedrecord data type. 4Test supports single- and multi-dimensional dynamic arrays and lists,which can be initialized statically or dynamically. Exception handling is built into thelanguage [via the do… except statement].
• WinRunner provides a non-typed, C-like procedural programming language called TSL.Variables and constants are either numbers or strings [conversion between these two typesoccurs dynamically, depending on usage]. There is no user defined data type such as a recordor a data structure. TSL supports sparsely populated associative single- and [pseudo] multidimensionarrays, which can be initialized statically or dynamically—element access isalways done using string references—foobar[“1”] and foobar[1] access the sameelement [as the second access is implicitly converted to an associative string indexreference]. Exception handling is not built into the language.The only way to truly understand and appreciate the differences between these two programmingenvironments is to use and experiment with both of them.
Exception Handling
Both tools provide the ability to handle unexpected events and errors, referred to as exceptions,but implement this feature differently:
• SilkTest’s exception handling is built into the 4Test language—using the do… exceptconstruct you can handle the exception locally, instead of [implicitly] using SilkTest’sdefault built-in exception handler [which terminates the currently running test and logs anerror]. If an exception is raised within the do block of the statement control is thenimmediately passed to the except block of code. A variety of built-in functions[LogError(), LogWarning, ExceptNum(), ExceptLog(), etc.] and 4Test statements[raise, reraise, etc.] aid in the processing of trapped exceptions within the except blockof code.
• WinRunner’s exception handling is built around
(1) defining an exception based on the typeof exception (Popup, TSL, or object) and relevant characteristics about the exeception (mostoften its error number);
(2) writing an exception hander, and
(3) enabling and disabling thatexception handler at appropriate point(s) in the code. These tasks can be achieved by handcoding or through the use of the Exception Handling visual recorder.
Test Results Analysis
• SilkTest’s test results file resolves around the test run. For example if you run 3 testcases[via a test suite or SilkOrganizer] all of the information for that test run will be stored in asingle results file. There is a viewer to analyze the results of the last test run or X previous tothe last run. Errors captured in the results file contain a full stack trace to the failing line ofcode, which can be brought up in the editor by double-clicking on any line in that stack trace.
• WinRunner’s test results file revolves around each testcase. For example if you run 3testcases [by hand or via a batch test or TestDirector] 3 test results files are created, each in asubdirectory under its associated testcase. There is a viewer to analyze the results of a test’slast run or if results have not been deleted, a previous run. Double clicking on events in thelog often expands that entry’s information, sometimes bringing up specialized viewers [forexample when that event is some type of checkpoint or some type of failure].
Managing the Testing Process
• SilkTest has a built-in facility, SilkOrganizer, for creating a testplan and then linking thetestplan to testcases. SilkOrganizer can also be used to track the automation process andcontrol the execution of selected groups of testcases. One or more user defined attributes[such as “Test Category”, “Author”, “Module”, etc.] are assigned to each testcase and thenlater used in testplan queries to select a group of tests for execution. There is also a modestcapability to add manual test placeholders in the testplan, and then manually add pass/failstatus to the results of a full test run. SilkTest also supports a test suite, which is a filecontaining calls to one or more test scripts or other test suites.
• WinRunner integrates with a separate program called TestDirector [at a substantialadditional cost], for visually creating a test project and then linking WinRunner testcases intothat project. TestDirector is a database repository based application that provides a variety oftools to analyze and manipulate the various database tables and test results stored in therepository for the project. A bug reporting and tracking tool is included with TestDirector aswell [and this bug tracking tool supports a browser based client].Using a visual recorder, testcases are added to one or more test sets [such as “TestCategory”, “Author”, “Module”, etc.] for execution as a group. There is a robust capabilityfor authoring manual test cases [i.e. describing of each test step and its expected results],interactively executing each manual test, and then saving the pass/fail status for each teststep in the repository. TestDirector also allows test sets to be scheduled for execution at atime and date in the future, as well as executing tests remotely on other machines [this lastcapability requires the Enterprise version of TestDirector].TestDirector is also capable of interfacing with and executing LoadRunner test scripts aswell as other 3rd party test scripts [but this later capability requires custom programming viaTestDirector APIs]. Additionally TestDirector provides API’s to allow WinRunner as well asother 3rd party test tools [and programming environments] to interface with a TestDirectordatabase.
External Files
When the tool’s source code files are managed with a code control tool such as PVCS or VisualSourceSafe, it is useful to understand what external side files are created:
• SilkTest implicitly creates *.*o bytecode-like executables after interpreting the source codecontained in testcases and include files [but it unlikely that most people will want to sourcecode control these files]. No side files are created in the course of using its recorders.SilkTest does though create an explicit *.bmp files for storing the expected and actualcaptured bitmap images when performing a bitmap verification.• WinRunner implicitly creates many side files using a variety of extensions [*.eve, *.hdr,*.asc, *.ckl, *.chk, and a few others] in a variety of implicitly created subdirectories[/db, /exp, /chklist, /resX] under the testcase in the course of using its visual recordersas well as storing pass/fail results at runtime.
Debugging
Both tools support a visual debugger with the typical capabilities of breakpoints, single step, runto, step into, set out of, etc.
