WATSUP Manual

WATSUP - Windows Application Test System Using PythonThe WATSUP toolkit is designed to allow the automated test of Windows applications. The system uses the "object-based" mechanism for identifying and invoking actions on controls and menu items.So much has been written about the scope, robustness, scalability and outstanding usability of Python that I'll go no further with it here, only to say that if you haven't yet had a chance to use this comprehensive, open source language, don't miss out on the opportunity to take a serious look at it!The examples in this document assume a basic familiarity with Python. If this isn't the case, have a look at a python tutorial or Mark Pilgrim's excellent Dive into Python. Functional TestsTesters/developers write automated functional tests which follow a prescriptive, possibly branching, possibly dynamic "user workflow". The script can check for changes in thegui itself, operating system environment, file system, database table and records, network, internet or extranet urls/pages/web services ... - in fact anywhere that there could be changes.Examination of the functions in module autoWinGui.py within the watsup package shows the variety of windows control items that can be checked/modified. These include: ∙Get and set text in editable controls∙Edit and select items from controls supporting lists∙Click and double-click controls to invoke their actions∙Determine the state of menu items and invoke themThe system also provides tools for finding windows by caption and/or class, controls by text/caption and/or class, and menu items by text or position. (One of the aspirations of this project is to continue to extend the list to include as many controls as possible) .Example 1 - automated writing on NotepadHere's a simple example of the control over applications that you can have with watsup. First, launch notepad from:Windows Start Menu - All Programs - Accessories - NotepadThen run the following script (Example 1)from watsup.winGuiAuto import findTopWindow,findControl,setEditTextfrom time import sleep# Locate notepad's edit area, and enter various bits of text. notepadWindow = findTopWindow(wantedClass='Notepad')editArea = findControl(notepadWindow,wantedClass="Edit")setEditText(editArea, "Hello, again!")sleep(0.8)setEditText(editArea, " You still there?",True)Finally, close notepad.Example 2 - testing a simple exampleIn functional tests, the tester wants to ensure that the cause - invoking a sequence of windows events (button clicks, menu item activation) has the predicted effect of, for example, a change in a value of a sindows control, the creation of a file, or the entry of a new database record. See the directory watsup/examples/simple directory for the executable simple.exe. If you run the application, you see a button and a text box. Enter a valid filename into the box, say xyz, and click the button; after the file is created, a message box appears containing a success message, and investigation of the directory watsup/examples/simple will show a file called 'xyz.txt' has been created (or overwritten). Now let's script a test to automate this functionality.First find and launch the application. Then run the following script (Example 2)from watsup.winGuiAuto import findControl,setEditText,findTopWindow,clickButtonimport osimport os.pathFILENAME='atestfile.txt'def main():# delete any occurrence of this file from the diskif os.path.exists(FILENAME):os.remove(FILENAME)form=findTopWindow(wantedText='Simple Form')button=findControl(form,wantedText='Create file')editbox=findControl(form,wantedClass='TEdit')# enter a filename:setEditText(editbox,[FILENAME])print 'clicking button to create file'clickButton(button)# now check that the file is thereif os.path.exists(FILENAME):print 'file %s is present' %FILENAMEelse:print "file %s isn't there" % FILENAMEif __name__=='__main__':main()Example 3 - automating program launch and terminationIt's a bit tedious having to start and close the application each time. Example 3 launches the application, if it isn't already running, and terminates it on completion of the test from uncher import launchApp,terminateAppfrom watsup.winGuiAuto import findTopWindowsimport example2# find an instance of SimpleForm. If one isn't there, launch itforms=findTopWindows(wantedText='Simple Form')if forms:form=forms[0]else:form=launchApp('simple.exe',wantedText='Simple Form')example2.main()# and terminate the formterminateApp(form)launchApp starts the application in a separate thread, and looks for a window with caption containing "Simple Form", returning the window handle of the recovered form. terminateApp attempts to close the form, by trying to activate menu item File-Exit, or, failing that, sending Alt + F4.Example 4 - finding windows and controlsIn building scripts, we need to be able to find the class and/or text of the many windows and controls to be investigated or invoked.In the tools directory, there's a tool - ShowWindows.bat - to assist us with this. Clicking the "Register" button persists information about the existing windows running on the system (and it tells you how many, FYI). Clicking the "Find new" button will report all non-trivial windows and all their constituent controls which have appeared in the windows environment since the last "Register" click. So to test our program simple.exe, launch ShowWindows, click Register. Then launch simple.exe and and click the Find New button. The associated text box shows n/m, where m is the total number ofnew windows found, and n is the number of those which are significant (ie have any controls) and are reported.Performance TestsPerformance tests, in this definition, are single-client scripts, which are similar in operation to the functional tests above, but for which certain steps of the tests must either be done within an acceptable timeframe ("CHECKING") and/or the time taken for those steps should be recorded for subsequent analysis ("RECORDING").WATSUP provides a simple mechanism to add such tests to existing functional tests. In examples 4a & 4b, we launch almost identical applications, perform.exe andperform2.exe respectively. When the button is clicked, the text "Finished" is written to the edit box. The difference between the programs is that the former is coded to wait for 1/2 second before "Finished" appears; in the latter case, the delay is 1.5 seconds.In both cases, we are setting a performance test that the process take no more than 1 second. Clearly, we should expect example 4a to be ok and example 4b to fail.So we have Example 4afrom example4 import mainmain('perform.exe','Performance Form 1')and Example 4bfrom example4 import mainmain('perform2.exe','Performance Form 2')which reference Example 4:from uncher import launchApp,terminateAppfrom watsup.winGuiAuto import findTopWindows,findControl,getEditText,clickButtonfrom watsup.performance import PerformanceCheck,PerformanceCheckError from time import sleep,timedef main(myExecutable,myWantedText):# find an instance of SimpleForm. If one isn't there, launch itforms=findTopWindows(wantedText=myWantedText)if forms:form=forms[0]else:form=launchApp(myExecutable,wantedText=myWantedText)button=findControl(form,wantedText='Click me')editbox=findControl(form,wantedClass='TEdit')#start a performance check instancep=PerformanceCheck()clickButton(button)# belts and braces to avoid infinite waiting!maxWaitTime=2.0startTime=time()while time()-startTime<maxWaitTime:t=getEditText(editbox)if t:breakelse:sleep(0.1)else:raise Exception,'Failed to get value after maxWaitTime of %s secs' % maxWaitTimetry:try:#do the check/recording step, identifying this step with the wantedtextp.check(myWantedText,1.0)except PerformanceCheckError,e:print '** Failed: %s' % e# and terminate the formfinally:terminateApp(form)if __name__=='__main__':print ' please run example4a or 4b'Key points in example4.pyImmediately prior to clicking the button, we establish a PerformanceCheck instance, which, among other things, starts a timer. Every time the check() method is called on a PerformanceCheck instance, the following occurs:Also if we are CHECKING (the default condition), that time is checked against the number of seconds added as the second parameter. If the elapsed time exceeds the value of the 2nd parameter, an exception is raised.So in example4.py, we see that the check method requires a 1 second acceptance. Hence example4a succeeds and example4b fails.Regression TestingFunctional and performance scripts such as those above are immediately useable within a test framework - the excellent python unit test framework is highly recommended (unittest.py, which comes along with your python installation). This then enables the tester to develop complete regression tests involving any combination of Functional & Performance testing.For an example of the use of functional and performance tests within the unit test framework, run the program framework.bat in the tools subdirectory. This will launch a nice user interface from which sets of test cases can be run:Select the File menu option, and you have the option to load files which contain testcases, or directories/directory trees. If select the "Load files" option, and navigate up one directory, and then down through examples and the unittests directories, you should find, and select, exampleTests.py. The Framework program examines the selected file(s) and extracts the TestCases, presents in the top frameSelecting Menu option Testing - Run Tests (or the long button labelled Run Tests), will cause each test to be run; success or failure is shown in the lower frameFor more information on python's unit test module, refer to the python documentation. Finally, it is worth noting that this framework will work with any unit tests, not just those specifically testing windows application, so you can test elements of the logic that you have written in other applications.Ongoing DevelopmentsThere are a number of undocumented features released in WATSUP, which include ∙the PControl class family, which is a first pass at providing class wrappers for the functions in winGuiAuto.py. The unit test example shows the use of this class.∙the recording of timesteps for subsequent analysis. This is an extension of the PerformanceCheck classs, which has method to support the optional calculation,persisting and retrieving of the actual times taken between the creation of aPerformanceCheck instance and any/all of its check method calls.Mailing Listswatsup-announce: This low volume list is largely for announcements relating to releases of WATSUPwatsup-users: This is for WATSUP users, who have questions and answers on the use of WATSUP.watsup-developers: This is for those who would want to contribute in discussion and beyond to the improvement and development of the tool.DownloadsDownload watsup 0.4.3 here(This package should be unzipped in the site-packages directory in your python installation)Dependencies∙Python (version at least 2.3)∙pywin32∙ctypes∙SendKeys∙wxPython library (for tools)CreditsThe framework test tool was built from parts of unittestgui.py by Chris Liechti. Much of the core functionality in WATSUP derives from the important work by Simon Brunning, who, in his module winGuiAuto.py provided concise, accessible mechanisms to access and "click" windows controls & menus; Simon's work recognises the huge contribution that Mark Hammond has made to the python community (and wider) in providing pywin32, comprehensive win32 API modules for python. And, last, but not least, I'd like to recognise Thomas Heller's work in helping make python and Windows that much closer - through not only ctypes, but also the great exe-making utility py2exe!Dr Tim Couper。