May 11, 2009

Test Validation Techniques

Filed under: Automation,Testing — Marcus Tettmar @ 1:42 pm

We recently received the following query to support:

I’m interested in Macro Schedular for GUI testing. How do I verify that the test has succeeded or not? 

I thought it would be useful to post my response here:

There are a number of ways you could do this. Which one you use might depend on the type of application you are automating, or your specific requirements. You could:

  • Capture object text to see if it contains the data you would expect using such functions as: GetControlText, GetWindowText or Windows API functions. See: http://www.mjtnet.com/AutomatedTesting.pdf
  • Capture object and other screen text using the Screen Scraping functions: GetWindowTextEx, GetTextAtPoint, GetTextInRect. Compare captured text to expected data. There’s a sample script called Text Capture which you can use to test what text you can capture. Run it, and point the mouse cursor at the text you want to capture and confirm you can see it on the dialog. See Screen Scrape Text Capture Example and Screen Scraping with Macro Scheduler
  • Compare visually: Capture screen shots (or just windows) and compare the captured bitmaps with bitmaps captured at design time. Use the ScreenCapture and CompareBitmaps functions. This solution has the benefit of working with ANY technology on ANY platform. When you create the routine you capture the screens as they should appear when valid. So at runtime after entering data and controlling the app the macro would capture the screen/window and then compare to the valid images thus determining success or failure. See: How to use Image Recognition
  • There may be other options, especially for non-UI processes, such as reading data from the apps database (using DBQuery) or reading from a text file (ReadLn, ReadFile) or CSV file, checking the existence of files etc – depending on what the application you are testing does and what signifies a valid outcome.

Are you using Macro Scheduler for automated testing?  What types of app are you testing and what methods are you using?  Please comment below.