May 11, 2009

Filed under: Automation,Testing — Marcus Tettmar @ 1:42 pm

We recently received the following query to support:

I’m interested in Macro Schedular for GUI testing. How do I verify that the test has succeeded or not? 

I thought it would be useful to post my response here:

There are a number of ways you could do this. Which one you use might depend on the type of application you are automating, or your specific requirements. You could:

  • Capture object text to see if it contains the data you would expect using such functions as: GetControlText, GetWindowText or Windows API functions. See:
  • Capture object and other screen text using the Screen Scraping functions: GetWindowTextEx, GetTextAtPoint, GetTextInRect. Compare captured text to expected data. There’s a sample script called Text Capture which you can use to test what text you can capture. Run it, and point the mouse cursor at the text you want to capture and confirm you can see it on the dialog. See Screen Scrape Text Capture Example and Screen Scraping with Macro Scheduler
  • Compare visually: Capture screen shots (or just windows) and compare the captured bitmaps with bitmaps captured at design time. Use the ScreenCapture and CompareBitmaps functions. This solution has the benefit of working with ANY technology on ANY platform. When you create the routine you capture the screens as they should appear when valid. So at runtime after entering data and controlling the app the macro would capture the screen/window and then compare to the valid images thus determining success or failure. See: How to use Image Recognition
  • There may be other options, especially for non-UI processes, such as reading data from the apps database (using DBQuery) or reading from a text file (ReadLn, ReadFile) or CSV file, checking the existence of files etc – depending on what the application you are testing does and what signifies a valid outcome.

Are you using Macro Scheduler for automated testing?  What types of app are you testing and what methods are you using?  Please comment below.


  1. The best thing about macro scheduler is that you will only be limited by your imagination. I use and especially love the last method that Marcus mentioned.

    First of all, Marcus posted some great straight forward checks that will work in many cases. However it really depends on what you need to test. MS acts like a human, so you should be able to do everything that a human tester can do. Yet, this might not be very easy or reusable, nor the best solution.

    What is very easy and applicable to many tests is checking the database, as every application/process I am testing is writing to the database. Hence, I often query tables and verify that I always have the same inputs and outputs match the known good set. This works for many scenarios of the application you’re testing. The more data involved, the greater the benefit.

    For instance I have to test a number of regression analysis scenarios. They all very in number of data points, but they involve two curves and a summary of the results. For each pair, I have to check 3 tables. The RegressionResults, DataPoints, and MissingDataPoints (that exist for one of the pair in the time-series, but missing for the other). I have a verification function that uses a Scenario name, a reference code (which is yet another name) and the table name to match a Text file that holds the “good results” set against the current query.

    I use the 3 parameters to (Scenario name, reference code and table name) to generate the text file’s name, insuring that there will always be a logical (and easily identifiable) match.

    With a push of a button I can save hours of verification time while insuring that not of field is missed. It’s simplicity sublime

    Comment by Antonius Momac — May 18, 2009 @ 1:55 pm

  2. Thanks for that Antonius. Good to hear that method works well for you. Of course not everyone will have access to their application’s database and not every application to be tested will have a database. So not everyone can use this approach. There might also be situations where reading the data is not sufficient to fully test a UI element, e.g. where data is validated at the user interface before being posted to the database – you’d have to monitor the on screen output to verify correct operation. While you could tell from reading the database that an invalid entry was not accepted, you wouldn’t know from that how the UI was reacting. I think this is where scraping screen text and capturing window images and comparing them to expected outcomes is so useful, especially as it is independent of the technology used to create the interface.

    Comment by Marcus Tettmar — May 18, 2009 @ 2:02 pm

RSS feed for comments on this post. TrackBack URL

Leave a comment