Comments on: Test Validation Techniques http://www.mjtnet.com/blog/2009/05/11/test-validation-techniques/ Mostly tips, tutorials, articles and news about Macro Scheduler & Windows Automation Sun, 31 Jan 2010 18:53:41 +0000 http://wordpress.org/?v=2.8.4 hourly 1 By: Marcus Tettmar http://www.mjtnet.com/blog/2009/05/11/test-validation-techniques/comment-page-1/#comment-2350 Marcus Tettmar Mon, 18 May 2009 14:02:47 +0000 http://www.mjtnet.com/blog/?p=618#comment-2350 Thanks for that Antonius. Good to hear that method works well for you. Of course not everyone will have access to their application's database and not every application to be tested will have a database. So not everyone can use this approach. There might also be situations where reading the data is not sufficient to fully test a UI element, e.g. where data is validated at the user interface before being posted to the database - you'd have to monitor the on screen output to verify correct operation. While you could tell from reading the database that an invalid entry was not accepted, you wouldn't know from that how the UI was reacting. I think this is where scraping screen text and capturing window images and comparing them to expected outcomes is so useful, especially as it is independent of the technology used to create the interface. Thanks for that Antonius. Good to hear that method works well for you. Of course not everyone will have access to their application’s database and not every application to be tested will have a database. So not everyone can use this approach. There might also be situations where reading the data is not sufficient to fully test a UI element, e.g. where data is validated at the user interface before being posted to the database – you’d have to monitor the on screen output to verify correct operation. While you could tell from reading the database that an invalid entry was not accepted, you wouldn’t know from that how the UI was reacting. I think this is where scraping screen text and capturing window images and comparing them to expected outcomes is so useful, especially as it is independent of the technology used to create the interface.

]]> By: Antonius Momac http://www.mjtnet.com/blog/2009/05/11/test-validation-techniques/comment-page-1/#comment-2349 Antonius Momac Mon, 18 May 2009 13:55:34 +0000 http://www.mjtnet.com/blog/?p=618#comment-2349 The best thing about macro scheduler is that you will only be limited by your imagination. I use and especially love the last method that Marcus mentioned. First of all, Marcus posted some great straight forward checks that will work in many cases. However it really depends on what you need to test. MS acts like a human, so you should be able to do everything that a human tester can do. Yet, this might not be very easy or reusable, nor the best solution. What is very easy and applicable to many tests is checking the database, as every application/process I am testing is writing to the database. Hence, I often query tables and verify that I always have the same inputs and outputs match the known good set. This works for many scenarios of the application you’re testing. The more data involved, the greater the benefit. For instance I have to test a number of regression analysis scenarios. They all very in number of data points, but they involve two curves and a summary of the results. For each pair, I have to check 3 tables. The RegressionResults, DataPoints, and MissingDataPoints (that exist for one of the pair in the time-series, but missing for the other). I have a verification function that uses a Scenario name, a reference code (which is yet another name) and the table name to match a Text file that holds the “good results” set against the current query. I use the 3 parameters to (Scenario name, reference code and table name) to generate the text file’s name, insuring that there will always be a logical (and easily identifiable) match. With a push of a button I can save hours of verification time while insuring that not of field is missed. It’s simplicity sublime The best thing about macro scheduler is that you will only be limited by your imagination. I use and especially love the last method that Marcus mentioned.

First of all, Marcus posted some great straight forward checks that will work in many cases. However it really depends on what you need to test. MS acts like a human, so you should be able to do everything that a human tester can do. Yet, this might not be very easy or reusable, nor the best solution.

What is very easy and applicable to many tests is checking the database, as every application/process I am testing is writing to the database. Hence, I often query tables and verify that I always have the same inputs and outputs match the known good set. This works for many scenarios of the application you’re testing. The more data involved, the greater the benefit.

For instance I have to test a number of regression analysis scenarios. They all very in number of data points, but they involve two curves and a summary of the results. For each pair, I have to check 3 tables. The RegressionResults, DataPoints, and MissingDataPoints (that exist for one of the pair in the time-series, but missing for the other). I have a verification function that uses a Scenario name, a reference code (which is yet another name) and the table name to match a Text file that holds the “good results” set against the current query.

I use the 3 parameters to (Scenario name, reference code and table name) to generate the text file’s name, insuring that there will always be a logical (and easily identifiable) match.

With a push of a button I can save hours of verification time while insuring that not of field is missed. It’s simplicity sublime

]]>