June 19, 2014

Load Testing – Measuring Elapsed Time

Filed under: Automation,Macro Recorder,Testing — Marcus Tettmar @ 12:02 pm

In a support email recently I was asked how to record a macro to perform load testing.

While you could certainly use the macro recorder for much of the process, in most cases you are probably going to want to insert some code to make the macro more dynamic in waiting for events to complete. Your load testing is going to slow things down and what you want is to measure how long things take. So you’ll need to be sure you’re waiting for “readiness” and calculating how long that took.

The macro recorder by its very nature records the firing time between distinct events like mouse clicks and keystrokes exactly as it happened at play back. It does however insert “WaitWindowOpen” commands when new windows appear and these will be dynamic – they will wait however long is needed for windows to open and close.

But more often than not you’ll want other forms of waiting. My personal recommendation is to use WaitScreenImage to wait for a visual cue on the screen. This is closely analogous to how a human uses the application – she uses her eyes to see what’s on the screen and determine “readiness”.

If you’re load testing then you’re probably going to run lots of macros – or virtual users – at the same time and are going to slow things down. This is exactly what you want to do – you want to see what the effect of adding more users is – and you’ll need to measure elapsed time. So you’ll need the macro to wait exactly as long as necessary.

The easiest way to do that is with Image Recognition. By all means use the macro recorders to record the simple bits – the bits that send keystrokes into fields. Or use the UI functions and the handy Find Object Wizard.

Also use the Screen Image Recognition wizard to record your waits. It’s real easy. Just capture the thing on the screen you want to wait for and it will spit out the code to do that.

So once you’ve done that you have a macro which will carry out the process and wait for things to complete and be ready before continuing. Now you will want to gather statistics to see what the elapsed times are and compare with different scenarios.

You might do this simply by looking at the log file for the macro. The log file outputs the time each command executed. So you could calculate the elapsed time between starting a step and when it completed. I’ve known customers simply open the log files in Excel and add their own formulas.

But I would make use of the Timer function to have the script itself measure the elapsed time of specific events. You might then output this elapsed time to an Excel or CSV file or store it in a database. It’s up to you.

Here’s a simple bit of code which sets a timer, runs Notepad, waits for it’s main window and then calculates elapsed time, storing it in a CSV file.

//start the Timer
Timer>startTime

//Run Notepad as an example
Run>notepad.exe
//wait for it to be ready
WaitWindowOpen>Untitled - Notepad

//How long did that take?
Timer>endTime
Let>elapsed=endTime-startTime

//elapsed is in milliseconds - write it to our CSV file
WriteLn>c:\temp\test_1.csv,wRes,STEP1;%elapsed%

This is really just to show you how the Timer function allows you to measure elapsed time – you’ll likely have a bit more going on and if “readiness” is not signalled by a new window as it is here then WaitScreenImage might be a better option.