December 5, 2006

Web Automation in Vista

Filed under: Announcements,Automation,Scripting,Vista — Marcus Tettmar @ 1:51 pm

To run WebRecorder, or VBScript macros which automate Internet Explorer in Vista (IE7), you will first need to disable “Protected Mode”. Consider this simple VBScript code:

Dim IE
Set IE = CreateObject(“InternetExplorer.Application”)
IE.Visible=1
IE.Navigate “http://www.mjtnet.com”

In IE6 and in IE7 under XP this code would create a new IE instance and then navigate it to mjtnet.com. But Vista adds something to IE called “Protected Mode” and for some reason Protected Mode causes the Navigate method to open up a new IE window. This is most annoying because it means any subsequent code in a script like the one above would fail because IE object refers to the instance created with the CreateObject call, but the page is now in the second instance.

You can disable “Protected Mode” in IE under Tools/Internet Options/Security. This puts things back to normal and means code like the above will continue to work.

One known issue with WebRecorder’s IEAuto.DLL is that the WaitNewIE function hangs when using IE7. We’re working on this now. This function is only used when an action in IE causes a new window/tab to appear. Apart from that, once you have disabled “Protected Mode” WebRecorder/IEAuto works fine with IE7/Vista.

Update: Version 1.78 is now available, fixing the issue with WaitNewIE on IE7.

October 26, 2006

Brilliant Image Recognition

Filed under: Automation,General — Marcus Tettmar @ 2:16 pm

Wells Anderson posts:

“Image Recognition is a brilliant new feature in 9.0. It allows you to write macros that press buttons that are “invisible” or have unpredictable locations. For example, add-ins to MS Outlook and other MS Office applications may have no menu equivalents and cannot be found using PushButton or FindWindowWithText. Image Recognition lets you find buttons and other screen elements and then perform actions on them, such as mouse clicks.”

He then goes on to share a great tip on how to deal with different font smoothing options. Check it out.

September 15, 2006

Running UI Automation Routines Concurrently

Filed under: Automation — Marcus Tettmar @ 11:55 am

Macro Scheduler is able to automate other applications by simulating what a user does. There are some exceptions – e.g. it is possible to automate Internet Explorer and Microsoft Office applications through their COM interfaces. But for the vast majority of applications the only way to automate them is by simulating user input and manipulating their UI (User Interface) just as an ordinary user does. Therefore Macro Scheduler has to simulate a human to do what it does best.

Two people would have a hard time trying to use the same PC at the same time. Imagine Fred and Sally sitting in front of the same computer, both trying to use the keyboard at once. Fred is trying to write a letter in Word and Sally is writing an email in Outlook. At the same time. Can you imagine what would happen? Utter chaos is what would happen. Fred opens Word and starts typing. Sally grabs the mouse and opens Outlook. In an instant focus has switched away from Word and Fred’s keystrokes land elsewhere. Outlook hasn’t finished opening yet and for a brief while the Desktop was the active window. Fred’s keystrokes therefore landed on some icons and started opening other applications randomly. Nightmare. Miraculously they manage to sort that out and now Sally has Outlook open as the active window and starts typing. Fred quickly hits ALT-TAB to bring Word back and types some more of his letter. But Sally is still hitting keys on the keyboard so parts of her email end up in Fred’s letter. I think you get the picture. It’s more like a game of “Tug Of War”.

The only way two people could effectively use the same computer is to take turns. Either Fred writes his letter and then tells Sally she can write her email, or, if they want to do bits at a time it might work like this:

Fred: “I’m typing a letter in Word, please don’t touch the keyboard”
Sally: “OK, I’ll wait”
Time Passes
Fred: “OK, I’m done writing in Word for a while, your turn”
Sally: “OK, I’ve activated Outlook. Please don’t touch the keyboard until I say”
Fred: “OK, I’ll wait”
Time Passes
Sally: “OK, I’m done, your turn”

You get the idea. Now imagine that there’s a little red flag on top of the monitor. When Fred starts his letter he raises the flag. When he’s ready and happy to let Sally take over, he lowers it. Sally can now take over, and raises the flag. Essentially they are using a form of semaphore to coordinate their sharing of the computer.

Two Macro Scheduler scripts could do the same thing. Only instead of a little red flag on top of the monitor we could use a value in an INI file, or a registry entry. We’ll call this value RunStatus and it could be initialised to IDLE. Script A needs to focus a window and send some keystrokes to it. It first checks to see if RunStatus is IDLE. If not it waits until it is. It then sets RunStatus to ACTIVE and does what it needs to do before setting RunStatus back to IDLE. Script B meanwhile wants to activate Outlook and click on a toolbar button. It finds RunStatus is IDLE, so waits until Script A has set it back to ACTIVE, and so on. We might put these simple subroutines in both scripts:

SRT>WaitForIdle
  ReadIniFile>ctrl.ini,Share,RunStatus,IsIdle
  Wait>0.2
  If>IsIdle=ACTIVE,WaitForIdle
  EditIniFile>ctrl.ini,Share,RunStatus,ACTIVE
END>WaitForIdle
SRT>SetIdle
  EditIniFile>ctrl.ini,Share,RunStatus,IDLE
End>SetIdle

And then before any part of the script where we need to manipulate the User Interface we can call the WaitIdle subroutine:

GoSub>WaitForIdle

And when we’re done activating windows, or sending keystrokes, we can do:

GoSub>SetIdle

We’ve implemented a very simple form of semaphore for task communication in a multitasking environment. Using this approach we can allow two UI scripts to run concurrently without interfering with each other. If necessary, we can get a bit cleverer. If Script A must perform some automation only after the Script B has completed a specific task, we could create a different flag, or set the flag to a certain value. Script A waits for this value before it can continue.

Clearly it is expensive and wasteful to use one PC for each automation routine you need to run, so solutions like this save money and this is one approach you can use. But depending on the complexity of your scripts you may need to add lots of status checks and pauses. A much simpler solution is to use Virtualization. Install Microsoft Virtual PC, or VMWare and run each automation in its own virtual machine. Effectively you end up multiplexing the physical PC into several virtual PCs and your scripts can run safely in their own virtual machines without interfering with each other.

But on those occasions you need two UI scripts to run at the same time and interact with the same environment, our simple little flag hoisting solution is one you can use. Simple, but effective. But note that at this stage we haven’t considered file-locking and what happens if both scripts read or try to change the flag at exactly the same time. A more robust solution might be to use a value in a SQL database and take advantage of the database’s built in record locking.

A similar approach can even be used to control distributed scripts. With scripts on multiple networked computers, and the INI file in a network share (or using a SQL database), a controlling script can be created which modifies the flag. The networked scripts wait for the active flag before doing anything, and can be paused when the controlling script changes the flag. And flags can be used to synchronize scripts on different machines. This approach has been used successfully in load testing environments where hundreds of Macro Scheduler scripts are simulating tasks simultaneously to create load on the system being tested. You can read more about using Macro Scheduler for automated testing here.

September 7, 2006

Get Selected Tab Index

Filed under: Automation,Scripting — Marcus Tettmar @ 6:51 am

As you probably already know, selecting a page of a tab control is usually best achieved by sending CTRL-TAB. So on opening an application we can use the following Macro Scheduler code to jump to the third tab:

SetFocus>Window_Title
Press CTRL
Press Tab * 2
Release CTRL

This issues CTRL-TAB twice and therefore cycles through two tab selections, leaving the third tab selected.

Usually when the window is first opened, the first tab is selected. But what if that is not always the case? How can we determine which tab is currently selected? This question came up in the forum recently.

To do this we can use the Windows API and send the TCM_GETCURSEL message to the tab control. When this message is sent to the control it returns the index of the selected tab.

Here’s an example. Right click on the desktop to open the Desktop Properties dialog and select one of the tabs to change the page. Then run this script:

//Initialise constants
Let>TCM_GETCURSEL=4875

//Get the handle of the Display Properties Window
GetWindowHandle>Display Properties,hwnd

//Find the handle of the SysTabControl32 (Tab Control) object
LibFunc>User32,FindWindowExA,tabctrl,hwnd,0,SysTabControl32,

//Send the TCM_GETCURSEL message to retrieve the selected tab index
LibFunc>user32,SendMessageA,tabindex,tabctrl,TCM_GETCURSEL,0,0

//Display the result
MessageModal>Selected Tab: %tabindex%

This finds the handle of the SysTabControl32 object on the Display Properties dialog and sends it the TCM_GETCURSEL message to retrieve the selected tab index.

You can use this approach with other applications. However, some applications use a different class of tab control, so you’d need to determine the class name of the tab control using the View System Windows tool in Macro Scheduler.

July 13, 2006

Activating System Tray Icons

Filed under: Automation,Scripting — Marcus Tettmar @ 8:47 am

This one has bugged me for years – how to automate activating a system tray icon. For starters, the System Tray is designed to be manipulated with the mouse. We could send mouse clicks to icons in the system tray but for the fact that icons don’t always appear in the same place in the tray. Some systems contract and expand the system tray and on startup icons can be reshuffled and appear in a different order. So sending mouse events is no good.

Well it turns out there is a way to control the system tray with the keyboard. I hadn’t realised this. Thanks to Henk for discovering this. Henk demonstrates here that if you press the Windows key to open the start menu, then Escape to close the start menu keyboard focus is now on the Start button. If you then press Tab a few times focus ends up in the System Tray. If you hover over a System Tray icon you will see a Tool Tip appear above it showing the name of the application. Well the first letter of this text will select this icon. So once you’ve tabbed into the System Tray you can now press the letter key corresponding to the first letter of the icon’s title and it will be selected. Then pressing Enter will activate it.

The number of tabs you need to send to get to the System Tray depends on how you have your task bar set up. If you have the QuickLaunch toolbar enabled you’d need an extra Tab. And in XP with the contracting/expanding System Tray you need to press the right arrow key to move off the contract/expand button. So this approach is not entirely portable but will work on your system.

So I gave this some more thought and came up with a portable approach which focuses the System Tray area directly – without having to open the Start menu and tab around. My approach uses the Win32 API functions FindWindow and FindWindowEx to find the handle of the System Tray Notification Area and then focus it directly. Here’s the code:

Let>WIN_USEHANDLE=1
LibFunc>User32,FindWindowA,h1,Shell_TrayWnd,
LibFunc>User32,FindWindowExA,h2,h1,0,TrayNotifyWnd,
LibFunc>User32,FindWindowExA,h3,h2,0,SysPager,
LibFunc>User32,FindWindowExA,h4,h3,0,ToolbarWindow32,Notification Area
SetFocus>h4
//Change to first letter of icon to activate
Send>v
//If more than one with same first letter, then repeat above line for each
Press Enter

Thanks again to Henk for the tip. See the forum post here.

July 5, 2006

Automated Adobe Reader Install – Waiting for Window Content

Filed under: Automation,Scripting — Marcus Tettmar @ 6:51 am

Here’s a script to automate the installation of Adobe Reader:

http://www.mjtnet.com/usergroup/viewtopic.php?t=3034

This script demonstrates how to wait for windows to change based on the text within them. As with the majority of installers, Adobe Reader’s installer has a series of windows with the same title with differing content, and “Next” buttons to progress through the installation. Since the window title remains the same we can’t wait for the next window based on the window title so we must wait for the content to change. My install script demonstrates use of the FindWindowWithText function inside loops to show how to wait for specific text within the window. Using this approach we can wait for each step in the install to complete.

An alternative method that some might prefer is to use a graphical approach using the Image Recognition Library. Using this we could write a script that waits for an enabled “Next” button and clicks on it when it is found, for example.

June 12, 2006

Automated Google Rank Checker

Filed under: Automation,Scripting,Web/Tech — Marcus Tettmar @ 10:43 am

I’ve just had a bit of fun writing a couple of small scripts that will search Google for a set of key phrases and find the position of a specified URL for those key phrases in the results. I have built two versions of this script. One with a simple dialog where you can enter a URL and key phrase to search for, and another which takes a set of key phrases in an input file and produces a CSV file with the results and date. This second script can be scheduled to take place every night. The CSV file will grow with the new results and can be opened in Excel and a chart produced showing the trend over time of your website position for those key phrases.

As well as being extremely useful as an automated way to monitor the performance of your keywords in Google, these scripts also demonstrate what can be done with some simple Macro Scheduler scripting and WebRecorder. The scripts are not long and give a glimpse at the sort of things WebRecorder and Macro Scheduler can do to automate web sites and parse the resultant HTML.

Download the scripts here:
http://www.mjtnet.com/usergroup/viewtopic.php?t=2983

Note that these scripts automate IE and therefore operate at the user level just as if you were using Google yourself.

June 6, 2006

Screen OCR – Recognising Graphical Text

Filed under: Automation,Scripting — Marcus Tettmar @ 10:26 am

[UPDATE: 27.3.2018 – Since this article was written Macro Scheduler now has built in Screen OCR functions]

We often need to write scripts that extract text from a window or some area of the screen. Usually this can be achieved via the clipboard, or commands such as GetWindowText and GetControlText, or via a Win32 API function. However, this only works with text that really is text, and with objects that expose text as a property of itself. For example, many Windows controls have a caption property which exposes the text associated with it. In this instance we can write code to capture the text by accessing that caption property.

But don’t forget that anything you see on the screen is really graphics. Text is rendered using fonts and fonts are just made up of a combination of coloured pixels. Not all text that you see on the screen is accessible via a simple object property. Sometimes the text you see is just pure graphics. Yes, it reads like text to us humans – we can make sense of it. But to the computer it is a sequence of dots. An arrangement of pixels. So how can we write a script to ‘read’ that text?

Well in these cases where the text is not exposed as a textual object, and is only represented graphically, the only way forward is to use Optical Character Recognition (OCR). There are many OCR packages available on the market which will read an image file, scan it for recognised characters and output ASCII text. So one approach would be to automate one of these OCR applications with Macro Scheduler. Macro Scheduler could first take a screen shot (using the ScreenCapture function) and then automate an OCR application, having it read the screenshot and output the text to a file.

But a less cumbersome approach is to use an excellent tool called Textract from Structurise. Textract is a library that will scan a window, screen area or image file, and output recognised text. Textract comes with a command line interface that makes it ideal for use within a Macro Scheduler script. Here is a small example script which scans the foreground window and retrieves all the text within it:

//Path to Textract
Let>TxPath=c:\\\textract

//Focus the App if necessary
//SetFocus>Notepad*

//Get bounds of active window
GetActiveWindow>title,x1,y1,w,h
Let>x2=x1+w
Let>y2=y1+h

//Set Run to run hidden and to wait for completion
Let>RP_WAIT=1
Let>RP_WINDOWMODE=0

//First time you run this on a new system uncomment next line
Run>%TxPath%\\\textra.exe /build

//Run Textra to retrieve all text in window.  Make sure we run hidden!
Run>%TxPath%\\\textra.exe /capture %x1% %y1% %x2% %y2% /ascii %TxPath%\\\tmpOutput.txt

//Read from output file into script variable and delete file
ReadFile>%TxPath%\\\tmpOutput.txt,TheText
DeleteFile>%TxPath%\\\tmpOutput.txt

//Just as an example display the text
MessageModal>TheText

This script simply displays all the text found in the entire window in a message box. You would probably want to parse the text in some way. Or you may just want text from a specific part of the window. You can do that by making the screen coordinates more specific and making the rectangle smaller. This is a really slick way of identifying text that isn’t otherwise visible as text to Macro Scheduler or Windows.

To get a list of options type textra.exe on the command line and see the documentation that comes with Textract.

You can download Textract from:
http://www.structurise.com/textract/

May 16, 2006

Switch off to Save the Planet

Filed under: Automation,General — Marcus Tettmar @ 8:31 am

Recent research from Computacenter and Fujitsu Siemens finds that the 200 largest UK companies waste £61m a year on unneeded electricity by using power-hungry, inefficient PCs. This wasted electricity equates to over 2.8 million kWh of energy.

Of course the recommendations from Computacenter and Fujitsu Siemens are that these businesses should throw out their desktops and replace them with “Green-PCs” made and supplied by, say, Computacenter or Fujitsu Siemens. Though, I’m not so sure that throwing out thousands of perfectly good desktop computers is that great for the environment either!

In fact the report also finds that “more than a third (37%) of workers don’t switch off PCs when they leave the office” and “by adopting green IT equipment and switching off desktops when not in use the top 200 companies will make annual savings of around £305,000 each”. One of the largest UK corporations could save up to £2m by adopting such policies.

So for those in a rush to get out the door at 5pm, or who forget to switch off their machine, why not create a simple Macro Scheduler macro which shuts down the PC automatically at, say, 6pm:

ShutDownWindows>0

Just in case you might be working late you could add a 30 second warning allowing you to cancel:

Let>ASK_TIMEOUT=30000
Let>Prompt=Shutting Down in 30 Seconds. No to Abort, Yes to Shutdown now.
Ask>Prompt,cont
If>cont=YES
  ShutDownWindows>0
Endif

May 2, 2006

Changing CMD’s Window Title

Filed under: Automation,Scripting — Marcus Tettmar @ 3:28 pm

When running command line tools, applications or batch files it is often necessary to watch the resultant command window. E.g. to detect their completion you can have the script wait until the command window has closed. The trouble with this is that when you run batch files the title of the command window is always just “C:\WINDOWS\system32\cmd.exe”. So if your script starts more than one batch file, or you have other CMD windows open, how do you know which CMD window is which?

Well how about predefining the window title of the new CMD window? We can do this with the Windows start command. The start command starts a new CMD window and lets you give it any title you want and run a program/command at the same time. For more info type start /? at the command prompt. So we can use this in Macro Scheduler with the Run Program command:

Run Program>cmd /c start "Window Title" "c:\bla\something.bat"

This enables us to monitor this specific window to see when the batch file completes:

Run Program>cmd /c start "My Window One" "c:\stuff\reports.bat"
WaitWindowOpen>My Window One*
WaitWindowClosed>My Window Two*
MessageModal>Finished!

Now you might be wondering why we’d want to do the above when you can just make the script wait for the program or command run to finish by setting RP_WAIT to 1, like this:

Let>RP_WAIT=1
Run Program>c:\stuff\reports.bat

Well, you may need to manipulate the window at some point during the process. Or you might want the script to detect if the batch file is taking too long to complete, like this:

Run>cmd /c start "My Window" "d:\test.bat"
WaitWindowOpen>My Window*
Let>WW_TIMEOUT=120
WaitWindowClosed>My Window*
If>WW_RESULT=FALSE
  //Timed out, email Administrator
  SMTPSendMail>[email protected],..........
Endif

Using the start command like this to set the CMD window’s title makes it much easier to distinguish between other command windows and ensure we manipulate the one belonging to our batch file.

« Newer PostsOlder Posts »