Marcus' Macro Scheduler Blog

Every now and then, in the forums and in emails to us, there seems to be some confusion over what these terms mean.  As a developer I take these terms for granted, but anyone else, depending on their walk of life, might think they refer to some kind of Modal Jazz; something to do with a modal matrix used in linear algebra; or a type of auxiliary verb used to indicate modality.  

I sometimes need to remind myself that not everyone coming to Macro Scheduler and Windows Automation in general are familiar with geeky software terms.  I can’t tell you much about modal jazz, linear algebra or auxiliary verbs, but I can do my best to set to rest any confusion about Modal and Non-Modal windows.

Modal Windows

When a window is modal it remains active and focused until the user has finished with it and dismisses it.  While it is active no other windows of the same application can be activated.  A modal window is therefore normally a child window.  The user needs to interract with it before control can be returned to the parent application. In effect the parent application is locked and nothing proceeds until the modal window is closed.

You’ll find a good definition of Modal Windows on Wikipedia, here.

Non-Modal Windows

So a non-modal window is the opposite. While it is active you can still activate other windows. The user can switch between windows of the same application. The window being active does not prevent the rest of the application from continuing.

Modal Dialogs

In Macro Scheduler you can create custom dialogs. These can be modal or non-modal to the script. When we refer to a modal dialog what we mean is that once the dialog is displayed, the script halts until the user closes the dialog. Script execution is paused until the dialog is dismissed. The script cannot do any processing while the modal dialog is active. A modal dialog is displayed with the Show command, with a return variable specified, in which the “modal result” of the dialog is reported, corresponding to which button was pressed to close the dialog.

Non-Modal Dialogs

A dialog can be made to be non-modal. In which case the script carries on even after it has displayed the dialog. The user can interact with the dialog while the script continues to perform other tasks. The developer would need to write code to retrieve the state of the dialog and the data that has been entered into it. Usually this would need to take place in a loop. A non-modal dialog is displayed with the Show command, with the return variable omitted. Inside the loop we would retrieve the state and data of the dialog with the GetDialogAction command, and set it with ResetDialogAction.

In this post I have concentrated on definitions rather than going into detail on how you can create and control custom dialogs. That is a subject for another post and you will find more information in the help file and online help. Some dialog examples ship with Macro Scheduler and you’ll find more in the forums.

What is Screen Scraping?

Screen Scraping is a term used to describe the process of a computer program or macro extracting data from the display output of another application.  Rather than parsing data from the database or data files belonging to an application, Screen Scraping pulls the data from the screen itself, extracting data that was intended to be displayed to the end-user as opposed to data designed for output to another application or database.  

Screen Scraping is necessary when there is a need to access the information displayed by the application but there is no method provided to access it behind the scenes.  The database or data files may not be accessible, or may be undocumented or proprietary and therefore cannot be parsed easily; the costs associated with interacting with the database may be too high; or the license agreement or warrenty prohibits it.  In the case of legacy systems that are no longer supported there may be no knowledge of the data structures, or the technology used is no longer compatible with current technology.  In these cases we are resorted to extracting the data from the screen – from the windows of the application.

The term Screen Scraping probably originates from the era of computer terminals when you could connect the terminal output of a computer to an input port on another and therefore record the screen data.  

Screen Scraping Methods

There are a number of ways we can retrieve information from the screen using Macro Scheduler, depending on the type of application the data is in.

Screen Scraping Web Applications

Applications like Macro Scheduler’s WebRecorder can access the data and objects inside an Interner Explorer window and can therefore be used to extract the data.  Technically speaking I would not call this screen scraping since WebRecorder is using an API interface provided by Internet Explorer, but the process of extracting information from web sites is commonly refered to as Screen Scraping.  With WebRecorder we can use the ExtractTag wizard to create code that extracts the text from a particular element in the page.   While WebRecorder is the easiest way to do it, it is also possible to automate IE and extract data from web pages by using VBScript. The following forum posts may help:

Automate Internet Explorer with OLE/ActiveX
Automate web forms with IE
HTTP GET and POST using VBscript

Screen Scraping Microsoft Office Applications

Microsoft Office Applications, like Internet Explorer, have a COM interface that allows scripts to manipulate them and access the data held within them.  Again, not really scraping data from the screen itself, as you are getting it directly from a programming interface.    There are a number of examples in the forums and blog archives and also some sample scripts that come with Macro Scheduler which demonstrate how to automate Office applications and retrieve data from them.  

Working with Excel

Screen Scraping Regular Windows Applications

Most other applications don’t offer a scripting interface like MS Office or Internet Explorer.  This is where we really need to work directly with the screen.   There are a number of ways we can do this kind of Screen Scraping with Macro Scheduler.

The Text Capture Functions

Macro Scheduler includes some Text Capture functions which can be used to extract text from a given window, rectangular screen area or screen point.  These functions use low level system hooks which monitor applications calling the various “TextOut” functions that Windows uses to output text to the screen.  By doing so they are able to capture this text.  The Text Capture functions return the text to  a variable which you can then use as needed.  

However, a few applications don’t use the Windows built-in functions to create and output text.  Don’t worry –Most do, but a few use their own techniques.  When you realise that text on the screen is just a sequence of small dots, if the application programmer decided to build his own routine to assemble text from dots rather than calling the Windows functions which already do that for you, you’re not going to be able to capture it.

The text capture functions and their limitations are explained here.  There is an example application, here, created with Macro Scheduler, which you can use to determine whether or not the text you want to capture can be captured using the text capture functions.

http://www.mjtnet.com/blog/2007/12/12/capturing-screen-text/
http://www.mjtnet.com/blog/2008/01/03/screen-scrape-text-capture-example/

Using the Clipboard for Screen Scraping

If the text you want to capture is selectable then you can use the clipboard to retrieve it.  A Macro Scheduler macro can send the keystrokes necessary to highlight and copy the text to the clipboard and then use the GetClipboard function to retrieve that text to a variable.  This is far less elegant than using the Text Capture functions but might be necessary if the application concerned is not utlising any of the Windows text out functions to create the text.

SetFocus>Notepad*
//Select ALL
Press CTRL
Send>a
Release CTRL
//Copy to clipboard
Press CTRL
Send>c
Release CTRL

//Get and display the data
WaitClipboard
GetClipboard>theData
MessageModal>theData

Screen Scraping via Optical Character Recognition

A final resort might be Optical Character Recognition (OCR). If the text is part of an image file neither the Text Capture or Clipboard methods are going to be able to extract it. But, depending on the quality of the image and the fonts used, OCR might. There are a number of OCR engines that can be used by Macro Scheduler, one of which is part of Microsoft Office. The following posts demonstrate how this can be done:

Screen OCR to Retrieve Otherwise Undetectable Text
Screen OCR – Recognising Graphical Text

More on Screen Scraping.