“See” the Screen with Image Recognition
October 11th, 2006 by Marcus TettmarI love the Image Recognition functions in Macro Scheduler 9.0. They make it easy to automate anything. I made a quick and dirty demo showing how to use the Image Recognition functions in v9 to automate a remote Citrix session. You can watch the demo here.
But imagine being able to build an automation script that will automate anything, regardless of the GUI technology or operating system - Web Sites, Java Applications, Desktop Applications, whatever - without writing a line of code. Just tell the computer what to click on by highlighting a screen object, what to do when it finds the object, what to do if it can’t be found, what objects to wait for etc. Wouldn’t that be cool? Well it’s not just a fantasy. We’re doing that right now and some customers are already using it. With this tool you just “teach” your computer what to do. Automation Scripts that took days, can now take minutes. I’m itching to show you, so watch out for a demo soon. If you can’t wait and if you think your company could benefit from something like that, give us a shout.
Bookmark this at:
November 7th, 2006 at 1:56 pm
Marcus, This is just what I was looking for. I need to automate several applications resident on multiple virtual servers and was encountering difficulty because Macro Scheduler was not loaded on each virtual server. Thanks!!!
November 7th, 2006 at 9:40 pm
Will this method work with non windows applications?
I was thinking that possibly image samples could be made using an image editor from a screen shot or running the macro program in windows mode. There are a number of programs using directx and non windows based.
The method to program a macro is the best I have seen. This product is at the top of my list. I also loved the example video. I would pay for video tutorials on macros as there is so much potential but they are confusing for a begineer.
November 8th, 2006 at 10:09 am
Mark,
Yes, that’s the great thing about it. It will work with ANY application, including non-Windows applications. Clearly Macro Scheduler itself must run on a Windows PC but using VNC it could control an application running on another operating system. My video demo is controlling another Windows environment under a Citrix session, but there’s no reason why the remote environment can’t be an X-Windows application, or a Mac PC, or even a command line environment. I would use VNC to access the remote machine and “through” VNC Macro Scheduler can control the non-Windows applications and detect screen objects. That’s the beauty of image recognition. It works by “seeing” the screen and therefore the nature of the underlying technology is irrelevant.
November 10th, 2006 at 2:41 pm
I agree with Mark. Would love to have video examples. I’m a forever beginner at scripting, so if i can see examples on video it’s a huge help.
November 10th, 2006 at 3:29 pm
Would be happy to create more video demos. But I need input. What features would you like me to demonstrate? Any particular functions, or concepts or automation scenarios you’d like to see?
December 30th, 2006 at 7:25 am
I have ordered this software now, was away for a while and came back fresh to it and saw features I thought were not there before, like compatability with non windows based applications. I will think of some things which would be nice to see tutorials of and post later.
January 2nd, 2007 at 7:54 pm
Is text recognition possible for text strings. Could I create images for each letter? Any thoughts?
January 3rd, 2007 at 2:28 pm
Joe,
For OCR (text recognition) see:
http://www.mjtnet.com/blog/2006/06/06/screen-ocr-recognising-graphical-text/
January 3rd, 2007 at 10:22 pm
I’m looking for a macro application that can handle navigating through trees such as Windows Explorer with it being able to detect which folder to find and open. Another case would be picklists where items are dragged and dropped from one pane to another.
Will Macro scheduler with image recognition work in these cases?
April 20th, 2007 at 11:50 pm
I have a graphics applictaion without a batch / automate facility.
I would like to be able to build a macro which would
1) Open the graphics app.
2) Load a graphics file.
3) Perform some operations / manipulation.
4) Re-save the graphics file with a different name.
In addition, to be able to select a folder, and for all graphics files in that folder to have steps 2) to 4) applied to each.
Is this possible with Macro Scheduler?
April 21st, 2007 at 5:56 pm
Hi Trevor,
Yes, it is, although this particular scenario has nothing to do with image recognition in the context of this post. But, yes, you could automate your process with Macro Scheduler. I would suggest you download the evaluation, read the getting started guide, and have a look at the samples/examples in our forum and post any questions there.