When we are automating Windows applications we nearly always need to be able to detect new windows and wait for windows to open and close and be ready etc.\u00c2\u00a0 Unfortunately there are times when this is not possible.\u00c2\u00a0 Flash applications for example are just exposed as graphics and expose no windowed controls that we can grab a hold of.\u00c2\u00a0 Java applets are also a bit tricky as they use internal objects which Windows doesn’t know about.\u00c2\u00a0 Another tricky scenario is trying to automate Terminal Server or Citrix sessions at the client level.\u00c2\u00a0 The client just gets a graphical representation of the server’s screen.\u00c2\u00a0 There is no access to the objects that make up the user interface within the session unless you are on the server itself.\u00c2\u00a0 So we used to have to install Macro Scheduler on the server if we wanted to automate applications inside the session.\u00c2\u00a0 This is fine if we have access to install stuff on the server.\u00c2\u00a0 There are times when we don’t.<\/p>\n
Another issue that came up recently was a customer that wanted to test the network lag time between a user entering some data into a Windows Terminal Server client and the server serving up the following page.\u00c2\u00a0 The conundrum is that if you install Macro scheduler on the server and make it wait for a new window there is no lag time since all the automation is happening locally to the server.\u00c2\u00a0 But on the client there is no way to know about the new window object.\u00c2\u00a0 Until now.<\/p>\n
Enter the Macro Scheduler Image Recognition Library. \u00c2\u00a0 This includes functions that compare images and find images within larger images.\u00c2\u00a0 In other words we can compare screens and find objects within screens by searching for their image.\u00c2\u00a0\u00c2\u00a0 We can find and click on buttons and other controls and wait for the screen or parts of the screen to change based on what they look like.\u00c2\u00a0 This lets us automate applications graphically rather than having to know about windows and GUI controls.\u00c2\u00a0 In fact this is more analogous to how a user uses a computer.\u00c2\u00a0 The user watches the screen for changes in appearance and finds objects by looking at them with his eyes.\u00c2\u00a0 He looks at what he sees.\u00c2\u00a0 He doesn’t worry about window handles, x and y coordinates.\u00c2\u00a0 So the Image Recognition Library means we can automate any application regardless of the technology it uses to expose it’s interface.\u00c2\u00a0 It works at the graphical level of the screen.\u00c2\u00a0 You can download the Image Recognition Library at http:\/\/www.mjtnet.com\/imagerecognition.htm<\/a><\/p>\n Note: Image Recognition is now native within Macro Scheduler and the separate library is no longer required.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":" When we are automating Windows applications we nearly always need to be able to detect new windows and wait for windows to open and close and be ready etc.\u00c2\u00a0 Unfortunately there are times when this is not possible.\u00c2\u00a0 Flash applications for example are just exposed as graphics and expose no windowed controls that we can […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[6],"tags":[21],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/posts\/7"}],"collection":[{"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/comments?post=7"}],"version-history":[{"count":2,"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/posts\/7\/revisions"}],"predecessor-version":[{"id":1335,"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/posts\/7\/revisions\/1335"}],"wp:attachment":[{"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/media?parent=7"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/categories?post=7"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.mjtnet.com\/blog\/wp-json\/wp\/v2\/tags?post=7"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}