Image recognition finds icon but does not put cursor correct

[kistonewa]

I am using MS14 with EPIC electronic medical record in a citrix environment. Running Win8.1 on a Surface Pro 3. I am trying to use image recognition to click on the ROS button. I used the image recognition wizard, and I am doing exact matches. I have put a line within the if loop to confirm that the icon is located; however, the cursor is not being moved over the icon. The center of the icon is about 87,90, but the cursor ends up at about 153,158. Cursor consistently moves to the same position. The needle icon is 79X26 colors: 32 bpp. The button is pale blue backgrkound with a verticle stack of 3 small rectangles with center one red. Just to right of this is an icon of a small man in blue suit with yellow head. To right of this is text "ROS". Code is included below. I have to start using EPIC in a couple months. Have not used MS much for several years, but the excessive clicking in EPIC is going to make MS my friend once again.
Thank you for the help.
ken

Code: Select all

// wait place to click on EPIC to be certain it is the active program.
Wait>5

ScreenCapture>0,0,1121,239,%TEMP_DIR%\screenrect.bmp
 //Find and Move Mouse Center of
FindImagePos>%BMP_DIR%\image_2.bmp,SCREEN,0,1,XArr,YArr,NumFound,EXACT
If>NumFound>0
  MouseMove>XArr_0,YArr_0

WriteLn>C:\epic2015ken\epiROSproject\foundimageconfirm.txt,nWLNRes,found image

Endif

[Marcus Tettmar]

I wonder if this is Citrix doing something to the mouse. But can you post a screen shot (taken at runtime just before the FindImagePos) and a copy of image_2.bmp so we can compare and see if we get same results. Maybe it's just a false positive and if we could see the images we might be able to figure out why. Having said that I see you are using EXACT so I wouldn't expect it a false positive.

[kistonewa]

Thank you for the response. I have uploaded to DropBox with the links below:

The screen capture of the epic screen before running the macro:
https://www.dropbox.com/s/eaayhmd6el0wa ... e.bmp?dl=0

The needle image file:
https://www.dropbox.com/s/owy4h40z62fkl ... 2.bmp?dl=0

The file capturing the location of the mouse cursor after the macro is run:
https://www.dropbox.com/s/i37qqoo2133he ... g.jpg?dl=0

One thing I find interesting and perplexing is that if I run the macro on the epic screen the mouse ends up below and to the right in a clear portion of the screen as shown in cursorwrong.jpg. However, if I take the epicscreenbeforecapture.bmp load it into IfranView & size the bmp to 100% the macro finds the needle image but the cursor positions in the small box just to the left of the "well-nourished" text. This is a fair amount lower and probably a few pixels to the right of where the mouse cursor ends up running on the epic program.

[Marcus Tettmar]

Well I'm baffled at this stage.

So I downloaded your screen shot and your needle. I opened the screen shot in image viewer to 100% so it was visible on the screen and then ran the following code:

Code: Select all

FindImagePos>%SCRIPT_DIR%\image_2.bmp,SCREEN,0,1,XArr,YArr,NumFound,EXACT
MouseMove>XArr_0,YArr_0

Where image_2.bmp is the needle which I downloaded to the Macro Scheduler script folder.

The mouse moved successfully to the expected place - right in the middle of the ROS tab. Bingo.

At this stage therefore I don't see an issue with the images or the script. There must be something with your machine which is affecting things.

Out of interest do you have another PC you can do a test on, maybe the same test as me, on a completely different PC. So we can rule out something on the PC.

[kistonewa]

I don't have another computer with me to try things out, but I did vary the screen resolution with interesting results. At the resolutions of 1024X768 and 1280X1024 things work great. AT higher resolutions it didn't. I did not check every resolution; therefore, I don't know exactly where the variances start.

Not being a programmer, my ability to analyze things is somewhat limited; however, I used the tools I have. Using the "Mouse Locater" from Macro Express 3 to determine screen positions and IrFanView to capture bitmaps I found the following descrepancies.
ME3 consistently shows the ROS icon to be at approximately 61,80 when measuring on the EPIC screen.
At 1600X1200 resolution setting IrFanView to capture 61,80 the area actually captured started at 50,62
At 1856X1392 resolution setting IrFanView to capture 61,80 the area actually captured started at 42,55
At 2160X1440 resolution setting IrFanView to capture 61,80 the area actually captured started at 41,56

I don't understand what is happening, but it is clearly not unique to MS14. I am hoping that the offset is consistent on all parts of the screen. That will allow me to program an offset that will allow me to use MS14. I have to run EPIC at the highest resolution my SP3 will run in order use it in the ED.

Do other machines have a similar problem at higher resolutions, or is this unique to the SP3?

[kistonewa]

One additional factor that may be relevant is that I remotely log in to citrix using CAG.

[timi2fly]

I suspect it may be the same issue I raised here :

viewtopic.php?f=2&t=8201

Unfortunately I never got a solution to it.