Image detection stops working after a time

rnoecker · Post by **rnoecker** » Fri Jun 07, 2013 4:59 pm

I'm having an issue where image detection stops working after a time (sometimes as soon as the first run). Both the FindImagePos and WaitScreenImage methods start returning -1 and do not start working again until I completely restart the macro program. I've also had some instances where the macro program crashes or hangs indefinitely on image detection both on my development machine and the test server.

I was able to develop the script under Windows 7 with some issues, but once I moved it to a Windows Server 2003 environment the image detection problem started happening very frequently. I've tested on 2 different servers running Windows Server 2003 (32/64 bit).

I am using remote desktop to access the test server and the resolution is 2560x1440 (my desktop resolution) with a color depth of 16 bit. An application is launched using a Citrix client and interacted with using image detection.

When the script works it will wait for the image to become available and then get the coordinates and click that location. Once image detection starts failing both methods return -1 every time according to the debugger and also evidenced by the timeout value not being reached.

I'm using the following pattern.

// reports top level menu
WaitScreenImage>%BMP_DIR%\menu_reports.bmp,%matchCoefficient%,CCOEFF
if>WSI_TIMEDOUT=FALSE
FindImagePos>%BMP_DIR%\menu_reports.bmp,SCREEN,%matchCoefficient%,7,XArr,YArr,NumFound,CCOEFF
If>NumFound>0
MouseMove>XArr_0,YArr_0
LClick
Else
GoSub>ErrorHandler,There was a problem accessing the reports menu
Endif
Else
GoSub>ErrorHandler,There was a problem accessing the reports menu
Endif

armsys · Post by **armsys** » Sun Jun 09, 2013 1:51 am

rnoecker wrote: // reports top level menu
WaitScreenImage>%BMP_DIR%\menu_reports.bmp,%matchCoefficient%,CCOEFF
if>WSI_TIMEDOUT=FALSE
FindImagePos>%BMP_DIR%\menu_reports.bmp,SCREEN,%matchCoefficient%,7,XArr,YArr,NumFound,CCOEFF
If>NumFound>0
MouseMove>XArr_0,YArr_0
LClick
Else
GoSub>ErrorHandler,There was a problem accessing the reports menu
Endif
Else
GoSub>ErrorHandler,There was a problem accessing the reports menu
Endif

Is %matchCoefficient% already initiated?
Have you verified the value of %matchCoefficient%?
MDL>%matchCoefficient%
Why not just write: WaitScreenImage>%BMP_DIR%\menu_reports.bmp,.7,CCOEFF

rnoecker · Post by **rnoecker** » Sun Jun 09, 2013 2:12 am

It is initialized at the top. I'm writing a script that primarily does image detection, I have it declared as a variable so that I can make a single change that affects the entire script which is helpful when determining what is a good value to use.

Although, I have used debugging to jump directly to image detection code, which I assume skips all variable initialization, and it still executes (when it works) so I don't think that would cause it to return -1. That does make me wonder how it is handling a situation where it isn't initialized (assume null or 0 but haven't checked).

Let>matchCoefficient=0.70

armsys · Post by **armsys** » Sun Jun 09, 2013 4:27 am

rnoecker wrote:Although, I have used debugging to jump directly to image detection code, which I assume skips all variable initialization, and it still executes (when it works) so I don't think that would cause it to return -1. That does make me wonder how it is handling a situation where it isn't initialized (assume null or 0 but haven't checked).

NumFound returns the number of matches found.
If an error is encountered, NumFound will be set to -1. E.g. if the needle image is larger than the haystack image, NumFound will be set to -1.
Please verify if menu_reports.bmp is the actual image presumably found on the SCREEN.

armsys · Post by **armsys** » Sun Jun 09, 2013 4:37 am

rnoecker,
1. Which version of MS are you running?
2. How do you create your menu_reports.bmp? I use both MS editor and SnagIt. Both works fine.
3. There's a possibility that the color of the same needle_bitmap may be different on different computers.
4. Make sure the webpage is of the same zoom scale. On FireFox, before running FIP and/or WSI, I always force default zoom scale by:
Let>SK_Delay=1
Press CTRL
Send>0
Release CTRL
Hope it helps.

rnoecker · Post by **rnoecker** » Sun Jun 09, 2013 4:38 am

The images are in the directory and they are found right up until the point it stops working. Also, when I have had a problem with a missing image while developing it is caught by the script editor which pops up a dialog letting me know.

If it is found once it should always be found. There may be a problem with getting the screen after a certain point which could cause the "needle larger than haystack" situation. I'm not sure how to verify this, but I mentioned that I was using a high resolution just in case it was relevant and might cause a problem with the screen.

I can go through on Monday and do some tests and try to replace the match coefficient variable with hard coded numbers just to remove that as a possibility. However, this seems like a bug, and the fact that the software hangs and crashes at times along with other intermittent DLL initialization errors (don't recall the specifics, but I think they were IE related) makes me question the overall stability.

armsys · Post by **armsys** » Sun Jun 09, 2013 4:56 am

rnoecker wrote:If it is found once it should always be found. There may be a problem with getting the screen after a certain point which could cause the "needle larger than haystack" situation. I'm not sure how to verify this, but I mentioned that I was using a high resolution just in case it was relevant and might cause a problem with the screen.

Actually you gave us an important cue. You mentioned the needle_bitmap has been found in the first attempt.
The said needle_bitmap may be locked after the first attempt. Try to elevate your MS privilege to ADMIN.

rnoecker wrote:I can go through on Monday and do some tests and try to replace the match coefficient variable with hard coded numbers just to remove that as a possibility. However, this seems like a bug, and the fact that the software hangs and crashes at times along with other intermittent DLL initialization errors (don't recall the specifics, but I think they were IE related) makes me question the overall stability.

Changing %matchCoefficient% to a constant appears unhelpful in this case because the first attempt of FIP was successful.

rnoecker · Post by **rnoecker** » Wed Jun 19, 2013 6:55 pm

I am already using the Adminstrator account. I gave up using Windows 2003 and am now working with Windows 7 where the software at least works (but has other issues). I can't imagine I'm the only one ever to use the software on Windows 2003, but I tested on 2 different computers and both had the same issue.

armsys · Post by **armsys** » Wed Jun 19, 2013 10:26 pm

rnoecker wrote:I am already using the Adminstrator account. I gave up using Windows 2003 and am now working with Windows 7 where the software at least works (but has other issues). I can't imagine I'm the only one ever to use the software on Windows 2003, but I tested on 2 different computers and both had the same issue.

Rnoecker,
Thanks for your feedback. Hypothetically assuming your screen/window name as "Remote Desktop", try:

Code: Select all

// reports top level menu
WaitReady>1
Let>WSI_Timeout=30
WindowAction>1,Remote Desktop*
Wait>1
WaitScreenImage>%BMP_DIR%\menu_reports.bmp,%matchCoefficient%,CCOEFF
if>WSI_TIMEDOUT=TRUE
  MDL>Menu_Report NOT found. Report to RNOECKER!
  Exit
Endif
FindImagePos>%BMP_DIR%\menu_reports.bmp,SCREEN,%matchCoefficient%,7,XArr,YArr,NumFound,CCOEFF
If>NumFound>0
  MouseMove>XArr_0,YArr_0
  LClick
Else
  GoSub>ErrorHandler,There was a problem accessing the reports menu
  Exit
Endif

Post by **Marcus Tettmar** » Thu Jun 20, 2013 8:53 am

We have identified a memory leak that can occur when using the CCOEFF image recognition engine on some versions of Windows (including Windows 2003 Server). This has been fixed in dev and will be in the next public maintenance release which I will try and get out today if possible.

armsys · Post by **armsys** » Thu Jun 20, 2013 8:55 am

Marcus Tettmar wrote:We have identified a memory leak that can occur when using the CCOEFF image recognition engine on some versions of Windows (including Windows 2003 Server). This has been fixed in dev and will be in the next public maintenance release which I will try and get out today if possible.

Marcus, thanks for your fast support.

Post by **Marcus Tettmar** » Thu Jun 20, 2013 10:10 am

It's up. Please download v14.0.11 from the usual places.

armsys · Post by **armsys** » Thu Jun 20, 2013 10:24 am

Marcus Tettmar wrote:It's up. Please download v14.0.11 from the usual places.

Just download and install it. Thanks for your lightning fast support.
How could we propose to TOIBE that Macro Scheduler be included in the monthly ranking?
Marcus, thanks again.

rnoecker · Post by **rnoecker** » Wed Jun 26, 2013 6:18 pm

Thanks for this. I didn't get an email for any of the replies after my last post so I wasn't aware there were additional ones.

I'm using Windows 7 now for testing, but if I go with this solution I will likely go back to using Windows 2003 for production, and it sounds like this may be the fix for the issue I was having.

Meryl · Post by **Meryl** » Thu Jun 27, 2013 9:46 pm

rnoecker wrote:I'm using Windows 7 now for testing, but if I go with this solution I will likely go back to using Windows 2003 for production, and it sounds like this may be the fix for the issue I was having.

Did it work?

Image detection stops working after a time

Image detection stops working after a time

Re: Image detection stops working after a time