Image Recog. - Can't read the image

Technical support and scripting issues

Moderators: Dorian (MJT support), JRL

Post Reply
stearno
Junior Coder
Posts: 22
Joined: Sun Jul 26, 2020 4:15 am

Image Recog. - Can't read the image

Post by stearno » Mon Jul 27, 2020 7:55 am

On a website, I need the Scheduler to recognozie this image, and then do soemthing when it does. But it does not recognize it at all.
Image

I have tried:
* zoomed into the website to make it bigger
* lowering the coefficient
* making it Exact
* making it look all over the screen
* making it look inside the rectange where it shows up
* adding the blue color on the left (which is always there) to give it a little more color to see
* having it by itself with no blue

What can I do?

Thanks for the guidance.

User avatar
Dorian (MJT support)
Automation Wizard
Posts: 1348
Joined: Sun Nov 03, 2002 3:19 am
Contact:

Re: Image Recog. - Can't read the image

Post by Dorian (MJT support) » Mon Jul 27, 2020 9:16 am

Is it possible the image is changing somehow? The image isn't attached to your post so I can't test. Can you point us to the web page?
Yes, we have a Custom Scripting Service. Message me or go here

stearno
Junior Coder
Posts: 22
Joined: Sun Jul 26, 2020 4:15 am

Re: Image Recog. - Can't read the image

Post by stearno » Mon Jul 27, 2020 9:55 am

No it is static unless another field has a value (which I need to recognize it).

The image is at https://ibb.co/BwnyfrC. It is at middle top, under "imgbb" blue words.

The other place to see it is myfxbook.com, but I dn't think you can access without an account.

User avatar
Dorian (MJT support)
Automation Wizard
Posts: 1348
Joined: Sun Nov 03, 2002 3:19 am
Contact:

Re: Image Recog. - Can't read the image

Post by Dorian (MJT support) » Mon Jul 27, 2020 10:32 am

I'm not experiencing any difficulties with Image Recognition using this image. Certainly incorporating the blue band in your needle image is a good idea, as I've found that sometimes a very subtle image can be a challege.

Both of these were able to identify the (attached) image when I had the image opened in Photo Viewer.

Code: Select all

//Find and Move Mouse Center of 
FindImagePos>%BMP_DIR%\image_3.bmp,SCREEN,0.7,1,XArr,YArr,NumFound,CCOEFF
If>NumFound>0
  MouseMove>XArr_0,YArr_0
Endif

//Find and Move Mouse Center of 
FindImagePos>%BMP_DIR%\image_3.bmp,SCREEN,0,1,XArr,YArr,NumFound,EXACT
If>NumFound>0
  MouseMove>XArr_0,YArr_0
Endif

Could the browser be squashing it somehow? What happens if you open the browser, run the Image Recognition Wizard and sample the image, then run that code immediately? Does it still not see it? Silly question : I am assuming the image is actually completely visible? It has to be visible (on screen) for Macro Scheduler to see it.
Yes, we have a Custom Scripting Service. Message me or go here

stearno
Junior Coder
Posts: 22
Joined: Sun Jul 26, 2020 4:15 am

Re: Image Recog. - Can't read the image

Post by stearno » Tue Jul 28, 2020 8:51 am

No, the icons are static. I have attached what the page looks like normally and then what it looks like when I zoomed it (thinking it needed a bigger icon would be easier to recognize).

I did a couple of tests. It is actually looking for the same size as the bmp picture. For example, I zoomed in really close so the icon was like 1 inch x 1inch. If I opened it in photoviewer, it found it. If I zoomed out in photo viewer to even 90%, it would not find it.

With this result, I went to the screen with your code. I took a bmp of the actual size. It would find it. But as you see in the attached screenshots, there are more than one. But "numfound" = 1. Any suggestions for that?

User avatar
Dorian (MJT support)
Automation Wizard
Posts: 1348
Joined: Sun Nov 03, 2002 3:19 am
Contact:

Re: Image Recog. - Can't read the image

Post by Dorian (MJT support) » Tue Jul 28, 2020 10:00 am

Yes, if you look at the help file for FindImagePos you'll see it says :

"Each method has benefits and drawbacks. CCOEFF is more intelligent and more tolerant but is slower and will return only one match. EXACT is faster and can return multiple matches but is precise and therefore less portable and will not cope with changes."

So using EXACT, you will see that NumFound may be more than 1. In your first screenshot, the image appears twice, and in the second we see it seven times. So we'd expect Numfound to be 2 and 7 respectively.

There's a curved ball though. On screenshot 2, while the image itself may be the same, the background isn't. The table alternates between grey and white. So we have to take that into consideration and start adjusting our tolerances in the hope of finding the sweet spot.

I found that using CCOEFF, it only ever found whichever variation I sampled - grey or white - even if I gave it a tolerance of 1 or 0.1 - the least sensitive. But of course CCOEFF will only ever find 1 match - the most likely candidate.

After experimenting with EXACT, I found a tolerance of 19 always returned NumFound 7. So it successfully ignores the colour difference while finding all the icons.

So if we want to click on, for example, the third icon, we use the appropriate value in the XArr YArr arrays. With 0 being the first match, 1 the second, 2 the third, and so on.

Code: Select all

//Move mouse to first match..
FindImagePos>%BMP_DIR%\image_7.bmp,SCREEN,19,1,XArr,YArr,NumFound,EXACT
If>NumFound>0
  MouseMove>XArr_0,YArr_0
Endif

Code: Select all

//Move mouse to third match..
FindImagePos>%BMP_DIR%\image_7.bmp,SCREEN,19,1,XArr,YArr,NumFound,EXACT
If>NumFound>0
  MouseMove>XArr_2,YArr_2
Endif
One more thing to note is that using EXACT with a tolerance on your entire screen may be very resource hungry. So maybe try to reduce that overhead by making it look within a known rectangle on the screen. When using the wizard under the "Where to look" tab, click "Search Within this rectangle", click Draw, then select your area.
Yes, we have a Custom Scripting Service. Message me or go here

stearno
Junior Coder
Posts: 22
Joined: Sun Jul 26, 2020 4:15 am

Re: Image Recog. - Can't read the image

Post by stearno » Tue Jul 28, 2020 10:45 am

This is great information. Thank you for clarifying about the number found with exact versus COEFFC. I was not aware of that.

So it sounds like I get to play with Exact and the tolerences. Your guidance has pointed me in the right direction. Thank you very much!

User avatar
Dorian (MJT support)
Automation Wizard
Posts: 1348
Joined: Sun Nov 03, 2002 3:19 am
Contact:

Re: Image Recog. - Can't read the image

Post by Dorian (MJT support) » Tue Jul 28, 2020 11:32 am

Always happy to help. A lot of people miss that. Although it is in the helpfile it is quite easy to miss. To be honest, I missed it too when that feature was introduced. :)
Yes, we have a Custom Scripting Service. Message me or go here

Post Reply
Sign up to our newsletter for free automation tips, tricks & discounts