Pixel color of an image

[mafiamoe]

Anyone know of a VBscript that can get the pixel color of a saved image (.bmp)?

[mafiamoe]

Well I am looking to find a pixel color of an image that is not on the screen but saved in a .bmp format. I have been searching around for code to do this and while I have found some examples in a VBscript format, I am not well versed in VBscript and have failed 'translating' it to work in MS.

FIP = FindImagePos.

[Marcus Tettmar]

To make FindImagePos 100% accurate set FIP_SCANPIXELS to the same number of pixels as are in the image.

[mafiamoe]

But if i were to try to get 100% accuracy with FIP i would need to create an image of each pixel color and then scan the image for every color. That would be 16777216 different images...

A VBscript would hopefully bring all that complexity down to something like 'ImagePixel(C:\ImageLocation\Image.bmp,X,Y,result)'

[gdyvig]

It sounds like you want to know the color of a particular pixel in an image, not find the location(s) of an image.

GetPixel>X,Y,result almost does what you want.
The problem is it works with screen coordinates, not with coordinates in an undisplayed bmp file.

What were you going to do with this information?

[mafiamoe]

What were you going to do with this information?

Take over the world of course...

This would be to run through the image data gathered to basically test the commands upon all previous runs of the program. Basically a personal testing system that gathers and tests the program because I don't have the time to do all that every time I make a change...

Its just a side project I'm looking into the feasibility of really, but it could turn out to be something pretty cool.

[gdyvig]

You should not need the actual pixel color codes to test your app, FIP should work great. We are using MS and AppNavigator for our testing.

We found the problem with object orientated test tools like WinRunner, QTP, SilkTest, etc is the partial or nonexistant support for the objects under test. As new technologies are created the problem gets worse.

These tools have image based workarounds, but they need the script to specify the image location on the screen. We tried some workarounds based on a WINAPI command similar to GetPixelColor to find the images. It worked but was nowhere as well developed or convenient to use as FIP. If you have bmp's the images your application will display, you can write a script before the application is coded.

MS and AppNav are great because the same solution works for all technologies runnable on Windows. Because the solution is image based it works in development, independent test, and production environments.

[mafiamoe]

But there are some instances where FIP simply doesn't work.

[gdyvig]

What scenario is giving you problems?

[Marcus Tettmar]

It sometimes seems to surprise people that the only reason FindImagePos will fail to find an image on the screen is because .... that image is NOT on the screen! I've had people swear blind the image they are seeking is on the screen but when I've opened up the screen image and the needle image in Paint I can see big differences.

Here are some common mistakes I've seen that can lead to not finding the image you want it to find (or rather, looking for the wrong image):

Mistake 1: Not using the built in Image Capture Tool. ALWAYS use the Macro Scheduler Image Capture tool. Other image editors may save with a different colour depth etc. To ensure like-with-like comparisons ALWAYS use the Macro Scheduler Image Capture Tool.

Mistake 2: Capturing the object when it is in a different state to how it appears at run time. E.g. buttons look different when they are focused, menus look different when they are selected, tooltips appear when the mouse is over objects, sometimes even window captions appear different when the window is not focused, sometimes objects look different when the mouse is hovered over them. Take all these things into consideration and make sure you capture the image as it will appear at the point in the process where you want to find it. You may need to use the delayed capture feature of the Image Capture Tool so that you can arrange focus before you capture the object. Consider making the script move the mouse to 0,0 in case the mouse ends up hovering over the button you're looking for at runtime. I've seen buttons look different if the window is focused. If you have the window focused when you captured it you'll need to make the script focus it before looking for it. Or vice-versa. Etc etc

Mistake 3: Capturing too much of the screen. FindImagePos compares a random sampling of pixels in the needle image and target area. So if you capture an area with lots of background it's feasible that only the background pixels get chosen and you'll end up with lots of matches. Be more specific in your captures, and if you need to, increase FIP_SCANPIXELS to make it scan more pixels.

Mistake 3a: Thinking it didn't find the image when in fact it DID find it, but it also found other possible matches. The function returns the number of images found and an array of matches. If you are clicking on the first match and it is not the right place, look at the number found. Consider what to do if number of matches is more than 1.

Mistake 4: Looking for the image at the wrong time - getting the order of events all wrong.

Some issues that can cause problems:

Graduated or patterned backgrounds. If you capture an object that is on a non-smooth background and have to capture some of the background as well, then if the object appears at a different position at runtime you may fail to find a match, because the background is not smooth. As with Mistake 3 above you probably just need to be more specific about what you capture. Or, if that isn't possible, see if you can change the background. There's no harm in making life easier for ourselves by setting the environment up to be a little more predictable so that we can automate/test more reliably.

Sometimes it's helpful for diagnostic purposes to use ScreenCapture to capture the screen to a file prior to your FindImagePos call. Then, later, if it didn't work as you expected you can open up the screen image and work out why.

[mafiamoe]

If there is a changeable background color in a program and that effects something you are trying to find, FIP breaks down. I have put together a system that checks for bright and/or dark pixels and it works where FIP does not. Before FIP was introduced, this was the method used to find everything. FIP being able to find an image in any location is a natural benefit and has been introduced wherever possible. FIP works very well in the places that it can be used.

[gdyvig]

I experimented with that problem when evaluating the product.

Depending on the colors involved, sometimes increasing the color tolerance will result in a FIP finding the image in spite of a variable background. I tested this method by looking for an icon and its label against a desktop wallpaper that was a photograph of a water scene. When I found the right balance, FIP could find the correct icon/label in a variety of locations.

Usually the best way is to make your needle smaller, just enough of the icon to be unique while avoiding the background.

Another possiblity is to capture the screen and have the script call 3rd party software to modify the haystack to make the needle easier to find. Is your system of looking for bright/dark pixels such a 3rd party or custom code you call from Macro Scheduler?


Others have posted a request for MS to support transparent color. I think the effect is to ignore the background.

[mafiamoe]

I use MS to check the pixel colors. Have it down to a simple-ish subroutine. The problem with FIP now is the background color can vary greatly, and some of the images I need to find are text only so the background color is not only around the letters, but also in the spaces of the letters as well.

This topic has gone so far off track from my original idea I think the workaround for now will be to actually open up the images captured to do the back-testing (will just take a little longer).

[idiot]

sometimes simpler is better maybe use getpixelcolor to just find the color of the text instead of whole image sometimes oldskool is better for somethings

[mafiamoe]

Have FIP working for the project i am working on, but finding the pixel color of an image would still be a lot faster than the current system.