Is Time-Based Image Recognition Possible?

[rjw524]

Hi,

I am running a program that examines a set of data and constantly generates two results: "Within Limit" and "Outside Limit"

Both messages appear constantly throughout the test. Sometimes the message appears so quickly that it's hard to catch.

I DON'T need MS13 to catch the quick instances of "Outside Limit".

What I need it to do is catch the instances where "Outside Limit" is on the page for a solid 1 to 2 seconds or more.

I was wondering if Macro Scheduler can actually do this?

Can I create a macro that watches a screen, recognizes an image and THEN waits to see if that image stays on the page for a set amount of time before triggering an action?

Thanks!

rj

[PepsiHog]

Hey rjw524,

This is a VBS timer. When OutsideLimit call timer. When InsideLimit again make note of time and write a report. Or visa-versa.

Code: Select all

let>InsideLimit=0

// When Outside Limit
GoSub>Timer

// Keep checking for Inside Limit here.

// Set InsideLimit to 1 when true.

// When Inside Limit
if>InsideLimit=1
       VBEval>Timer-%inicio%,elapsed
       mdl>%elapsed%
       // Write a report or something.
endif

srt>Timer
VBSTART
VBEND
VBEval>Timer,inicio
let>InsideLimit=1
END>Timer

Hope this helps you,
PepsiHog

[rjw524]

This is great Pepsi.

I have one question.

I'm still quite inexperienced with this stuff unfortunately.

How would I integrate the image recognition with the code you provided above?

Thanks!

rj

[PepsiHog]

@ rjw524,

Please post the code you are using so I can help further. Is the "program" you have running the test written in Macro Scheduler?

I'm confused. It sounds like you already are running a macro that generates "WithinLimit" and "OutsideLimit". Is "InsideLimit" and "OutsideLimit" based upon the images?

I need what you have so far, so I am not making assumptions as to what your program is doing.

PepsiHog

[rjw524]

@ rjw524,

Please post the code you are using so I can help further. Is the "program" you have running the test written in Macro Scheduler?

I'm confused. It sounds like you already are running a macro that generates "WithinLimit" and "OutsideLimit". Is "InsideLimit" and "OutsideLimit" based upon the images?

I need what you have so far, so I am not making assumptions as to what your program is doing.

PepsiHog

Hi Pepsihog,

I actually haven't written the code for this macro yet. I was moreso inquiring if MS13 has the functionality to even do this.

It's actually for my job. At work we use a web based application for monthly audit purposes. The application is available only through my company's intranet. It reviews our data line by line and gives the "within limit" or "outside limit" reading in the lower right hand corner. The only common example of what it looks like is in Microsoft Excel. Excel will keep tallying the sum of a set numbers in the lower right hand of the screen if you keep dragging the mouse over a cells.

With this work application, this image is the only way I have to actually track the changes.

So, there's no code yet.

[PepsiHog]

@ rjw524,

Yes. It can do everything you have described. It can recognize an image. It can capture text from the screen. And it can alert you of any length of time passing you specify. And a lot more.

Text capturing sometimes can be tricky. But from what you describe it would be a straight forward process. Not very difficult at all.

I have done many things that involve every aspect of what you describe. And MS is extremely easy to use.

Good Luck,
PepsiHog

[rjw524]

@ rjw524,

Yes. It can do everything you have described. It can recognize an image. It can capture text from the screen. And it can alert you of any length of time passing you specify. And a lot more.

Text capturing sometimes can be tricky. But from what you describe it would be a straight forward process. Not very difficult at all.

I have done many things that involve every aspect of what you describe. And MS is extremely easy to use.

Good Luck,
PepsiHog

Hi PepsiHog,

Thanks for the reply and the insight on the capabilities within MS.

I'm going to give the code for this a try and post it for you to take a look at.

It doesn't have to be a text capture (I've tried that myself and it is a bit tricky). The within limit and outside limit both have a unique color to them that I can use. The main thing, of course, is being able to distinguish a quick instance from a longer one (1.0 seconds or longer).

Thanks for all of your help!

RJ

[armsys]

rjw524,

I actually haven't written the code for this macro yet. I was moreso inquiring if MS13 has the functionality to even do this.
It's actually for my job. At work we use a web based application for monthly audit purposes. The application is available only through my company's intranet. It reviews our data line by line and gives the "within limit" or "outside limit" reading in the lower right hand corner. The only common example of what it looks like is in Microsoft Excel. Excel will keep tallying the sum of a set numbers in the lower right hand of the screen if you keep dragging the mouse over a cells.

You didn't exactly describe your app requirement. For example, you didn't mention whether you wish to extract images and/or texts. In any case, yes, Macro Scheduler can by and large handle your requirements so far vaguely narrated by you.

[rjw524]

Hi armsys,

Thanks for the reply.

And thanks regarding the lack of clarity on my part.

The most important item was the ability of MS-13 to use duration as a requirement for an action. Afterwards, the macro steps required are quite simple. A copy/paste here, or a mouse click there, etc.

The red-colored "outside limit" message just turns up so frequently when this in-house app runs, that I wouldn't want the macro to perform the mouse move or click every time. I know in my limited (but growing) experience that MS13 can wait until an image shows up or etc, but to be able to recognize when that image is up for say .02 seconds, versus 2 seconds for instance, was the main query.

[armsys]

The red-colored "outside limit" message just turns up so frequently when this in-house app runs, that I wouldn't want the macro to perform the mouse move or click every time. I know in my limited (but growing) experience that MS13 can wait until an image shows up or etc, but to be able to recognize when that image is up for say .02 seconds, versus 2 seconds for instance, was the main query.

Hope the following script helps:

Code: Select all

Let>WSI_Timeout=5
WSI>Outside_Limit.bmp
If>WSI_Timedout=TRUE
  MDL>RED ALERT: Outside_Limit NOT FOUND. Please report to rjw524.
Endif

[JRL]

Here's another sample. Rather than looking for an image I thought it might be more reliable to look for a specific pixel color at a specific screen location.

Code: Select all

//Pick an X and Y position where the red "outside limit" message appears
Let>CheckFor_Xpos=1000
Let>CheckFor_Ypos=800

//Assign the color number appropriate for the red "outside limit" message
//at the previously assigned X and Y locations.
Let>CheckFor_PixelColor=255

//The goal is to do something when the message has displayed for
//a specified amount of time.  Set that time here in seconds.
Let>OSLTimeout=2

//timeout is a result and a flag.  Should be preset to zero.
Let>TimeOut=0


Label>WaitForRedLoop
  GoSub>CheckForRed
  Wait>0.01
  //If the time the message displayed exceeded the predefined timeout
  // then run the "Process" subroutine.
  If>%TimeOut%>%OSLTimeout%
    GoSub>Process
  EndIf
Goto>WaitForRedLoop

SRT>CheckForRed
  //Timer function created in version 13
  Timer>StartTime
  Label>CheckTheLocation
  //Check the pixel color at the predefined location.
  GetPixelColor>CheckFor_Xpos,CheckFor_Ypos,ColorResult
  //If the checked color matches the predefined color,  compare the 
  //time with the "StartTime"
  If>%ColorResult%=%CheckFor_PixelColor%
    Wait>0.01
    Timer>IntermediateTime
    Let>TimeOut={(%IntermediateTime%-%StartTime%)/1000}
//UnRemark these next lines if you want to wait a specific amount of time
//Leave them remarked if the goal is to perform an action after the message
//is gone but the message has been displayed at least "OSLTimeout" seconds.
//I would recommend leaving this remarked. It will require more code to actually 
//make this work correctly.
/*
    If>%TimeOut%>%OSLTimeout%
      //Timeout has been reached.  Script should next run the "Process" subroutine
      //even though the message might still be displayed.
      Goto>EndOfCheckForRed
    EndIf
*/
  Else
    //The location is not red.  If it was not red the last cycle, we go back to
    //waiting for the location to become red.  If it was red the last cycle
    //then the message has displayed and gone.  Next we need to see
    //if it displayed longer than the predefined time limit "OSLTimeout"
    Goto>EndOfCheckForRed
  EndIf
  
  Goto>CheckTheLocation
  Label>EndOfCheckForRed
END>CheckForRed

SRT>Process
  //Do whatever needs to be done
  //For example write the time to a file
  WriteLn>%temp_dir%CheckForRed.txt,wres,TimeOut
  Wait>0.01
  //timeout is a result and a flag.  Should be reset to zero.
  Let>TimeOut=0
END>Process

[armsys]

Here's another sample. Rather than looking for an image I thought it might be more reliable to look for a specific pixel color at a specific screen location.

Hi JRL,
Thanks for your much better instructive script.
Why is CheckFor_PixelColor initiated with 255?
Does 255 means Red color in Macro Scheduler?

You script would may assume the screen is of the same zoom scale and color.

[JRL]

Does 255 means Red color in Macro Scheduler?

Yes 255 is red. An assumption on my part and also assuming that rjw524 can figure out how to determine an exact color match at a specific pixel location on the screen and also assuming that the "red" "outside limit" message appears in the exact same location every time it pops up. All reasonable assumptions I believe.

You script would may assume the screen is of the same zoom scale and color.

Yes. I assume he will be running this on the same computer day after day and if it has to be run on a second or third etc. computer, the values for location and color may need to be adjusted... or may work perfectly well if the computers are similar. I don't think of scripts used in a single workplace as needing to be excruciatingly flexible. They do need to be excruciatingly reliable.

[armsys]

JRL,
Appreciate your excruciatingly meticulous illumination.
I find your emphasis of being "excruciatingly reliable" very inspiring.
Thanks again.

[armsys]

Dear Samper,
You voted me with -1.2 negative reputation points here for this thread-Is Time-Based Image Recognition Possible? on Tue Jun 11, 2013 9:30 pm HK Time.
You didn't explain the cause of your negative vote.
Neither did you ever participate in the thread conversation.
The factual and the sole cause of my seemingly low reputation vote was my total donation of all my 90 reputation points to Marcus on Mon Feb 14, 2011 11:35 pm HK Time.
Samper, would you please explain your negative vote so that I can improve my participation here henceforth?
Thanks for your helpful comment in advance.