Screen OCR To Retrieve Otherwise Undetectable Text

[Marcus Tettmar]

Thanks to gpulawski for posting this tip.

If you have Microsoft Office 2003 or Microsoft Office 2007 installed you have something called Microsoft Office Document Imaging, or MODI, which can perform OCR on Tiff and BMP files and is scriptable. Since Macro Scheduler can easily capture the screen, or a window, to a bitmap file, we can use MODI to perform OCR on the screen, or a window, to get its text. Here's how:

Code: Select all

VBStart
Function DoOCR(bitmapfile)
  Dim miDoc
  Dim miLayout
  Dim stringOut

  set miDoc=CreateObject("MODI.Document")
  miDoc.Create (bitmapfile)
  ' Perform OCR.
  'You can change the mousepointer here to an hourglass or something.
  miDoc.Images(0).OCR
  'Change the mouse back to normal default.
  set miLayout = miDoc.Images(0).Layout
  stringOut=miLayout.Text
  'MsgBox(stringOut)
  DoOCR = stringOut
  Set miLayout = Nothing
  Set miDoc = Nothing
End Function
VBEND

/*
//This example OCRs entire screen

//Capture Entire Screen to bitmap
GetScreenRes>x,y
ScreenCapture>0,0,x,y,%TEMP_DIR%\~scrn.bmp

//Use OCR to get text from screen
VBEval>DoOCR("%TEMP_DIR%\~scrn.bmp"),TheText

//Display text
MessageModal>TheText

*/

//This example OCRs just the active window
GetActiveWindow>title,X,Y,W,H
ScreenCapture>x,Y,{%X%+%W%},{%Y%+%H%},%TEMP_DIR%\~scrn.bmp

//Use OCR to get text from active window
VBEval>DoOCR("%TEMP_DIR%\~scrn.bmp"),TheText

//Display the text
MessageModal>TheText

This is really handy if you need to get text from a window which is not otherwise detectable - not exposed by the application as real text objects.

Edit: If running Office 2007 MODI is not installed by default. Run your office setup and select from the installation options to install the MODI components.

[CyberCitizen]

Hi Marcus,

I recently ran into an issue where I needed to pull some data from FrontRange's Heat. I was after the Incident Number of the ticket. I was generating an email to the user using my details, however all the gettext type commands didn't work. I ended up using the mouse selecting the field and copying all the text manually via mousemoves & clicks. Can't even use Ctrl + A to Select All in a field, instead that creates a new assignment. Basically I am doing a mouse click to the far right of the field and then sending a mouse click to the far left, I am sure I could use the shift + home, but this is working ok.

The issue I had was getting the incident number. It doesn't appear in the Windows Title, it appears in 2x sections, all the text capture commands failed for this, so I went with OCR.

The issue I then faced was that the MODI isn't included in Office 2010. There were a few ways to still get it though, eg using the Office 2007 install files & custom install or Sharepoint Designer. When I tried these options the posted script didn't work. But I wanted a more robust option instead of having to install MODI on these machines.

What I found was a portable open sourced tool which so far has been working wonders.

It is called Tesseract OCR (http://code.google.com/p/tesseract-ocr/).
They have a portable version of this software which allows me to include it with the script so I don't have issues with installing MODI and because its portable it just runs.

I did have an issue with the OCR clarity and it would find some numbers and letters incorrect.

I thought about another little tool I had called Photo Resize (http://www.rw-designer.com/picture-resize).

By using this file I could increase the size of the screen capture and increase its sharpness all via command line, thus helping the OCR detection.

By renaming PhotoResize from its default name to PhotoResizeP200NIOL.exe when I call the exe it increases the dimensions of the screen capture by 200%, sharpens the image, overwrites the original, keeps compression to low and skips the waiting for enter.

There is a good little configuration tool on the webpage which goes though the options.

Long story short here is the little part of the screen capture script.

Code: Select all

SetFocus>Call Logging*
DeleteFile>%TEMP_DIR%screen.txt
DeleteFile>%TEMP_DIR%screen.jpg
ScreenCapture>2351,99,2440,122,%TEMP_DIR%screen.jpg
Let>RP_WAIT=1
Let>RP_WINDOWMODE=0
Let>RP_DISPLAYERROR=0
RunProgram>M:\Portables\Tesseract-OCR\PhotoResizeP200NIOL.exe "%TEMP_DIR%screen.jpg"
Wait>0.1
RunProgram>M:\Portables\Tesseract-OCR\tesseract.exe %TEMP_DIR%screen.jpg %TEMP_DIR%screen
ReadFile>%TEMP_DIR%screen.txt,screen
MessageModal>%screen%

[Me_again]

Good you found a work around I can't imagine why removing MODI in Office 2010 was a good idea One Note does have OCR and AFAIK it's included in all versions of 2010 and can be scripted (VBA at least) so maybe that could be an alternative solution.

[CyberCitizen]

Good you found a work around I can't imagine why removing MODI in Office 2010 was a good idea One Note does have OCR and AFAIK it's included in all versions of 2010 and can be scripted (VBA at least) so maybe that could be an alternative solution.

Thanks for that Me_again,

I have VBA so I tend to avoid that unless really required. But this is working ok for me. Now that I am increasing the image size and sharping it, it has so far worked 100% for me. I am quite happy with that. Thought I would post it hear in case anyone else had issues with OCR on Win7 with Office 2010.

I did try that TopOCR however it doesn't work very well with newer created jpgs. Plus that abandoned the free version anyway, was hard enough to find a copy of it.

[mightycpa]

Thanks for the Tesseract heads up. I have a very similar situation, an application being run in Internet Explorer where the text of many fields is simply not selectable, and the size of the text prevented Tesseract from translating very well. I tried the approach where I did a quick screen capture then resize it, but I was still getting character recognition errors.

Enter the Ctrl-plus command. This command makes everything bigger in the window almost instantly. The resolution is excellent, far better than the resize was able to achieve, and Tesseract OCR's this large text 100% accurately. I do a Ctrl-minus when done.

I had one additional problem. When I selected the item, the colors were reversed from black text on white field to white text on black field. Tesseract could not recognize this. I found another free utility, Negative Screen
(http://arcanesanctum.net/negativescreen/). It has a few settings, including a simple reverse color. Using these two free utilities, I was able to overcome all of these obstacles, and the time spent doing it is neglible, especially if the alternative is to do it manually.

[CyberCitizen]

Thanks for providing the additional feed back.