November 25, 2008

Easy Text Extraction and Conversion

Filed under: Announcements,Scripting — Marcus Tettmar @ 11:58 am

If you’ve ever had to write code to read through large text files looking for patterns of text and extracting data you’ll know how fiddly it can be, even with Macro Scheduler‘s simple file reading and text parsing functions.

Well, there’s an amazing tool called TextPipe from DataMystic which makes extracting and transforming text data much easier. TextPipe can read huge files really quickly and allows you to create filters to process and extract the data without having to write any code. And the bonus is that TextPipe plays well with Macro Scheduler too, via its command line interface or ActiveX COM interface which you can call via VBScript.

I think TextPipe is a really nice add-on to Macro Scheduler so we’ve cut a deal with DataMystic to give Macro Scheduler customers a 25% discount off TextPipe Lite, TextPipe Std or TextPipe Pro. Registered users can log into the secure download area to grab a discount coupon which you can use at their site.

Once you’ve created a filter in TextPipe you can easily run it from Macro Scheduler via the command line, or from some VBScript code. And if you want to get really clever you can create filters on the fly with VBScript.

Here’s a simple macro which grabs some log files from a server, then runs a TextPipe filter and does something with the output:

VBSTART
Sub RunTextPipeFilter(filter_file)
  Dim TextPipeApp, Filter

  Set TextPipeApp = CreateObject("TextPipe.Application")
  Set Filter = TextPipeApp.newWindow
  Filter.openFilter( filter_file )
  Filter.execute()
  Set Filter = Nothing
  Set TextPipeApp = Nothing
End Sub
VBEND

//download the server logs
FTPGetFile>server,admin,7Gh63^,21,c:\filters,\logs\,*.log,A

//run the filter
VBRun>RunTextPipeFilter,c:\filters\extract_login_errors.fil

//transfer the output file
CopyFile>c:\filter\login_errors.txt,\\pluto\reports\login_errors.txt

For more on what you can do with TextPipe’s ActiveX interface see the VBScript Examples at DataMystic.com which can easily be used inside Macro Scheduler.