July 2, 2007

Methods for Accessing Excel Data

Filed under: Automation, Scripting — Marcus Tettmar @ 11:11 am

UPDATE: 25/09/2012 – This article was written in 2007 before Macro Scheduler contained native Excel functions and native database access. Furthermore, since this article was written Excel no longer supports DDE. This article has therefore been superseded and should only be used as a reference for use with older versions of Macro Scheduler and/or older versions of Excel. For the latest version (currently v13) look at the native Excel functions (XLOpen, XLGetCell, XLSetCell etc) and the native database functions. For an example of querying Excel using the native database functions see this post.

There are a number of ways that Macro Scheduler can interact with Excel to retrieve or modify data. I’ll introduce you to three methods here.

1) Forget Excel!

Yes, sometimes you can forget about Excel. Often people actually just want to read from/write to a CSV (Comma Separated Values) file. Because Excel is on so many people’s computers and is the default file association for CSV files it seems many people think CSV files are Excel files. Not so. If the file you want to read is a .CSV file then it’s really just a text file containing the data. Each line represents a row of data and each column is separated by commas (or sometimes semicolons – but it could be anything else). Open the CSV file up into Notepad and you will see something like this:

"id","description","quantity","price"
5343,"earl gray tea",32,5.64
2435,"decaffeinated coffee",26,4.67
1433,"chocolate cookies",12,3.98

String values may or may not be delimited by quote characters, as in my example above.

We can forget about Excel if we want to read this data in with Macro Scheduler. We can instead just go direct and use the ReadFile or ReadLn functions which are used to access text files. This post shows you how to read a text file in line by line. The only part missing is to extract each field or column from each line (record). Well, as with the ReadFile loop in example 2 of this post, we can use the Separate command, and specify the comma character as the delimiter:

Let>comma=,
Separate>line,comma,fields

This would give us fields_1, fields_2, fields_3 and fields_4 containing each item in the line. Combining that code into my ReadFile loop example to show each field would give:

ReadFile>c:\\\temp\\\test.txt,TheFileData
Separate>TheFileData,CRLF,Lines
Let>comma=,
Let>k=1
Repeat>k
  Let>CurrentLine=Lines_%k%
  //extract each field
  Separate>CurrentLine,comma,fields
  MessageModal>%fields_1% %fields_2% %fields_3%
                       %fields_4%
  Let>k=k+1
Until>k>Lines_Count

Want to remove the quotes from the string fields? Use StringReplace:

StringReplace>fields_2,",,fields_2

To write data out to a CSV file simply use the WriteLn command:

Let>LineToWrite=1234,cornflakes,1,1.99
WriteLn>filename.csv,result,LineToWrite

Update: Here’s another way to read CSV files which also solves the problem of quoted field values containing commas.

2) Use DDE

EDIT: DDE is no longer supported in the latest versions of Excel. See editor’s note at top.

DDE is an abbreviation for Dynamic Data Exchange and is an old protocol for interfacing with applications. It has largely been superseded by other methods such as COM but Microsoft Excel still supports it and it provides a simple way to access cells in Excel directly.

Use the DDERequest command to retrieve an item of data from Excel. Excel must be running and the spreadsheet must already be open. We could easily make our macro open the worksheet for us using the ExecuteFile command (or Run Program). We then use DDERequest something like this:

DDERequest>Excel,c:\\\documents\\\example.xls,R1C1,
      myVariable,60

The first parameter is the Server name, which is always just Excel for Microsoft Excel. The second parameter, the DDE “Topic”, should be set to the filename of the spreadsheet (which must be open). The third parameter specifies the cell to retrieve. This is constructed RnCn (Row n, Column n). Use numeric values for both Row and Column. So Cell A1 would be R1C1. Cell B1 would be R1C2. Cell D14 would be R14C4, etc. The next parameter is the variable name to store the result in. And finally we specify a timeout value in seconds. If the command fails to locate within that amount of time it will timeout and the result variable would contain “DDE_TIMEOUT”.

The second parameter can also contain a sheet name when you want to access data on a particular worksheet. The Sheet name is added after the filename, separated from the filename by a colon. So parameter two would become: Filename:Sheetname

One thing to note is that Excel often appears to append a CRLF sequence to the end of the data. Therefore we should remove it as follows:

StringReplace>myVariable,CRLF,,myVariable

Take a look at the “Extract From Excel” sample script which comes with Macro Scheduler for an example which uses the DDE method to retrieve data from Excel and paste it into Notepad.

To modify a cell value, or write a value to Excel we use the DDEPoke command as follows:

DDEPoke>Excel,c:\\\documents\\\example.xls,R2C3,
                                      56.75

Again, the spreadsheet must be open for this to work. The above puts the value 56.75 into cell C2 (row 2, column 3). As with DDERequest you can add a worksheet name in parameter two if needed (Filename:SheetName).

For another example see:
Exchange Data with Microsoft Excel The Easy Way

3) VBScript via COM

VBScript offers the most flexible and powerful method of interacting with Microsoft products. Not only can we modify data using this method we can also control almost any part of Excel. Anything that you can program or record in Excel can be converted into VBScript code and used inside a Macro Scheduler script.

Here is an example script with some functions you can use to retrieve and modify Excel cells:

//Put this VBSTART..VBEND block at top of script
//to declare the functions once
VBSTART
Dim xlApp
Dim xlBook

'Opens the Excel file in Excel
Sub OpenExcelFile(filename)
  Set xlApp = CreateObject("Excel.Application")
  xlApp.visible = true
  Set xlBook = xlApp.Workbooks.open(filename)
end sub

'Use this to close Excel later
Sub CloseExcel
   xlApp.quit
   Set xlApp = Nothing
End Sub

'Retrieves a cell value from the specified
'worksheet
Function GetCell(Sheet,Row,Column)
  Dim xlSheet
  Set xlSheet = xlBook.Worksheets(Sheet)
  GetCell = xlSheet.Cells(Row, Column).Value
End Function

'Sets specified cell of specified worksheet
Function SetCell(Sheet,Row,Column,NewValue)
  Dim xlSheet
  Set xlSheet = xlBook.Worksheets(Sheet)
  xlSheet.Cells(Row,Column).Value = NewValue
End Function
VBEND

//Do the business
VBRun>OpenExcelFile,%SCRIPT_DIR%\example.xls
VBEval>GetCell("Sheet1",5,4),theValue
MessageModal>Cell value: %thevalue%
VBEval>SetCell("Sheet1",28,2,998),nul
//VBRun>CloseExcel

This code is constructed in such a way to allow you to call the VBScript functions from regular MacroScript code and therefore retrieve the data to regular variables.

For another example see:
Automate Excel with VBScript

You can also run Excel macros:
Run Excel Macros and VBA Routines

And, as stated above, you can do a whole heap of other things. Search the forums for Excel to find other examples. Converting from Excel VBA to Macro Scheduler VBScript is a topic for another day but hopefully you can get an idea from these simple examples.

___
You may also be interested in: Accessing Databases

Reading through text files in memory

Filed under: Scripting — Marcus Tettmar @ 10:20 am

Looping through a text file is commonly achieved with the ReadLn command in a loop, like the example in the help file:

Let>k=1
Label>start
ReadLn>c:\\\temp\\\test.txt,k,line
If>line=##EOF##,finish
MessageModal>line
Let>k=k+1
Goto>start
Label>finish

This uses a very simple loop and loops through the file one line at a time using the ReadLn command and a counter variable.

An alternative method is to read the entire file into memory and then put the lines into an array. This means seeking through the file becomes faster, and it becomes easier to change individual lines. Use the ReadFile command to read the entire file content into a variable:

ReadFile>c:\\\temp\\\test.txt,TheFileData

We now have a variable called TheFileData containing all the lines in the file. Each line will be separated by a Carriage Return, Line Feed combination (CRLF). So we can now split the file into an array using the Separate command, specifying CRLF as the delimiter:

Separate>TheFileData,CRLF,Lines

Lines_Count now tells us how many lines there are and we can access each line with Lines_1, Lines_2, etc. So here’s the new version of the full script:

ReadFile>c:\\\temp\\\test.txt,TheFileData
Separate>TheFileData,CRLF,Lines
Let>k=1
Repeat>k
  Let>CurrentLine=Lines_%k%
  MessageModal>CurrentLine
  Let>k=k+1
Until>k>Lines_Count

June 27, 2007

Office 2007 VBA Shell Bug?

Filed under: Automation, Scripting — Marcus Tettmar @ 3:12 pm

When trying to execute a third party application, such as Macro Scheduler, from VBA in Office 2007 using the Shell function the following error is returned:

“Invalid procedure call or argument”

Sometimes you might want to run a Macro Scheduler macro from Office using this method and get this error. Searching the web reveals many people reporting the same error when trying to run other applications, including other Office apps! E.g.:

Shell “c:\my macros\macroA.exe”

I suspect there’s an issue with spaces in the path but trying to embed extra quotes with chr(34) doesn’t seem to help. The Shell function also doesn’t provide a way to specify command line parameters other than including them in the entire command string.

The solution is to use the ShellExecute API function. Declare the function in the General Declarations section of your VBA module using this code:

Private Declare Function ShellExecute Lib “shell32.dll” Alias “ShellExecuteA” (ByVal hwnd As Long, ByVal lpOperation As String, ByVal lpFile As String, ByVal lpParameters As String, ByVal lpDirectory As String, ByVal nShowCmd As Long) As Long

Then you can use the function as follows:

ShellExecute 0&, “”, “c:\program files\macro scheduler\msched.exe”, “calculator”, “”, 1

This code runs the Calculator sample macro in Macro Scheduler.

To run an application with no command line parameters leave the fourth parameter empty:

ShellExecute 0&, “”, “c:\my macros\mymacro.exe”, “”, “”, 1

May 18, 2007

First Winner of Scripts & Tips Competition

Filed under: Announcements, Automation, Scripting — Marcus Tettmar @ 1:19 pm

We have our first successful entry to the Scripts & Tips 10 Year Anniversary Competition. Robert White sent in this excellent Cookie Monitor script. It’s a nicely written script which is both very useful and also demonstrates effective use of a non-modal dialog and other scripting features such as looping, reading a list of files, string manipulation, and reading from the registry. The beauty of this script, as a demonstration script, is you can run it as-is and it will work on your machine without any modification. Grab the script here.

We’ve rewarded Robert with an extra year of upgrade protection. Well done Robert, and thanks for the script!

May 15, 2007

Post to Scripts & Tips and Win Free Upgrades

Filed under: Announcements, General — Marcus Tettmar @ 8:12 am

We’re running a competition where you can win your next upgrade free if a script you submit is chosen for inclusion in the Scripts & Tips archive. We’re looking for useful scripts and examples which can be added to the Scripts & Tips archive and will benefit others. Any scripts that are accepted for inclusion will be rewarded with a free year of upgrade protection. Find out more here and get scripting!

May 3, 2007

Celebrations All Round

Filed under: General — Marcus Tettmar @ 7:53 am

So a day after celebrating our 10th anniversary I’m pleased to also congratulate Nagarsoft on their 1st year in business. Nagarsoft have achieved much in their first year, with the launch of their superbly crafted Direct Access:

Direct Access speeds up your personal computing experience by reducing time consuming, repetitive tasks through the use of abbreviations. Type an abbreviation in any application and Direct Access will detect it and perform the task you desire.

And by the way – Macro Scheduler customers can get 25% off Direct Access. You’ll see details when you log in to the registered download area.

Here’s wishing Andrea at Nagarsoft many more years in business. I look forward to being able to celebrate his company’s 10 year anniversary with our 20th!

May 1, 2007

Macro Scheduler 10 Year Anniversary Offer on Bits du Jour!

Filed under: Announcements, General — Marcus Tettmar @ 12:01 am

Today, 1st May 2007, Macro Scheduler is on sale at Bits du Jour.

Ten years ago on 1st May 1997 we sold our very first copy of Macro Scheduler. Back then there was only one edition of Macro Scheduler and it sold for just $40. Today, for one day only, you can buy the latest version of Macro Scheduler Standard for the same price! That’s a huge discount of 58%! And the discount is valid for Professional and Enterprise versions too.

58% Off Macro Scheduler Today Only

Get your discounted copy now – use coupon code BDJ10YR.

The mathematically astute amongst you may have noticed that the discount is actually 58.76%. That is the percentage needed to get the price of Macro Scheduler Standard down to an even $40 – the price in May 1997!

Thanks to Andrea Nagar of Nagarsoft for the banner!

April 30, 2007

Macro Scheduler 9.1 Released

Filed under: Announcements — Marcus Tettmar @ 8:41 am

Coinciding nicely with our 10 year anniversary, Macro Scheduler 9.1 was released today. I guess version 10 would be a more apt label, but this is not a major release – more a maintenance update with a few bonus extras thrown in:

Fixes:

  • Possibly fixed random “Cannot open Clipboard” errors
  • Fixed an issue with dialog designer toolbar button in editor sometimes getting invalid content
  • Select All (CTRL+A) now works in dialog memo fields
  • Fixed CTRL-A (Select All) in Code Snippets editor
  • Fixed WaitWindowOpen/Closed to work with handles as per docs (WIN_USEHANDLE)
  • New stuff:

  • Ability to include bitmap data for dialog images and buttons in scripts
  • Script now creates object handle variables: DialogName.Handle, DialogName.ObjectName.Handle after Dialog block is executed
  • Added DIALOG_CHANGE OnEvent handler:
    OnEvent>DIALOG_CHANGE,DialogName,ObjectName,Subroutine
  • Dialog Designer toolbar button in editor will now edit dialog if cursor on first line of a dialog block
  • Improved RetrievePOP3 to decode attachments and output body text parts.
  • Deleting macros will now delete to recycle bin
  • Being able to include dialog bitmap data inside scripts instead of referring to external image files will make it easier to distribute compiled dialog macros (yes, I know how much one or two of you have wanted this!). The new DIALOG_CHANGE event will also make life easier when writing scripts that need to respond to the user making selection changes on dialogs. And having the object handles exposed as soon as the dialog is created will reduce code complexity for working with API functions that manipulate the objects directly. So I think those of you who use Macro Scheduler to build distributable solutions with their own user interfaces will appreciate this release.

    Trial Download | Registered Downloads

    April 2, 2007

    Macro Scheduler Saves the World 27,000 Years!

    Filed under: General — Marcus Tettmar @ 4:53 pm

    In case you didn’t already know it, Macro Scheduler will be 10 years old this year. In fact, I had a quick look through our old sales records the other day and discovered that we sold the first ever copy of Macro Scheduler on 1st May 1997. So 1st May this year will be a special day. We’ve got a few things up our sleeves so keep posted for more news.

    In the mean time Andy Brice, of PerfectTablePlan.com, suggested it might be fun to work out how much time Macro Scheduler has saved people in total since its launch in 1997. So we pulled some figures from our sales database, made a few conservative estimates, consulted our marketing people and ran a few number crunching macros and discovered that in total Macro Scheduler has saved approximately 27,000 years in lost productivity over the last 10 years.

    In that time you could have built:

    • 6875 Golden Gate Bridges, or
    • 3928 Channel Tunnels, or
    • 5500 Wembley Stadiums, or
    • 1350 Great Pyramids of Giza.

    [Although to be fair Herodotus estimated you’d need 100,000 workers to build the Giza Pyramids in 20 years]

    Seriously. Sure, we had to make a few estimates but we’ve been conservative. This is based on our most complete year on year user-base figures and an assumption that on average each user saves 2 hours per working day. We’ve assumed a working day is 8 hours and a working year is 210 days. While some people may use the software on a more ad-hoc basis we know many run dedicated Macro Scheduler workstations running 24 hours a day. Anyway, we need something to work from, so we’ve chosen an average figure of 2 hours per day. But even if you halve that it’s still a pretty impressive outcome. And it could be more – that’s only working days and our user base figures most likely under-read due to various custom licensing arrangements and unlimited user licenses etc.

    Hey, it’s just a bit of fun 🙂

    March 14, 2007

    Hijacking Spammer

    Filed under: General — Marcus Tettmar @ 8:00 am

    Some dirty little spammer appears to have used valid mjtnet.com email addresses for the reply-to address of a recent spamming campaign. We’re getting dozens of mail failures for emails we didn’t send out, many of which are advertising a certain men’s pharmaceutical product. Nothing to do with us.

    What can be done? Not a lot. We already have SPF records set up for our domain. SPF defines which mail servers are allowed to send mail for a specific domain. Unfortunately the vast majority of mail servers don’t bother to do SPF checks. Any that do will be rejecting those spam messages as they were sent from a different mail server to the one our domain permits to be used. But most won’t.

    So I guess all we can do is wait for this to die down, filter out the mail failures and hope that this recent hijacking doesn’t get us unfairly blacklisted!