Marcus' Macro Scheduler Blog

This morning, forum user hip asked how a script could sort a CSV file. Seems CSV files, sorting and Excel has become a bit of a hot topic here lately. Since Excel can read and save to CSV and also has a Sort function we might as well make use of it. Here’s a little VBScript function I put together which will sort a CSV file by a given column:

VBSTART
Sub SortCSVFile(file,sort_column)
  Set XLApp = CreateObject("Excel.Application")
  XLApp.Workbooks.Open(file)
  
  XLApp.ActiveWorkbook.ActiveSheet.Sort.SortFields.Clear
  Set SortCol = XLApp.Range(sort_column)
  XLApp.ActiveWorkbook.ActiveSheet.Sort.SortFields.Add(SortCol)
  Set Cell = XLApp.ActiveWorkbook.ActiveSheet.Cells.Find("*", XLApp.ActiveWorkbook.ActiveSheet.Cells(1, 1),,,,2)
  Set mYRange = XLApp.Range("A1", XLApp.ActiveWorkBook.ActiveSheet.Cells(Cell.Row, Cell.Column))
    
  XLApp.ActiveWorkbook.ActiveSheet.Sort.SetRange(mYRange)
  XLApp.ActiveWorkbook.ActiveSheet.Sort.Apply
  
  XLApp.ActiveWorkBook.Save
  XLApp.ActiveWorkBook.Close false
  XLApp.quit
End Sub
VBEND

The function opens the CSV file in Excel – without making Excel visible – then performs the sort, saves the file and closes Excel. You won’t see Excel appear on the screen.

Here’s how to use it. If you want to sort your file on column B and don’t have a header row, use:

VBRun>SortCSVFile,C:\Documents\my.csv,B1

To sort on column A and you DO have a header row use:

VBRun>SortCSVFile,C:\Documents\my.csv,A2

UPDATE: Here’s a simplified version which should work in Excel XP/2003 and should be able to detect whether or not your data has a header row:

VBSTART
Sub SortCSVFile(file,sort_column)
  Set XLApp = CreateObject("Excel.Application")
  XLApp.Workbooks.Open(file) 
  XLApp.ActiveWorkbook.ActiveSheet.Range("A1").Sort XLApp.ActiveWorkbook.ActiveSheet.Range(sort_column),,,,,,,0  
  XLApp.ActiveWorkBook.Save
  XLApp.ActiveWorkBook.Close false
  XLApp.quit
End Sub
VBEND

What if you don’t have Microsoft Excel?

As mentioned in my reply on the forum another method you could use is to open the CSV file as a recordset using DBQuery and apply an “ORDER BY” clause to your SQL statement. That would sort it into memory. Then you would need to loop through the array and write the data back out to a CSV file using WriteLn.

Something like:

//new file to write to
Let>new_file=%SCRIPT_DIR%\example_sorted.csv
IfFileExists>new_file
  DeleteFile>new_file
Endif

//set WriteLn NOT to add line breaks (we'll add those ourselves)
Let>WLN_NOCRLF=1
//The CSV delimiter
Let>comma=,

//Connect to and query the CSV using ORDER BY PRICE to sort on Price column
Let>ConStr=Provider=Microsoft.Jet.OLEDB.4.0;Data Source=%SCRIPT_DIR%;Extended Properties='text;HDR=YES;FMT=Delimited'
DBConnect>ConStr,dbH
DBQuery>dbH,select * from example.csv ORDER BY PRICE,fields,num_recs,num_fields

//If we have records, loop through records and fields, writing out to new file
If>num_recs>0
  //first need to copy the header record ...
  ReadLn>%SCRIPT_DIR%\example.csv,1,header_line
  WriteLn>new_file,wlnres,%header_line%%CRLF%

  //the loop
  Let>rec=0
  Repeat>rec
    Let>rec=rec+1
    Let>field=0
    Repeat>field
      Let>field=field+1
      Let>this_field=fields_%rec%_%field%
      //write the field out to the new file
      WriteLn>new_file,wlnres,this_field
      //add the comma delimiter if needed
      If>field<num_fields
        WriteLn>new_file,wlnres,comma
      Endif
    Until>field=num_fields
    //add CRLF at end of line
    WriteLn>new_file,wlnres,%CRLF%
  Until>rec=num_recs
Endif
DBClose>dbH

//if you want, delete the input file and replace with the new sorted one:
//DeleteFile>%SCRIPT_DIR%\example.csv
//RenameFile>%SCRIPT_DIR%\example_sorted.csv,%SCRIPT_DIR%\example.csv

Earlier this month I wrote about how you can read and write data from/to CSV files as if they were database tables using DBQuery. Andrew asked if I knew of a way that XML files could be treated as database tables in the same way. I did find a couple of references to some ODBC drivers online but one was not yet available and the other was part of a larger commercial package so I was unable to try them.

Since then it has occurred to me that there is a way we can convert XML files to CSV (or XLS) files using Excel. Once the data is in CSV file format we can then retrieve it easily using DBQuery.

The more technically aware will probably ask why we can’t just use the MSXML.DomDocument object in VBScript to read in the XML and parse it. Well, while that is of course possible, many will find the ability to access the XML data in a tabular database-like format easier and more familiar.

So, without further ado, here’s my ConvertXMLtoCSV function:

VBSTART
Sub ConvertXMLtoCSV(xmlFile,csvFile)
    Set xlApp = CreateObject("Excel.Application")
    Set xlBook = xlApp.Workbooks.OpenXML(xmlFile,2)
    xlBook.SaveAs csvFile, 6
    xlBook.close false
    xlApp.quit
End Sub
VBEND

You will need Excel installed for this to work. Use the function like this:

VBRun>ConvertXMLtoCSV,c:\docs\my.xml,c:\docs\converted.csv

And now you can quickly read the data into an array using DBQuery.

The function works quietly in the background. You won’t see Excel open – just a short delay while the CSV file is created.

To convert to an Excel file rather than a CSV file change the code to:

VBSTART
Sub ConvertXMLtoXLS(xmlFile,xlsFile)
    Set xlApp = CreateObject("Excel.Application")
    Set xlBook = xlApp.Workbooks.OpenXML(xmlFile,2)
    xlBook.SaveAs xlsFile, 56
    xlBook.close false
    xlApp.quit
End Sub
VBEND

Please note I’m using Excel 2007 and haven’t tried this with earlier versions though I believe Excel 2003 can also read from XML files. Success may also depend on the format of your XML.

See:
Reading From CSV Files
More on Reading and Writing CSV Files
Retrieve Entire Excel Sheet Using DBQuery