Forty times faster!!!

General Macro Scheduler discussion

Moderators: JRL, Dorian (MJT support)

Post Reply
User avatar
JRL
Automation Wizard
Posts: 3529
Joined: Mon Jan 10, 2005 6:22 pm
Location: Iowa

Forty times faster!!!

Post by JRL » Thu May 22, 2008 9:34 pm

Hi All,

Macro Scheduler version 10.1.15 was released earlier this week. The number one item on the list of improvements states:

"- Optimized script variable handling to speed up script execution, especially when a script processes a large number of variables."

I would like to emphasize the importance of this nearly innocuous text blurb. I'm experiencing a 40X speed increase in a loop that concatenates variable values using version 10.1.15 compared to using 10.1.02. I have not stepped back in time to test the speed of processing variables in versions 7, 8 or 9 but I imagine the speed difference will be similar. Running the following test script using version 10.1.02 (using the script default 2500 loops) I get an average time of about 32.303 seconds to concatenate the 2500 variable values. Running this same script in version 10.1.15 averaged about 0.763 seconds to concatenate the same variable values. Dividing 32.303 by 0.763 would be 42.337 times faster under the new and improved version 10.1.15. (Yes this is all on the same computer....)

Different processes will probably experience different speed increases. For example, creating the variables to be processed was only 3 times faster. Even that is impressive. If someone took a car to the Bonneville Salt Flats and tripled the current land speed record, the world would take notice...

Here are my actual results:

Cycles --- Var_Create --- File_Create ----- Version -------- When
2500 -------- 0.531 -------- 0.75 -------------- 10.1.15 -------- 05/22/2008 07:11:43 PM
2500 -------- 0.5 ----------- 0.766 ------------ 10.1.15 -------- 05/22/2008 07:11:37 PM
2500 -------- 0.531 -------- 0.766 ------------ 10.1.15 -------- 05/22/2008 07:11:29 PM
2500 -------- 0.516 -------- 0.766 ------------ 10.1.15 -------- 05/22/2008 07:11:21 PM
2500 -------- 0.5 ----------- 0.766 ------------ 10.1.15 -------- 05/22/2008 07:11:11 PM
2500 -------- 1.5 ----------- 32.25 ------------ 10.1.02 -------- 05/22/2008 06:58:17 PM
2500 -------- 1.531 -------- 32.484 ----------- 10.1.02 -------- 05/22/2008 06:57:32 PM
2500 -------- 1.5 ----------- 32.188 ----------- 10.1.02 -------- 05/22/2008 06:56:39 PM
2500 -------- 1.5 ----------- 32.25 ------------- 10.1.02 -------- 05/22/2008 06:55:53 PM
2500 -------- 1.547 -------- 32.344 ------------ 10.1.02 -------- 05/22/2008 06:55:11 PM

Here is the script. The script creates the chart above. Test it yourself. Run the script using an earlier version of Macro Scheduler, then run the script using version 10.1.15. Be amazed.

Code: Select all

///////Variable initialization section
//////////////////////////////////////////
Let>msg_width=600
Let>kk=0
Let>kkk=0
Let>data=
Let>file=%temp_dir%\ConsecutiveNumbers.txt
IfFileExists>file
  DeleteFile>file
EndIf
Let>Log=%temp_dir%\SpeedTestLog.txt
Let>LogBak=%temp_dir%\SpeedTestLog.bak
IfFileExists>Log
  ReadFile>Log,LogData
  Let>CF_OVERWRITE=1
  CopyFile>Log,LogBak
  DeleteFile>Log
  Separate>LogData,When,LDVar
  Let>LogEntry=%LDVar_1%When%CRLF%
  Let>LogData=LDVar_2
Else
  IfFileExists>LogBak
    CopyFile>LogBak,Log
	MDL>Log File does not exist. Reading from Log File Backup.
	ReadFile>Log,LogData
    DeleteFile>Log
    Separate>LogData,When,LDVar
    Let>LogEntry=%LDVar_1%When%CRLF%
    Let>LogData=LDVar_2
  Else
  Let>LogEntry=Cycles%TAB%%TAB%Var_Create%TAB%File_Create%TAB%Version%TAB%%TAB%When%CRLF%
    //WriteLn>Log,wres,LogEntry
    Let>LogData=
  EndIf
EndIf
///////End variable initialization section
//////////////////////////////////////////


///////Cycles input and validation section
//////////////////////////////////////////
Let>INPUT_BROWSE=0
Goto>Start
Label>ReTry
MDL>You typed "%qty%". Input MUST be numeric.

Label>Start
input>qty,number of loops,2500
If>qty=,EOF

VBSTART
'IsNumber Contributed by Marcus Tettmar Jan 19, 2007
Function IsNumber(var)
  IsNumber = (LCase(var) = UCase(var)) and isNumeric(var)
End Function
Function IsInteger(var)
  IsInteger = IsNumber(var) and (InStr(var,".") = 0)
End Function
VBEND
VBEval>IsNumber("%qty%"),IsNum
If>%IsNum%=False,ReTry
///End Cycles input and validation section
//////////////////////////////////////////

///////Test loop section
//////////////////////////////////////////
VBEval>timer,starttime
Repeat>kk
Add>kk,1
Let>var_%kk%=%kk%
Until>kk=qty

VBEval>timer,endtime
Let>Totaltime=%endtime%-%StartTime%
Let>TotalTime={(round(%TotalTime%*1000))/1000}
Message>Creating list of variables took %TotalTime% seconds
ConCat>LogEntry,%QTY%%TAB%%TAB%%TotalTime%%TAB%%TAB%

VBEval>timer,starttime
Repeat>kkk
Add>kkk,1
Let>value=var_%kkk%
ConCat>data,%value%%CRLF%
Until>kkk=qty
WriteLn>file,wres,data
VBEval>timer,endtime
Let>Totaltime=%endtime%-%StartTime%
Let>TotalTime={(round(%TotalTime%*1000))/1000}
GetDate>today
GetTime>now
ConCat>LogEntry,%TotalTime%%TAB%%TAB%%msched_ver%%TAB%%TAB%%today% %now%%LogData%
WriteLn>Log,wres,LogEntry
MessageModal>%LogEntry%
///////End test loop section
//////////////////////////////////////////

Label>EOF

Me_again
Automation Wizard
Posts: 1101
Joined: Fri Jan 07, 2005 5:55 pm
Location: Somewhere else on the planet

Post by Me_again » Thu May 22, 2008 9:57 pm

Uncompiled - right? What resources does the PC have?

I'm stuck in the dark ages of 9.1.02 and get:

Var_Create 10.047 File_Create 56.688

on a modest 2.66 GHz P4 with 768 megs.

User avatar
JRL
Automation Wizard
Posts: 3529
Joined: Mon Jan 10, 2005 6:22 pm
Location: Iowa

Post by JRL » Thu May 22, 2008 10:29 pm

Uncompiled - right?
Yes... and NOT in the editor, running in the editor slows it down a bit.
a modest 2.66 GHz P4 with 768 megs
I have a more modest 2.4 GHz with 512 Mb RAM... OS is WinXP SP1. (I turned off updates when some of my Access 97 abilities were hosed overnight.)

Me_again
Automation Wizard
Posts: 1101
Joined: Fri Jan 07, 2005 5:55 pm
Location: Somewhere else on the planet

Post by Me_again » Thu May 22, 2008 10:41 pm

JRL wrote:
Uncompiled - right?
Yes... and NOT in the editor, running in the editor slows it down a bit.
Whoa, you're not kidding! Run from Quick Launch Var_Create in 1.406 vs 10.047 in the editor. Not surprising I guess but something to remember.
I have a more modest 2.4 GHz with 512 Mb RAM... OS is WinXP SP1. (I turned off updates when some of my Access 97 abilities were hosed overnight.)
Pretty comparable, this one is also XP, SP2 though, no auto updates either :roll:

User avatar
Marcus Tettmar
Site Admin
Posts: 7395
Joined: Thu Sep 19, 2002 3:00 pm
Location: Dorset, UK
Contact:

Post by Marcus Tettmar » Fri May 23, 2008 7:58 am

When you are running in the editor you are running in DEBUG mode and the WATCH LIST has to keep track of all the variables. This adds a performance hit. Plus it now also shows the most recently modified variable at the top of the watch list. It therefore has to sort the variable list by modified time at each step. This will introduce a further delay. But running in DEBUG mode is designed for single stepping which by its very nature incurs a delay anyway. Any slight delay during debug cannot be avoided and does not matter.

Hopefully no one is running production scripts by running them from the editor!

Yet - note that with 10.1.15 it is still faster under the editor than it is when run normally in previous versions.

Yet another compelling reason to upgrade to version 10 :-)
Marcus Tettmar
http://mjtnet.com/blog/ | http://twitter.com/marcustettmar

Did you know we are now offering affordable monthly subscriptions for Macro Scheduler Standard?

JBurger
Junior Coder
Posts: 33
Joined: Wed Nov 12, 2003 7:16 pm
Location: NY

Post by JBurger » Wed Jun 04, 2008 2:14 pm

Should I recompile my scripts converted to exe files, will this version speed any of them up?

-Joe

User avatar
JRL
Automation Wizard
Posts: 3529
Joined: Mon Jan 10, 2005 6:22 pm
Location: Iowa

Post by JRL » Wed Jun 04, 2008 3:45 pm

Good Question. If the script processes a large number of variables, then recompiling could make a big difference. My experience has been that recompiling under version 10.1.15 is making everything faster, but not always significantly faster. For example, the scripts I have that process windows in my ERP software are still at the mercy of the response time of the ERP software.

You'll have to judge for yourself based on what level of effort is involved in recreating the executables compared to the potential benefit.

Post Reply
Sign up to our newsletter for free automation tips, tricks & discounts