virtual memory error - any ideas?

howl · Post by **howl** » Mon Aug 13, 2007 1:39 pm

I've created a macro with an intentional continuous loop that reads the same 5 web pages over and over all day long. I eventually get a virtual memory error from windows. Nothing else is running on the PC. Of course I realize that it's probably not a good idea to have an intentional continuous loop. I am experimenting with different alternatives, but I was hoping that someone might have some insight into where I should start. For example, is IE the virtual memory hog, is it the macro, macro scheduler?

By the way, I love the product. Even with my flawed design, we're able to do some web monitoring that would not have been feasible without this tool. I'm looking forward to using it for many other automation tasks.

Thanks!

Post by **Marcus Tettmar** » Mon Aug 13, 2007 1:47 pm

No way to answer without seeing your script. Does your script create IE instances? If you create IE instances, do you free them? Perhaps you've created thousands of IE instances without closing them afterwards for example ... any objects you create will need freeing - or reusing, rather than recreating - or you will be using up resources.

howl · Post by **howl** » Mon Aug 13, 2007 2:09 pm

Here's the code. I'm only creating one IE instance. Thanks in advance.

VBSTART
Sub CheckAndInsert(Batch)
Set objConnection = CreateObject("ADODB.Connection")
objConnection.Open "MyConnection"
szSQLSelect = "Select * from db.dbo.MyTable " & _
"where batch = "'" & Batch & "'"
szSQLInsert = "Insert into db.dbo.MyTable " & _
"select '" & _
Batch & "', getdate(), null from db.dbo.MyOtherTable where ccco = '00000'"
set rsSelect = objConnection.Execute(szSQLSelect)

' Make sure we DO NOT get records back. If we get records back, we know this is not a new availability row
If rsSelect.EOF Then
' insert
set rsInsert = objConnection.Execute(szSQLInsert)
End If

' Close the recordset
rsSelect.Close
objConnection.Close

End Sub

VBEND

LibLoad>IEAuto.dll,hIE
If>hIE=0
MessageModal>Could not load IEAuto.dll, make sure it is in the path or edit the LibLoad line.
Goto>end_script
EndIf

//Move the mouse cursor out of harm's way to avoid causing mouseover events to interrupt
MouseMove>0,0
Let>delay=1

LibFunc>hIE,CreateIE,IE[0],0

LibFunc>hIE,Navigate,r,%IE[0]%,https://www.website.com
LibFunc>hIE,WaitIE,r,%IE[0]%
Wait>delay

Let>FrameName={""}
Let>FormName={"loginForm"}
Let>FieldName={"username"}
Let>FieldValue={"User"}
LibFunc>hIE,FormFill,r,%IE[0]%,str:FrameName,str:FormName,str:FieldName,str:FieldValue,0

Let>FrameName={""}
Let>FormName={"loginForm"}
Let>FieldName={"j_password"}
Let>FieldValue={"password"}
LibFunc>hIE,FormFill,r,%IE[0]%,str:FrameName,str:FormName,str:FieldName,str:FieldValue,0

Let>delay=3

Let>FrameName={""}
Let>FormName={"loginForm"}
Let>TagValue={"https://www.website.com/loginButton.gif"}
LibFunc>hIE,ClickTag,r,%IE[0]%,str:FrameName,str:FormName,INPUT,SRC,str:TagValue
LibFunc>hIE,WaitIE,r,%IE[0]%
Wait>delay

LibFunc>hIE,Navigate,r,%IE[0]%,https://www.website.com/begin.do
LibFunc>hIE,WaitIE,r,%IE[0]%
Wait>delay

Label>loop

LibFunc>hIE,Navigate,r,%IE[0]%,https://www.website.com/getList.do?prodLineID=A
LibFunc>hIE,WaitIE,r,%IE[0]%
Wait>delay

Let>TDCount=13

Repeat>TDCount

//Get Product Data...
Let>Product_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,Product
MidStr>r_6,1,r,Product

If>Product=No Products Found for this Product Line
Let>TDCount=120
ENDIF

let>Prods=Product
MidStr>Product,1,4,Prods
If>Prods=Cust
Let>TDCount=120
ENDIF

If>Prods=No P
Let>TDCount=120
ENDIF

If>Product=" "
Let>TDCount=120
ENDIF

If>TDCountTDCount=TDCount+1

//Get Data...
Let>X1_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,X1
MidStr>r_6,1,r,X1

Let>TDCount=TDCount+1

//Get Batch Data...
Let>Batch_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,Batch
MidStr>r_6,1,r,Batch

Let>TDCount=TDCount+1

//Get Data...
Let>X2_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,X2
MidStr>r_6,1,r,X2

Let>TDCount=TDCount+1

//Get Data...
Let>x3_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,x3
MidStr>r_6,1,r,x3

Let>TDCount=TDCount+1

//Get Data...
Let>x4_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,x4
MidStr>r_6,1,r,x4

Let>TDCount=TDCount+1

//Get Data...
Let>x5_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,x5
MidStr>r_6,1,r,x5

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

VBRun>CheckAndInsert,%Batch%

ENDIF

Until>TDCount>119

LibFunc>hIE,Navigate,r,%IE[0]%,https://www.website.com/getList.do?prodLineID=B
LibFunc>hIE,WaitIE,r,%IE[0]%
Wait>delay

Let>TDCount=13

Repeat>TDCount

//Get Product Data...
Let>Product_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,Product
MidStr>r_6,1,r,Product

If>Product=No Products Found for this Product Line
Let>TDCount=120
ENDIF

let>Prods=Product
MidStr>Product,1,4,Prods
If>Prods=Cust
Let>TDCount=120
ENDIF

If>Prods=No P
Let>TDCount=120
ENDIF

If>Product=" "
Let>TDCount=120
ENDIF

If>TDCountTDCount=TDCount+1

//Get Data...
Let>X1_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,X1
MidStr>r_6,1,r,X1

Let>TDCount=TDCount+1

//Get Batch Data...
Let>Batch_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,Batch
MidStr>r_6,1,r,Batch

Let>TDCount=TDCount+1

//Get Data...
Let>X2_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,X2
MidStr>r_6,1,r,X2

Let>TDCount=TDCount+1

//Get Data...
Let>x3_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,x3
MidStr>r_6,1,r,x3

Let>TDCount=TDCount+1

//Get Data...
Let>x4_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,x4
MidStr>r_6,1,r,x4

Let>TDCount=TDCount+1

//Get Data...
Let>x5_SIZE=4098
LibFunc>hIE,ExtractTag,r,%IE[0]%,,TD,TDCount,0,x5
MidStr>r_6,1,r,x5

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

//ditto...

Let>TDCount=TDCount+1

VBRun>CheckAndInsert,%Batch%

ENDIF

Until>TDCount>119

Goto>loop

Label>end_script

Post by **Marcus Tettmar** » Mon Aug 13, 2007 2:33 pm

The problem is probably that you are creating the "objConnection" object many times inside the loop but never freeing it. Add this line to the end of your CheckAndInsert VBScript sub:

Set objConnection = Nothing

You are calling CheckAndInsert inside a nested loop. Inside CheckAndInsert you create a reference to a new object but never release the object by setting it to Nothing afterwards. So you are creating thousands of instances of this ADODB connection object.

Whenever you do a Set something = something in VBScript you should always set that something to nothing later to tell VBScript to release it and all memory associated with it.

howl · Post by **howl** » Mon Aug 13, 2007 7:08 pm

I tried setting my objects to nothing but the the script continues to slowly eat up memory. I've found that killing IE frees the memory so I'm going to do that periodically within the script.

howl · Post by **howl** » Mon Aug 13, 2007 7:31 pm

Can you step me through the CreateIE and KillIE commands? Specifically, how do I reference the IE session number so I can kill the correct one. I'll be repeating Create and Kill commands.

Thanks,

Post by **Marcus Tettmar** » Mon Aug 13, 2007 8:08 pm

I don't see how it can be that as you are only creating an IE instance once. So you only need to kill it once, and you are probably killing it manually when you close IE.

howl · Post by **howl** » Mon Aug 13, 2007 8:27 pm

Yes, in the script I sent you, I create IE once and I kill IE once. However, I tested a script where I:
create IE
loop through pages multiple times
kill IE
create IE
loop through pages multiple times
kill IE

So when I run this script, my memory usage ticks up a few hundred k each time I loop through the pages. Then when I kill IE, memory usage drops. So I would like to try and include the create IE and kill IE in a repeat but I need a little help understanding how to keep track of the session #.

Thanks

Post by **Marcus Tettmar** » Mon Aug 13, 2007 8:30 pm

LibFunc>hIE,CreateIE,IE[0],0

IE[0] is the instance reference. So we pass it to KillIE.

LibFunc>hIE,KillIE,r,IE[0]

Since you've killed it you could use IE[0] again in subsequent CreateIE calls.

You could call it what you like:

LibFunc>hIE,CreateIE,fred,0
..
LibFunc>hIE,KillIE,r,fred

howl · Post by **howl** » Tue Aug 14, 2007 12:58 pm

FYI,
My script now does the following:

repeat
create IE
cycle through the same 5 web pages 40 times (roughly 10 minutes)
kill IE
until repeat count > 100

If I monitor system performace while the script runs, I can see memory usage slowly creep up while cycling through web pages. Then memory is released on the kill. This solution works well but I'm not sure why the memory usage goes up while cycling through the web pages.

Thanks for your help.

Post by **Marcus Tettmar** » Tue Aug 14, 2007 1:37 pm

Sounds more like a problem with IE then. Which version of IE are you running?

howl · Post by **howl** » Tue Aug 14, 2007 2:23 pm

That could be the problem. We're running this on an old spare PC with v5 of IE.