Getting Information from a 10 MB big Text file SLOW

Technical support and scripting issues

Moderators: JRL, Dorian (MJT support)

Post Reply
olllllliii
Pro Scripter
Posts: 60
Joined: Tue Dec 22, 2009 9:51 am
Location: Mannheim ( Germany )
Contact:

Getting Information from a 10 MB big Text file SLOW

Post by olllllliii » Fri Nov 09, 2012 2:59 pm

Hi all ,

i have a txt file ...with a size of 10 MB
Let>zu=CRLF
Let>diezeile=2

Reading the file into an Array....i tryed that line by line with Readln...
but this takes 5 Minutes for each ...article ...but they are more than 50000
and i get new prices each day.
My Computer is an i7 third generation , 16 GB RAM , 512 GB SSD....its normally very fast


The file looks like

Artikel-Nr-oP Verkaufspreis2exkl
10140354 7,7
10140356 15,4
10140357 44,9
10140358 29,4
10170000 44,1
10170001 15,6
10170005 25
10180000 19,8
10180001 52,2
10180004 19,8
10180008 26,7

Let>SEnummer=10170001

ReadFile>C:\Users\Steffen\Desktop\Secomp_eKatalog.txt,strinfo
separate>strinfo,%zu%,Arrkomplett

Label>zeilending

Let>strLine=Arrkomplett_%diezeile%
separate>strLine,TAB,ArrZiel
Let>vergleichsnummer=ArrZiel_1

IF>%SEnummer%=%vergleichsnummer%
Let>outPut=ArrZiel_2
WaitClipboard
PutClipboard>%outPut%
MDL>%outPut%

//

EXIT
ELSE
Add>diezeile,1
Message>%diezeile%
ENDIF
Goto>zeilending

the first element is the article number ....the 2nd is the new price
Oliver Hilger Mannheim
alias Olllllliii

User avatar
Marcus Tettmar
Site Admin
Posts: 7395
Joined: Thu Sep 19, 2002 3:00 pm
Location: Dorset, UK
Contact:

Post by Marcus Tettmar » Sat Nov 10, 2012 8:28 am

A faster way would probably be to use ADO and DBConnect, DBquery, treating the txt file as a sql data source, assuming you can specify which rows to return to limit the data in memory.

Another method would be to use VBScript and FileSystemObject.
Marcus Tettmar
http://mjtnet.com/blog/ | http://twitter.com/marcustettmar

Did you know we are now offering affordable monthly subscriptions for Macro Scheduler Standard?

olllllliii
Pro Scripter
Posts: 60
Joined: Tue Dec 22, 2009 9:51 am
Location: Mannheim ( Germany )
Contact:

Post by olllllliii » Mon Nov 12, 2012 8:49 am

Thank you Marcus

....i will try it this way .....

Kind Regards
Oliver Hilger
Oliver Hilger Mannheim
alias Olllllliii

Post Reply
Sign up to our newsletter for free automation tips, tricks & discounts