Searching huge text file - solution!

[Dorian (MJT support)]

Wohooo! I just figured out how to search large text files quickly. i just tested this script and it completed a search on a 176MB text file in only a second or so.

_____________________________

let>filename=type c:\testfile.txt | find "searchtext" /i > c:\testfile2.txt
WriteLn>c:\testfile.bat,result,%filename%


Let>RP_WINDOWMODE=0
Run Program>c:\testfile.bat

REMARK>I had to enter a wait command because
REMARK>this script seems to search huge files
REMARK>so fast it will try to open the output
REMARK>file before windows can generate it.
REMARK>If anyone finds a better way of doing
REMARK>this please let me know.

message>Finished Search, generating output file
wait>5
executefile>c:\testfile2.txt

[Dorian (MJT support)]

One question - how do I get around the space problem in a path when using a .bat file? for instance, "c:\testfile.txt" is fine, but "c:\my document\testfile.txt" won't work. I know there's probably a very simple answer.

[Ernest]

Hi,
congratulations to that "oneliner"

Cause there's nothing like "RunWait" (stops execution of the script till program has finished) which I know from AutoIt, I've helped myself with this:

Code: Select all

Label>FileProgressCheck
FileSize>c:\testfile2.txt , fsize1
Wait>0.5
FileSize>c:\testfile2.txt , fsize2
If>fsize2>fsize1, FileProgressCheck

Since WIN9x,

START "c:\my document\testfile.txt"

should work with "space" lines in batch files.


Rgds,
Ernest

[Marcus Tettmar]

Hi,

In version 7.1 there *is* something like RunWait. Do this:

Let>RP_WAIT=1
Run>yourbatfile.bat

I often wonder if anyone ever reads the history list or the list of new features in our update announcements, and often wonder how many new features go completely unnoticed

See the help file topic for Run Program. Note that this is new to version 7.1

Regarding spaces in the filename, simply use quotes, as you would anywhere else in Windows:

Run>"c:\some directory with spaces\some place\mybatfile.bat"

[Dorian (MJT support)]

Thanks guys.

Those "'s work perfectly. I amended the search line to :

let>filename=type "c:\my documents\testfile.txt" | find "test" /i > "c:\my documents\testfile2.txt"

and now it allows me to search a file if there's a space in the path.

Then I added :

DeleteFile>c:\testfile.bat

... at the end, because each time the script was run it added another line to the batch file and it ended up searching the textfile four or five times.

I cannot believe how fast this searches such a large file.

So now I know this can be done easily I'll have to make the script do something actually useful.

[Ernest]

Hi H2000,

">" will create a new file and pipes the output to it. So you'll get always the result of the last execution. Nothing more. Therefore you don't have to delete it with every session. When you've finished completely, get rid of it ...

Code: Select all

//cmd = NT, command = W9x/...(?)
Run Program>cmd /c find "this" on\that\file.txt > output\file.txt

">>" will append output to the same file. You'll get the result of previous runs as well, piped to the same file (I'm using it for 24/7 logs).

Code: Select all

//cmd = NT, command = W9x/...(?)
Run Program>cmd /c find "this" on\that\file.txt >> output\file.txt

Ciao
Ernest

BTW: Marcus,
I won't get the new release (I'm blocked by "BeanCounters" at the finance department)

I's the current command reference (7.1) seperately available for download ? So I won't be endangered to advise "historic" ways of MSched coding ...

[Marcus Tettmar]

Hi H2000,

I's the current command reference (7.1) seperately available for download ? So I won't be endangered to advise "historic" ways of MSched coding ...

Yes, as always it is available for viewing from the support pages and available for download from the download page:

http://www.mjtnet.com/support.htm
http://www.mjtnet.com/downloads.htm

[Dorian (MJT support)]

Hi Ernest, many thanks for the pointers.

I added the delete statements as this macro is just the beginning of a much larger one and my needs dicate the files are not appended. The output file is just a temporary file for the macro to run a few more checks on.

The "real" version will allow the user to input the filename and search term, which then uses concat and writes the new batch file, and i didn't want my macro to generate huge obsolete textfiles on their computer.

[Ernest]

Hi H2000,
Is it necessary to use a batch?
Why not run the scan directly using a command shell (the DOSBox)?

Code: Select all

If>%OS_VER%=NT.4.*,WNT
If>%OS_VER%=<CodeOfWIN2K>,WNT
If>%OS_VER%=<CodeOfWIN95>,W9x
If>%OS_VER%=<CodeOfWIN98>,W9x
Goto>Exit

Label>WNT
Let>ComIntpr=cmd.exe
Goto>ScanSet

Label>W9x
Let>ComIntpr=command.exe

Label>ScanSet
Let>scanfile=%SYS_DIR%\%ComIntpr% /c find "<searchpattern>" "c:\my documents\testfile.txt" > c:\testfile2.txt

Let>RP_WINDOWMODE=0
Run Program>%scanfile%

Label>Exit

Ciao
Ernest

[Bob Hansen]

Hello Ernest.

I believe there is an error in the last script:
Label>W9x
Let>ComIntpr=command.exe
====================================

I think that "command.exe" should be "command.com". Correct line should be:
Label>W9x
Let>ComIntpr=command.com

But I like the approach to allow a script run on any Windows system. That's neat. Thanks for the new tool.

[Ernest]

Hi,
Ooops, you're right (I'm used to run it without any file extension)!

I don't know if MSched 7.1 cares about the DOS SysVar "COMSPEC".
If that's the case, you don't have to predefine diff. CmdIntpr. sections!

E.

[Dorian (MJT support)]

Hi Ernest, that's really helpful, thank you. Would that work with XP too, or what would the line for XP be?

[Ernest]

Hi H2K,
I guess - but sorry, I'm currently not using XP! Run SET at the commandline and check for the variable COMSPEC. Here you should? find the path to the command interpreter (no idea what XP offers in case of DOS functionality). I expect MSoft hasn't changed it. Should be "cmd.exe". Please check ...

BTW: You can do the same for WinME.

E.

[Dorian (MJT support)]

hi Ernest, FYI I tried "SET" in XP and this is what was returned under COMSPEC :

c:\windows\system32\cmd.exe

Over the weekend I bought a really useful book to help remind me of dos commands. It's called "Windows User's guide do DOS" by Carolyn Z Gillay & Bette A Peat.

It's an oldie but a goodie.

I already found I can add SORT to the FIND command, so it's proved realy helpful already.