Looking for a Better way to find a line and insert

[kpassaur]

I have a csv file and I want to search for a value and there are no line numbers. I can use Regex to find the value, but there is no line number in the csv file and I want the contents of the line. not just one value.

Naturally I can read each line, use the separate command, check for the value and when I have a match I will have the line number. However, I think this would be rather slow on a file that has thousands of lines.

The reason for the line number is so that I can read the entire contents of the line. Once I have the contents, I will be inserting a line after it.

I realize that I could add a colum of line numbers, however, when I insert a line I would have to rename the line number column to the end of the file. Any ideas?

[ainterne]

Hi,

For what it's worth, I wanted to share with you what I use. I look for positions and use that not looping lines one by one.

I ran this through a file with 20,000 lines and it took a fraction of a seconds to find two instances of a part of line and returns the whole line.
Then from the CRLF just add another line with what you want.

However this is what I wanted to share.

I deal with files that build up millions of lines every 3 or 4 hours. The thing I have learned is how disks play an influence on reading and writing files.
For example you can only write to a disk at a particular speed. It's something like, 0.0043 of a second per write. Marcus always tells me about writing out to temp files, deleting the original and copying the temp back to the original file name and the benefits of that.
There lies the problem in the "writes" You are governed by the speed of the disk. I have tried stripping raid setup's but no substantial gain.

The way I got over this is I run a "ram drive". I copy files that I need to read and write from the ram drive (backed up of course). The difference is huge when dealing with really big files. I would take a look at "http://www.superspeed.com/desktop/ramdisk.php"
I have a 64bit machine with 16gig of memory so assigning 2 gig to temp files etc is not a problem. However the software above can use more than the 3.2 gig that people put in 32bit machines.

Anyway, hope that helps some.....I am no expert with the scripting but I
am pretty good at embedded systems and how to use memory to my advantage.

Here is the basis of my script I use to get though big files looking for instances of something and pulling the whole lines.
So this will return the whole line if you just give it something in the line you are looking for.

Code: Select all

ReadFile>c:\tt\sandpit\big.txt,data
//Anything you are looking for
Let>token=QQQ
Length>data,len
repeat>k
    position>token,data,1,pos_1
    if>pos_1>0
    MidStr>data,pos_1,len,new_data
    Let>data=new_data
    Let>pos_1=0
    Position>crlf,data,1,pos_2
    MidStr>data,pos_1,pos_2,answer
    MidStr>data,5,len,new_data
    Let>data=new_data
    MessageModal>answer
    else
    if>pos_1=0
    goto>end
    endif
  Endif
until>k
Label>end

[kpassaur]

Thank you for the feedback and sample. I will use a similar method.