Retriving last text entry from table.

crowder80537 · Post by **crowder80537** » Fri Jan 04, 2008 7:07 pm

I have a program that in real time monitors my aircraft engine and constantly logs many parameters of performance in a text file. The file quickly becomes large. I want to retrive the latest (last entry) to constantly graph these parameters in real time. I need a script to accomplish this data retrival repededly in real time. Would it be something along (Line Number End-of-file minus one?) How would I put this concept into a script? I have done very little scripting.

Thanks, Jim

JRL · Post by **JRL** » Fri Jan 04, 2008 7:35 pm

Define "large" and "real time". The reason I ask is that reading deep into text files gets slower as the file gets larger. If these are really large files and the time between additions is really short, "real time" may not be possible using readln>.

This discussion might have an answer for you.

If this concept sounds plausible, we'll help you work on a script.

crowder80537 · Post by **crowder80537** » Fri Jan 04, 2008 8:10 pm

You have hit on my concerns. I do not know the size of the file, but I am sure it will quickly become quite large. The manufacturer of the actual instrument recording the data supplied the logging software, and I have so far been unable to run the monitor. I am connecting it to a HP tablet computer that has no serial port. It will be a week or so before I receive a USB to Serial converter that will allow me to actually observe the resulting log file. I do have a sample of the file that will be created and have been working with that.

As to what is real time, I suspect it will be every second or so. I have the graph display page designed using Excel's graphing and after a couple of weeks of fun work and a lot of work-arounds, it is pretty sharp and working with sample data. But not real time. I plan to use "DDEPoke>Excel, filename, row number and col number, data for cell" example, from Marcus Tettmar, with my thanks for his work, to insert the data. A "last entry" retrival command of some sort would be perfect if it exists.

Many thanks for your perceptive and speedy response.

Jim

JRL · Post by **JRL** » Fri Jan 04, 2008 8:37 pm

Jim,
Another possibility, would be to readfile> the original log file, delete the original log file, then immediately writeln> the data just read to a backup file and finally deal with the copied data. Using a method like this would mean that you would always read from a small file and you would have all of your data stored in a second file that you created.

Something like:

Label>Loop
ReadFile>filename,data
DeleteFile>filename
WriteLn>newfilename,wres,%data%

//Do something appropriate with %data%
//Poke data bits into excel chart

Goto>Loop

The engine monitoring application would have to release the file between writes for this to work or else you will not be able to delete the log file. It would also be necessary that a delete cannot occur while the application writes to the log file or you could lose data.

Does this make sense? Just a thought.

crowder80537 · Post by **crowder80537** » Fri Jan 04, 2008 10:12 pm

That would work except that I will have the tablet computer mounted on my instrument panel, keyboard folded under and be flying my airplane. I will be scanning six cylinder heads temps, six exhaust gas temps, RPM, oil temperature, oil pressure, fuel pressure, and more bar charts which show the values, and also change color depending on the limits I have programed. As a part of pilots scan, I will be mostly looking at green, but then yellow when limits are approached. When out of limits, the offending bars and numbers turn red--not good. Older gauges are mostly round with dials and much harder to monitor.

If needed, there is software that will, separately of the log file, sniff the serial port and provide the data separately. I should then be able to poke that into Excel. However, that would be another overhead program and not as much fun as using a script or macro. Also, I would have something else to buy. I plan to use a little less that half of the display screen for the engine graphics, and the other half for GPS driven moving aeronautical chart for navigation and position awareness. I already have that working.

For your interest, I am retired, 71 1/2, have build the two seater high wing airplane, but was unhappy with some of the results. I am in the process of changing the engine type and about half of the instrumentation.

I plan to budget a couple of weeks to resolving this current issue and if not, I will buy the serial port sniffing software and try that.

Many, many thanks for helping me.

Jim

JRL · Post by **JRL** » Fri Jan 04, 2008 10:48 pm

Wow!... sounds very interesting. Always happy to brainstorm. Let us know if you need any real help.

crowder80537 · Post by **crowder80537** » Fri Jan 04, 2008 11:48 pm

OH, I need help alright. There's got to be some way to quickly select that newest line of data. If I brainstorm it enough publicly, maybe someone will come up with it.

Thanks again, Jim

JRL · Post by **JRL** » Sat Jan 05, 2008 3:30 am

Rereading your third post I'm wondering whether I made the concept clear. You said: "That would work except that I will have the tablet computer mounted on my instrument panel, keyboard folded under and be flying my airplane." As though you feel that you will need to be doing something on the computer. Just to be clear, the script would be doing all the work. You would be watching the screen and flying your airplane.

crowder80537 · Post by **crowder80537** » Sat Jan 05, 2008 4:09 am

You're right. I didn't read that clearly. If I don't come up with a straight forward answer, it might make a good work-around. One thought, the logging program starts out by writing column headers. I don't know the coding and it might rely on the spacing of those to format the subsequent writes. I might try to take a look at that.

I have no idea what language it is written in. If it is scripted, maybe I could have it do two writes and have it write directly to Excel. If it is complied, I have no concept how to reverse engineer it. I wish I was smarter.

Thanks for making me think.

Jim

Post by **Marcus Tettmar** » Sat Jan 05, 2008 8:24 am

Answering your original post (how to quickly get the last line of a file):

ReadFile>filename,data
Separate>data,CRLF,lines
Let>last_line=lines_%lines_count%

Discussion seems to have wandered a bit since your first post so forgive me if I have missed any extra requirement. Not sure why you think it would make a difference having the keyboard folded up. Macro Scheduler will run scripts whether or not you have a keyboard, or where it is placed.

crowder80537 · Post by **crowder80537** » Sat Jan 05, 2008 4:46 pm

You are correct. I misunderstood the previous suggestion thinking that action clearing the directory and log file required operator action rather than being done by the script.

My discussion on your group caused me to check the properties of the logging software that was provided me by the monitoring hardware manufacturer that provides the data I am working with. It turns out that it itself is a script program and that a Inno Setup Unpacker exists that would allow me to modify that script. As that program already has the data, it would seem to be the one to poke it into Excel. It is an open source and free program, so there is no issue there. I have also written the software company in Germany and await a reply. I am moving slowly on this because my background is so limited. I think I need to educate myself before I plunge in.

Many thanks again, Jim