Preserve Log files option

pgriffin · Post by **pgriffin** » Wed Dec 07, 2005 2:45 pm

I see only the option to clear the log files for a script on each run or to append to the existing log file. When a script hangs on a line of code or a loop, it might run for quite a while. This would make the log file so large that it can't be easily edited. Currently I use a script to pull out only a number of lines from the log. My customers sometimes stop and restart scripts and, in some cases, this clears the log and there is no trail to investigate.

I would like to see an option to archive log files instead of only clear the log file or append the existing log file.

Bob Hansen · Post by **Bob Hansen** » Thu Dec 08, 2005 2:30 am

I would like to see an option to archive log files instead of only clear the log file or append the existing log file.

What is wrong with appending? I am not clear on what you are looking for.

On the screen for logging options, remove the checkmark from "Empty Log File Before Each Run" will do that. I have log files that are over 3 years of history. This log should not clear if customer starts/stops, etc. And you can make a simple copy/rename for archiving as separate file. I find that the appended file is a great archive.

What am I not understanding?

pgriffin · Post by **pgriffin** » Thu Dec 08, 2005 3:08 am

You are not understanding that if the program gets stuck (and hence a log would be very handy) and it runs for a couple of hours, the log file size can be so large that it is uneditable. This happened just two days ago and I had to write a script to read the log file for me. Also, when the customer has no option (as when I'm not available) but to stop/start a script it does clear the log because I don't append the log because of the size they can reach.

I am thinking of calling the main scripts from a "setup" script which first archives the previous log file, then calls the scripts I actually need to run.

Many of my customer's scripts run constantly for 10 hours per day. That could end up being a ridiculously large log file if it was constantly appended.

Bob Hansen · Post by **Bob Hansen** » Thu Dec 08, 2005 3:56 am

OK, I have a better picture, thanks.....

Any way of modifying your script to make a copy every every five minutes somehow? May probably have a sharing problem. Thinking of a small sub routine, something like this structure:

Start Sub routine
Record Time of log start (first time only in this loop,
Check current time for 5 mins duration
When five minutes, copy current.log to log_n.log, log_n+1.log, log_n+2.log, etc.
Delete current.log
Create blank current.log
Record Time of log start
Increase "n" for log file name.
End of sub routine

jpuziano · Post by **jpuziano** » Thu Dec 08, 2005 5:18 am

While most of us probably only use logging when trying to track down a problem, I suspect there are others who actually use logging on certain macros as a way of recording what actually did happen and when, say a file being transfered, etc.

If people have logfiles out there recording the results of macros that have been running for many years... then I can see additional reasons why being able to archive logfiles might be useful.

The problem here is the huge size of the logfiles being created so... what if we had an option to archive the logfile based on size? If we could enter a Max Logfile Size value in the Macro Properties/Logging Tab, then when the logfile reached or exceeded that, it could:

- take the existing logfile: macro.log
- and change its name to: macro(archived 2005-12-07 13:10.05).log
- open a new(empty) file: macro.log
- and continue logging

That way, it would never create log files bigger than our favorite editors can handle as we could set the max size.

Just an idea... If this or any other flavors of Logfile Archiving sounds useful to anyone out there, you should let support know.

Post by **Marcus Tettmar** » Thu Dec 08, 2005 8:24 am

You could just write a script to archive log files if they are over a certain size:

FileSize>Logfile,size
If>size>max_size
Year>yy
Month>mm
Day>dd
MoveFile>Logfile,LogFile.%yy%%mm%%dd%
Endif

Our web server has a log rotate script running under a cron job which does the same kind of thing. You could have a macro with code similar to above which archives each of your log files. Have it run once a day, or whenever, and the log files are archived if over a certain size.