I have been trying to come up with a simple plan to make log rotation
more robust, particularly in not splitting log lines across files. Greg
Smith complained that lines could be split if logs are rotated on size,
but AFAICS the danger also exists for time based rotation.
My thought is to have the logger mark each true log line line end with a
null byte. The syslogger would have a state variable called, say,
safe_to_rotate, which would be set true if and only if the last thing
written to the log file was a "\n" from the input stream which was
followed by a null byte. The null bytes would of course never actually
be sent to the files. Log rotation would be delayed until variable this
was set true.
There is a small danger that the delay in log rotation might be
substantial - with a series of partial lines read. I suspect that in
practice this danger would be vanishingly small, and we might reasonably
expect that the condition would come true within a few cycles.
An alternative scheme would involve keeping some extra buffer(s) of data
so that we ensure we never write out a partial line to the file. But
that seems to me to involve a lot more processing and so I'm wary of it.
I don't think we should contemplate providing for CSV logs until we have
this problem solved, so it's rather important.
Thoughts?
cheers
andrew