Monitoring a Log File
From CodeCodex
There is frequently a need to scan system or application log files for notifications of particular events and the like. There are some subtle points about reading from a file that another process is actively writing to:
- When you reach the current end of the file, some more content may be appended a few seconds or a few minutes later. Thus, you need to wait a while and then try to resume reading from the current position. This requires clearing the EOF indication without closing and reopening the file.
- Many systems implement logfile rollover, where periodically the current log file is closed and renamed, and a new log file opened with the previous name. This is to keep the log files from growing without bounds.
Some Python code that handles both these issues is as follows:
LogFileName = "/var/log/messages" # or whatever log file you're wanting to monitor MonitorInterval = 60 # number of seconds between checks for new content LogFile = None # to begin with while True : if LogFile == None : LogFile = open(LogFileName, "r") #end if while True : Line = LogFile.readline() if Line == "" : break # end of file reached #end if ... do whatever processing of Line is necessary ... #end while if \ os.path.realpath("/proc/self/fd/%d" % LogFile.fileno()) \ != \ os.path.realpath(LogFileName) \ : # rollover has started, might still be in progress LogFile.close() LogFile = None else : LogFile.seek(0, 1) # clear EOF state #end if time.sleep(MonitorInterval) # wait for more content, also gives time for rollover to complete #end while
Note that the check whether the log file still has the same name is reliant on the functionality of procfs under Linux—will this work under any other systems?