tail in Python

Tags:

Well, Drac is proving unstable for my new mail server, so I'm having to think of an alternative. As much as I hate log file readers such as exact, I thought I might write a short Python program to do the reading and database management (since it's trivial to write and debug). Here's the tail with following portion of the code:

tail.py, including some code from the Python Cookbook by Ed Pascoe and Erik Max Francis.

#!/usr/bin/env python

import os
import sys
import time
from optparse import OptionParser

def tail_lines(fd, linesback = 10):
    # Contributed to Python Cookbook by Ed Pascoe (2003)
    avgcharsperline = 75

    while 1:
        try:
            fd.seek(-1 * avgcharsperline * linesback, 2)
        except IOError:
            fd.seek(0)

        if fd.tell() == 0:
            atstart = 1
        else:
            atstart = 0

        lines = fd.read().split("\n")
        if (len(lines) > (linesback+1)) or atstart:
            break

        avgcharsperline=avgcharsperline * 1.3

    if len(lines) > linesback:
        start = len(lines) - linesback - 1
    else:
        start = 0

    return lines[start:len(lines)-1]

def handle_line(line):
    print line,


def do_tail(filename, lines, follow, func = handle_line):
    fd = open(filename, 'r')

    for line in tail_lines(fd, lines):
        func(line + "\n")

    if not follow:
        return

    while 1:
        where = fd.tell()
        line = fd.readline()
        if not line:
            fd_results = os.fstat(fd.fileno())
            try:
                st_results = os.stat(filename)
            except OSError:
                st_results = fd_results

            if st_results[1] == fd_results[1]:
                time.sleep(1)
                fd.seek(where)
            else:
                print "%s changed inode numbers from %d to %d" % (filename, fd_results[1], st_results[1])
                fd = open(filename, 'r')
        else:
            func(line)

def main(argv = sys.argv):
    parser = OptionParser()
    parser.add_option("-n", "--number", action="store", type="int", dest = "number", default=10)
    parser.add_option("-f", "--follow", action="store_true", dest = "follow", default=0)
    (options, args) = parser.parse_args()
    do_tail(args[0], options.number, options.follow, handle_line)

if __name__ == "__main__":
    try:
        main(sys.argv)
    except KeyboardInterrupt:
        pass

3 old-style comments

  1. SlestakMarch 18, 2004 at 07:43 AM.

    How would I overload handle_line from my calling script to be able to receive a list from tail.py? A lambda in the do_tail 4th arg? I apologize if its an elementary python question. Could you point me to a reference that'll explain handle_line. I noticed on google 3-4 references to python code using this construct.
  2. Neil Blakey-MilnerMarch 25, 2004 at 09:57 PM.

    Sorry it took me so long to reply, I've been away at a conference.

    Where it says:

            do_tail(args[0], options.number, options.follow, handle_line)
    

    Replace that with:

            lines = []
            do_tail(args[0], options.number, options.follow, custom_line)
    

    Somewhere, declare the custom_line function:

    def custom_line(line):
        lines.append(line)
    

    Then, after do_tail returns, the 'lines' list will contain each line of the file.

    Good luck!

  3. mysurfaceOctober 29, 2007 at 11:47 AM.

    That is great! I am searching for this solution.
    But got one case, this tail.py didn't cater. Let me illustrate the case:

    ./tail.py -f log.txt

    at another terminal i do
    for ((a=0;a<10;a++)) do echo "xxx $a" >> log.txt;done;
    echo " " > log.txt
    for ((a=0;a<10;a++)) do echo "xxx $a" >> log.txt;done;

    It fails to tail after I do echo " " > log.txt
    I understand the normal tail fails too, but I am seeking for solution to cater that too. I am very new to python, I am currently reading your codes and try to understand.

    Thanks for sharing.
blog comments powered by Disqus