Python自带TimedRotatingFileHandler巨坑，日志无法按天分割，使用需谨慎（附源码逻辑解释）

      日前公司需要用到python做一个后台的自动任务。以前使用python写日志都是写入同一个文件里头。但考虑到该任务每日日志量较大，为以后有什么问题方便排查，便到网上查找python按天分割的日志记录方法，发现python自带一个TimedRotatingFileHandler,便按照网上的代码例子把日志给配到了自己的项目中。

        相关链接：https://blog.csdn.net/energysober/article/details/53263295

    TimedRotatingFileHandler类的重点就在于log_file_handler = TimedRotatingFileHandler(filename=LOG_PATH+"thread_", when="D", interval=1, backupCount=7)这个初始化方法。filename是日志名，when是间隔时间单位，interval是间隔时间数值，backupCount是要保留几个分割后的日志文件。

    我当时想要配的是每天的日志都分开来打吧，于是便设置了when="D",interval=1。从字面上来看也是没问题了。但是自测的时候把间隔调成了秒，方便调试嘛。调试也没什么问题，每隔两三秒运行一次打印日志，日志信息都会打印到最新的日志文件中，而以前的日志信息会按照设定好的命名格式放在另一个日志文件中。

     于是我就把这个项目开发自测好后，就上测试环境了。测试人员测了两三天，发现日志根本就没有滚动呀。我去看了也是一头雾水。于是点进去TimedRotatingFileHandler看了源码，看了里面的时间计算逻辑才恍然大悟为什么测试环境无法分割的原因。附上init方法

def __init__(self, filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, atTime=None):
    BaseRotatingHandler.__init__(self, filename, 'a', encoding, delay)
    self.when = when.upper()
    self.backupCount = backupCount
    self.utc = utc
    self.atTime = atTime
    # Calculate the real rollover interval, which is just the number of
    # seconds between rollovers.  Also set the filename suffix used when
    # a rollover occurs.  Current 'when' events supported:
    # S - Seconds
    # M - Minutes
    # H - Hours
    # D - Days
    # midnight - roll over at midnight
    # W{0-6} - roll over on a certain day; 0 - Monday
    #
    # Case of the 'when' specifier is not important; lower or upper case
    # will work.
    if self.when == 'S':
        self.interval = 1 # one second
        self.suffix = "%Y-%m-%d_%H-%M-%S"
        self.extMatch = r"^\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}(\.\w+)?$"
    elif self.when == 'M':
        self.interval = 60 # one minute
        self.suffix = "%Y-%m-%d_%H-%M"
        self.extMatch = r"^\d{4}-\d{2}-\d{2}_\d{2}-\d{2}(\.\w+)?$"
    elif self.when == 'H':
        self.interval = 60 * 60 # one hour
        self.suffix = "%Y-%m-%d_%H"
        self.extMatch = r"^\d{4}-\d{2}-\d{2}_\d{2}(\.\w+)?$"
    elif self.when == 'D' or self.when == 'MIDNIGHT':
        self.interval = 60 * 60 * 24 # one day
        self.suffix = "%Y-%m-%d"
        self.extMatch = r"^\d{4}-\d{2}-\d{2}(\.\w+)?$"
    elif self.when.startswith('W'):
        self.interval = 60 * 60 * 24 * 7 # one week
        if len(self.when) != 2:
            raise ValueError("You must specify a day for weekly rollover from 0 to 6 (0 is Monday): %s" % self.when)
        if self.when[1] < '0' or self.when[1] > '6':
            raise ValueError("Invalid day specified for weekly rollover: %s" % self.when)
        self.dayOfWeek = int(self.when[1])
        self.suffix = "%Y-%m-%d"
        self.extMatch = r"^\d{4}-\d{2}-\d{2}(\.\w+)?$"
    else:
        raise ValueError("Invalid rollover interval specified: %s" % self.when)

    self.extMatch = re.compile(self.extMatch, re.ASCII)
    self.interval = self.interval * interval # multiply by units requested
    if os.path.exists(filename):
        t = os.stat(filename)[ST_MTIME]
    else:
        t = int(time.time())
    self.rolloverAt = self.computeRollover(t)

     它在初始化时会计算出什么时候日志文件需要进行分割。实例变量self.rolloverAt便存放着下次分割的时间。而判断是否需要把日志分割开来的逻辑，居然是根据当前文件的修改时间或当前时间戳来的。每次初始化这个类，这个时间就会重新算一次。这就是在测试环境两三天都无法进分割日志的原因！！！因为测试环境经常需要发布版本，而这个类经常被重新初始化，然后就会重新计算24小时才会进行日志分割。附上它计算什么时候分割日志的时间点的代码

    def computeRollover(self, currentTime):
        """
        Work out the rollover time based on the specified time.
        """
        result = currentTime + self.interval
        # If we are rolling over at midnight or weekly, then the interval is already known.
        # What we need to figure out is WHEN the next interval is.  In other words,
        # if you are rolling over at midnight, then your base interval is 1 day,
        # but you want to start that one day clock at midnight, not now.  So, we
        # have to fudge the rolloverAt value in order to trigger the first rollover
        # at the right time.  After that, the regular interval will take care of
        # the rest.  Note that this code doesn't care about leap seconds. :)
        if self.when == 'MIDNIGHT' or self.when.startswith('W'):
            # This could be done with less code, but I wanted it to be clear
            if self.utc:
                t = time.gmtime(currentTime)
            else:
                t = time.localtime(currentTime)
            currentHour = t[3]
            currentMinute = t[4]
            currentSecond = t[5]
            currentDay = t[6]
            # r is the number of seconds left between now and the next rotation
            if self.atTime is None:
                rotate_ts = _MIDNIGHT
            else:
                rotate_ts = ((self.atTime.hour * 60 + self.atTime.minute)*60 +
                    self.atTime.second)

            r = rotate_ts - ((currentHour * 60 + currentMinute) * 60 +
                currentSecond)
            if r < 0:
                # Rotate time is before the current time (for example when
                # self.rotateAt is 13:45 and it now 14:15), rotation is
                # tomorrow.
                r += _MIDNIGHT
                currentDay = (currentDay + 1) % 7
            result = currentTime + r
            # If we are rolling over on a certain day, add in the number of days until
            # the next rollover, but offset by 1 since we just calculated the time
            # until the next day starts.  There are three cases:
            # Case 1) The day to rollover is today; in this case, do nothing
            # Case 2) The day to rollover is further in the interval (i.e., today is
            #         day 2 (Wednesday) and rollover is on day 6 (Sunday).  Days to
            #         next rollover is simply 6 - 2 - 1, or 3.
            # Case 3) The day to rollover is behind us in the interval (i.e., today
            #         is day 5 (Saturday) and rollover is on day 3 (Thursday).
            #         Days to rollover is 6 - 5 + 3, or 4.  In this case, it's the
            #         number of days left in the current week (1) plus the number
            #         of days in the next week until the rollover day (3).
            # The calculations described in 2) and 3) above need to have a day added.
            # This is because the above time calculation takes us to midnight on this
            # day, i.e. the start of the next day.
            if self.when.startswith('W'):
                day = currentDay # 0 is Monday
                if day != self.dayOfWeek:
                    if day < self.dayOfWeek:
                        daysToWait = self.dayOfWeek - day
                    else:
                        daysToWait = 6 - day + self.dayOfWeek + 1
                    newRolloverAt = result + (daysToWait * (60 * 60 * 24))
                    if not self.utc:
                        dstNow = t[-1]
                        dstAtRollover = time.localtime(newRolloverAt)[-1]
                        if dstNow != dstAtRollover:
                            if not dstNow:  # DST kicks in before next rollover, so we need to deduct an hour
                                addend = -3600
                            else:           # DST bows out before next rollover, so we need to add an hour
                                addend = 3600
                            newRolloverAt += addend
                    result = newRolloverAt
        return result

     如果项目是持续运行的，这个类只初始化一次，那这个日志分割还是可以达到目的的。而且里面还设计有按年，按月分割的，一年内项目不重启不升级，说实话概率还是蛮低的，一重启这个时间就会重新算，然后日志分割时间就会推迟了。阅读源码后看到它有一个按midnight分割日期的功能。就是它会在计算当前时间到凌晨零点的秒数，然后每次写日志时都会判断过没过凌晨，到了后就会自动分割日志。这个功能还蛮符合我需求的。

       但这次我自己在开发环境自测了下，先自己打印一条日志。然后把系统时间（调整时区）调为第二天的时间，再一次打印日志，发现日志还在一起，并没有分割。怎么回事？ 看了下源码，发现代码里是根据time.time()返回一个当前时间戳来判断是否过凌晨的，不管系统时间如何调整，程序返回的时间戳都是一秒一秒加的。所以如果想要真的分割一次日志，还真得跨越一次凌晨才行。附上判断是否需要分割的源代码：

    def shouldRollover(self, record):
        """
        Determine if rollover should occur.

        record is not used, as we are just comparing times, but it is needed so
        the method signatures are the same
        """
        t = int(time.time())
        if t >= self.rolloverAt:
            return 1
        return 0