背景
了解如何使用 python logging, 输出日志到文件中.
并初步结合 flask 框架以及多模块的环境配置, 尚不太成熟, 待改进
至于为什么要用 logging, 而不是之前的 print, 原因也简单, logging 的信息量更大, 如果输出到文件中也更方便.
Python Logging 碎碎念之配置文件
python 有自带的 logging 模块, 足够用了, 也挺好用.
用前先想想:
- 要能兼顾所有模块
- 要有时间戳
- 要方便配置, 比如更改 log文件名, log level 等
- 文件名不写死在代码中...
- ...
自然需要外置 conf 配置文件.
当然, 外置 conf 文件虽然配置起来方便, 但要在代码中操控它就有点麻烦, 不过从用户角度看, 这个还是首选. 具体的 pro 和 cons 请参考Logging — The Hitchhiker's Guide to Python
那:
- conf 文件格式有什么要求?
- conf 文件内容怎么写?
- conf 文件怎么读?
文件格式一般有几种:
- ini 文件: 类似 key=value
- yml 文件
- json 文件
对他们的解析不一样, 读取的时候用到的模块:
- ini 文件:
logging.config.fileConfg - yml/json 文件:
logging.config.dictConfig
示例见下节
文件内容怎么写?
一般比较重要的得包含:
- level: debug/info/warn...
- handlers
- file handler
- file name: 比如带时间戳等
- console handler
- file handler
- formatters
- file formatter
- console formatter
简单的示例如下, 详细示例请见文末的附录
[logger_root]
level=DEBUG
handlers=fileHandler
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=logFormatter
args=("_log/" + time.strftime("%%Y%%m%%d%%H%%M%%S")+'.log', 'a')
[formatter_logFormatter]
format=%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s
在代码中加载 log 配置文件
加载 yml/json 文件:
import logging
import logging.config.dictConfig(yaml.load(open('logging.yml', Loader=yaml.FullLoader))))
加载 .ini 文件
import logging
logging.config.fileConfig("logging.ini")
在模块中使用就很简单了,
logger = logging.getLogger(__name__)
logger.debug("log %s", it)
当然这只是很简单的场景, 实际应用中会需要用到 filter, 更改文件名, 获取 log 文件名等等
比如, 我想在程序运行结束的时候把 log 文件地址也输出到 log 文件中, 然而却发现有点难度, 试了几种方法, 只成功了下面这个:
LOG_FILE = logging.getLoggerClass().root.handlers[0].baseFilename
logger = logging.getLogger(__name__)
logger.debug(f"LOG_FILE: , {LOG_FILE}")
未成功之一: 我拿到的 handlers list 总是为空...
import logging
dir(logging.FileHandler)
>>> handler = job_logger.handlers[0]
>>> filename = handler.baseFilename
>>> print(filename)
'/tmp/test_logging_file'
还想输出 exception details
先在需要处理 exception 的地方 raise exception
raise Exception(msg)
然后, catch 的时候输出到 log 中
import traceback
except Exception as e:
msg = traceback.from_exec()
logger.debug(msg)
注意, 这里的 form_exec() 是不带参数的
我之前忽略了, 想当然的加了 e 作为参数, 结果错误: '>=' not supported between instances of 'Exception' and 'int'
Flask Logging
如果代码中用到了 flask 框架, 如何输出 log 呢?
同样, 重用前面的 log 配置, 在 app.py 中直接用 app.logger 即可, 底层用的还是 python logging 模块
app = create_app()
logger = app.logger
Reference
Python logging
- logging — Logging facility for Python — Python 3.8.3rc1 documentation
- Logging — The Hitchhiker's Guide to Python
flask logging
Logging — Flask Documentation (1.1.x)
附录: 大致的目录结构
log 配置文件的读取是在 module2/init.py 中;
import os
import logging
from logging.config import fileConfig
if not os.path.exists("_log"):
os.mkdir("_log")
logging.config.fileConfig("logging.conf"))
LOG_FILE = logging.getLoggerClass().root.handlers[0].baseFilename
logger = logging.getLogger(__name__)
logger.debug(f"LOG_FILE: , {LOG_FILE}")
main.py 启动 flask app
.root
├── logging.conf
├── logging.yml
├── main.py
├── README.md
├── _log
└── 20200503204102.log
├── module1
├── __init__.py
├── requirements.txt
├── src
│ ├── __init__.py
│ └── app.py
└── test
└── module2
├── __init__.py
├── src
├── __init__.py
└── test
├── __init__.py
├── requirements.txt
├── README.md
├── data
├── docs
├── invoke.yml
附录: logging.ini sample (摘自网络),
因为带了注释, 所以摘录在此供参考:
如需 yml 文件, 可参考python - Flask logging - Cannot get it to write to a file - Stack Overflow
#These are the loggers that are available from the code
#Each logger requires a handler, but can have more than one
[loggers]
keys=root,Admin_Client
#Each handler requires a single formatter
[handlers]
keys=fileHandler, consoleHandler
[formatters]
keys=logFormatter, consoleFormatter
[logger_root]
level=DEBUG
handlers=fileHandler
[logger_Admin_Client]
level=DEBUG
handlers=fileHandler, consoleHandler
qualname=Admin_Client
#propagate=0 Does not pass messages to ancestor loggers(root)
propagate=0
# Do not use a console logger when running scripts from a bat file without a console
# because it hangs!
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=consoleFormatter
args=(sys.stdout,)# The comma is correct, because the parser is looking for args
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=logFormatter
# This causes a new file to be created for each script
# Change time.strftime("%Y%m%d%H%M%S") to time.strftime("%Y%m%d")
# And only one log per day will be created. All messages will be amended to it.
args=("_log/" + time.strftime("%%Y%%m%%d%%H%%M%%S")+'.log', 'a')
[formatter_logFormatter]
#name is the name of the logger root or Admin_Client
#levelname is the log message level debug, warn, ect
#lineno is the line number from where the call to log is made
#04d is simple formatting to ensure there are four numeric places with leading zeros
#4s would work as well, but would simply pad the string with leading spaces, right justify
#-4s would work as well, but would simply pad the string with trailing spaces, left justify
#filename is the file name from where the call to log is made
#funcName is the method name from where the call to log is made
#format=%(asctime)s | %(lineno)d | %(message)s
#format=%(asctime)s | %(name)s | %(levelname)s | %(message)s
#format=%(asctime)s | %(name)s | %(module)s-%(lineno) | %(levelname)s | %(message)s
#format=%(asctime)s | %(name)s | %(module)s-%(lineno)04d | %(levelname)s | %(message)s
#format=%(asctime)s | %(name)s | %(module)s-%(lineno)4s | %(levelname)-8s | %(message)s
format=%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s
#Use a separate formatter for the console if you want
[formatter_consoleFormatter]
format=%(asctime)s | %(levelname)-8s | %(filename)s-%(funcName)s-%(lineno)04d | %(message)s
ChangeLog
- 2020-05-03 init