Python 在多个模块中使用日志记录

Using logging in multiple modules

提问by Quest Monger

I have a small python project that has the following structure -

我有一个具有以下结构的小型 python 项目 -

Project 
 -- pkg01
   -- test01.py
 -- pkg02
   -- test02.py
 -- logging.conf

I plan to use the default logging module to print messages to stdout and a log file. To use the logging module, some initialization is required -

我计划使用默认的日志记录模块将消息打印到标准输出和日志文件。要使用日志记录模块，需要进行一些初始化 -

import logging.config

logging.config.fileConfig('logging.conf')
logger = logging.getLogger('pyApp')

logger.info('testing')

At present, I perform this initialization in every module before I start logging messages. Is it possible to perform this initialization only once in one place such that the same settings are reused by logging all over the project?

目前，我在开始记录消息之前在每个模块中执行此初始化。是否可以仅在一处执行此初始化，以便通过在整个项目中记录来重复使用相同的设置？

采纳答案by Vinay Sajip

Best practice is, in each module, to have a logger defined like this:

最佳实践是，在每个模块中，定义一个记录器，如下所示：

import logging
logger = logging.getLogger(__name__)

near the top of the module, and then in other code in the module do e.g.

在模块顶部附近，然后在模块中的其他代码中执行例如

logger.debug('My message with %s', 'variable data')

If you need to subdivide logging activity inside a module, use e.g.

如果您需要细分模块内的日志记录活动，请使用例如

loggerA = logging.getLogger(__name__ + '.A')
loggerB = logging.getLogger(__name__ + '.B')

and log to loggerAand loggerBas appropriate.

并根据需要登录loggerA和loggerB。

In your main program or programs, do e.g.:

在您的主程序或程序中，执行例如：

def main():
    "your program code"

if __name__ == '__main__':
    import logging.config
    logging.config.fileConfig('/path/to/logging.conf')
    main()

or

或者

def main():
    import logging.config
    logging.config.fileConfig('/path/to/logging.conf')
    # your program code

if __name__ == '__main__':
    main()

See herefor logging from multiple modules, and herefor logging configuration for code which will be used as a library module by other code.

请参阅此处了解来自多个模块的日志记录，以及此处了解将被其他代码用作库模块的代码的日志记录配置。

Update:When calling fileConfig(), you may want to specify disable_existing_loggers=Falseif you're using Python 2.6 or later (see the docsfor more information). The default value is Truefor backward compatibility, which causes all existing loggers to be disabled by fileConfig()unless they or their ancestor are explicitly named in the configuration. With the value set to False, existing loggers are left alone. If using Python 2.7/Python 3.2 or later, you may wish to consider the dictConfig()API which is better than fileConfig()as it gives more control over the configuration.

更新：调用 时fileConfig()，您可能需要指定disable_existing_loggers=False是否使用 Python 2.6 或更高版本（有关更多信息，请参阅文档）。默认值是True为了向后兼容，这会导致所有现有记录器被禁用，fileConfig()除非它们或其祖先在配置中明确命名。将值设置为 时False，现有记录器将保持不变。如果使用 Python 2.7/Python 3.2 或更高版本，您可能希望考虑dictConfig()比fileConfig()它更好的API，因为它可以更好地控制配置。

回答by Yarkee

I always do it as below.

我总是这样做如下。

Use a single python file to config my log as singleton pattern which named 'log_conf.py'

使用单个 python 文件将我的日志配置为名为 ' log_conf.py' 的单例模式

#-*-coding:utf-8-*-

import logging.config

def singleton(cls):
    instances = {}
    def get_instance():
        if cls not in instances:
            instances[cls] = cls()
        return instances[cls]
    return get_instance()

@singleton
class Logger():
    def __init__(self):
        logging.config.fileConfig('logging.conf')
        self.logr = logging.getLogger('root')

In another module, just import the config.

在另一个模块中，只需导入配置。

from log_conf import Logger

Logger.logr.info("Hello World")

This is a singleton pattern to log, simply and efficiently.

这是一种单例模式，可以简单有效地记录日志。

回答by Stan Prokop

Actually every logger is a child of the parent's package logger (i.e. package.subpackage.moduleinherits configuration from package.subpackage), so all you need to do is just to configure the root logger. This can be achieved by logging.config.fileConfig(your own config for loggers) or logging.basicConfig(sets the root logger). Setup logging in your entry module (__main__.pyor whatever you want to run, for example main_script.py. __init__.pyworks as well)

实际上，每个记录器都是父包记录器的子项（即package.subpackage.module从 继承配置package.subpackage)，因此您需要做的只是配置根记录器。这可以通过logging.config.fileConfig（您自己的记录器配置）或logging.basicConfig（设置根记录器）来实现. 在您的入口模块中设置日志记录（__main__.py或您想要运行的任何内容，例如main_script.py.__init__.py也可以）

using basicConfig:

使用基本配置：

# package/__main__.py
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)

using fileConfig:

使用文件配置：

# package/__main__.py
import logging
import logging.config

logging.config.fileConfig('logging.conf')

and then create every logger using:

然后使用以下命令创建每个记录器：

# package/submodule.py
# or
# package/subpackage/submodule.py
import logging
log = logging.getLogger(__name__)

log.info("Hello logging!")

For more information see Advanced Logging Tutorial.

有关更多信息，请参阅高级日志记录教程。

回答by deeshank

You could also come up with something like this!

你也可以想出这样的东西！

def get_logger(name=None):
    default = "__app__"
    formatter = logging.Formatter('%(levelname)s: %(asctime)s %(funcName)s(%(lineno)d) -- %(message)s',
                              datefmt='%Y-%m-%d %H:%M:%S')
    log_map = {"__app__": "app.log", "__basic_log__": "file1.log", "__advance_log__": "file2.log"}
    if name:
        logger = logging.getLogger(name)
    else:
        logger = logging.getLogger(default)
    fh = logging.FileHandler(log_map[name])
    fh.setFormatter(formatter)
    logger.addHandler(fh)
    logger.setLevel(logging.DEBUG)
    return logger

Now you could use multiple loggers in same module and across whole project if the above is defined in a separate module and imported in other modules were logging is required.

现在你可以在同一个模块和整个项目中使用多个记录器，如果上面是在一个单独的模块中定义的，并在其他模块中导入，则需要日志记录。

a=get_logger("__app___")
b=get_logger("__basic_log__")
a.info("Starting logging!")
b.debug("Debug Mode")

回答by deeshank

@Yarkee's solution seemed better. I would like to add somemore to it -

@Yarkee 的解决方案似乎更好。我想补充一些 -

class Singleton(type):
    _instances = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances.keys():
            cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
        return cls._instances[cls]


class LoggerManager(object):
    __metaclass__ = Singleton

    _loggers = {}

    def __init__(self, *args, **kwargs):
        pass

    @staticmethod
    def getLogger(name=None):
        if not name:
            logging.basicConfig()
            return logging.getLogger()
        elif name not in LoggerManager._loggers.keys():
            logging.basicConfig()
            LoggerManager._loggers[name] = logging.getLogger(str(name))
        return LoggerManager._loggers[name]    


log=LoggerManager().getLogger("Hello")
log.setLevel(level=logging.DEBUG)

So LoggerManager can be a pluggable to the entire application. Hope it makes sense and value.

所以 LoggerManager 可以插入整个应用程序。希望它有意义和有价值。

回答by Tommy

Throwing in another solution.

扔在另一个解决方案。

In my module's init.py I have something like:

在我的模块的init.py 中，我有类似的内容：

# mymodule/__init__.py
import logging

def get_module_logger(mod_name):
  logger = logging.getLogger(mod_name)
  handler = logging.StreamHandler()
  formatter = logging.Formatter(
        '%(asctime)s %(name)-12s %(levelname)-8s %(message)s')
  handler.setFormatter(formatter)
  logger.addHandler(handler)
  logger.setLevel(logging.DEBUG)
  return logger

Then in each module I need a logger, I do:

然后在每个模块中我需要一个记录器，我这样做：

# mymodule/foo.py
from [modname] import get_module_logger
logger = get_module_logger(__name__)

When the logs are missed, you can differentiate their source by the module they came from.

当日志丢失时，您可以通过它们来自的模块区分它们的来源。

回答by phil_20686

Several of these answers suggest that at the top of a module you you do

其中一些答案表明，在模块的顶部，您可以

import logging
logger = logging.getLogger(__name__)

It is my understanding that this is considered very bad practice. The reason is that the file config will disable all existing loggers by default. E.g.

据我了解，这被认为是非常糟糕的做法。原因是文件配置将默认禁用所有现有记录器。例如

#my_module
import logging

logger = logging.getLogger(__name__)

def foo():
    logger.info('Hi, foo')

class Bar(object):
    def bar(self):
        logger.info('Hi, bar')

And in your main module :

在你的主模块中：

#main
import logging

# load my module - this now configures the logger
import my_module

# This will now disable the logger in my module by default, [see the docs][1] 
logging.config.fileConfig('logging.ini')

my_module.foo()
bar = my_module.Bar()
bar.bar()

Now the log specified in logging.ini will be empty, as the existing logger was disabled by fileconfig call.

现在 logging.ini 中指定的日志将为空，因为现有的记录器已被 fileconfig 调用禁用。

While is is certainly possible to get around this (disable_existing_Loggers=False), realistically many clients of your library will not know about this behavior, and will not receive your logs. Make it easy for your clients by always calling logging.getLogger locally. Hat Tip : I learned about this behavior from Victor Lin's Website.

虽然当然可以解决这个问题（disable_existing_Loggers=False），但实际上您图书馆的许多客户不会知道这种行为，也不会收到您的日志。通过始终在本地调用 logging.getLogger 使您的客户更容易。帽子提示：我从Victor Lin 的网站上了解到这种行为。

So good practice is instead to always call logging.getLogger locally. E.g.

因此，好的做法是始终在本地调用 logging.getLogger。例如

#my_module
import logging

logger = logging.getLogger(__name__)

def foo():
    logging.getLogger(__name__).info('Hi, foo')

class Bar(object):
    def bar(self):
        logging.getLogger(__name__).info('Hi, bar')    

Also, if you use fileconfig in your main, set disable_existing_loggers=False, just in case your library designers use module level logger instances.

此外，如果您在 main 中使用 fileconfig，请设置 disable_existing_loggers=False，以防万一您的库设计者使用模块级记录器实例。

回答by Ben Yitzhaki

There are several answers. i ended up with a similar yet different solution that makes sense to me, maybe it will make sense to you as well. My main objective was to be able to pass logs to handlers by their level (debug level logs to the console, warnings and above to files):

有几个答案。我最终得到了一个对我有意义的类似但不同的解决方案，也许它对你也有意义。我的主要目标是能够按级别将日志传递给处理程序（调试级别日志到控制台，警告和以上到文件）：

from flask import Flask
import logging
from logging.handlers import RotatingFileHandler

app = Flask(__name__)

# make default logger output everything to the console
logging.basicConfig(level=logging.DEBUG)

rotating_file_handler = RotatingFileHandler(filename="logs.log")
rotating_file_handler.setLevel(logging.INFO)

app.logger.addHandler(rotating_file_handler)

created a nice util file named logger.py:

创建了一个名为 logger.py 的不错的 util 文件：

import logging

def get_logger(name):
    return logging.getLogger("flask.app." + name)

the flask.app is a hardcoded value in flask. the application logger is always starting with flask.app as its the module's name.

flask.app 是烧瓶中的硬编码值。应用程序记录器总是以 flask.app 作为模块的名称开头。

now, in each module, i'm able to use it in the following mode:

现在，在每个模块中，我可以在以下模式下使用它：

from logger import get_logger
logger = get_logger(__name__)

logger.info("new log")

This will create a new log for "app.flask.MODULE_NAME" with minimum effort.

这将以最小的努力为“app.flask.MODULE_NAME”创建一个新日志。

回答by Mousam Singh

The best practice would be to create a module separately which has only one method whose task we be to give a logger handler to the the calling method. Save this file as m_logger.py

最佳实践是单独创建一个模块，该模块只有一个方法，我们的任务是为调用方法提供记录器处理程序。将此文件另存为 m_logger.py

import logger, logging

def getlogger():
    # logger
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.DEBUG)
    # create console handler and set level to debug
    #ch = logging.StreamHandler()
    ch = logging.FileHandler(r'log.txt')
    ch.setLevel(logging.DEBUG)
    # create formatter
    formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
    # add formatter to ch
    ch.setFormatter(formatter)
    # add ch to logger
    logger.addHandler(ch)
    return logger

Now call the getlogger() method whenever logger handler is needed.

现在，只要需要记录器处理程序，就调用 getlogger() 方法。

from m_logger import getlogger
logger = getlogger()
logger.info('My mssg')

回答by npjohns

New to python so I don't know if this is advisable, but it works great for not re-writing boilerplate.

python的新手所以我不知道这是否可取，但它非常适合不重写样板。

Your project must have an init.py so it can be loaded as a module

您的项目必须有一个init.py 以便它可以作为模块加载

# Put this in your module's __init__.py
import logging.config
import sys

# I used this dictionary test, you would put:
# logging.config.fileConfig('logging.conf')
# The "" entry in loggers is the root logger, tutorials always 
# use "root" but I can't get that to work
logging.config.dictConfig({
    "version": 1,
    "formatters": {
        "default": {
            "format": "%(asctime)s %(levelname)s %(name)s %(message)s"
        },
    },
    "handlers": {
        "console": {
            "level": 'DEBUG',
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stdout"
        }
    },
    "loggers": {
        "": {
            "level": "DEBUG",
            "handlers": ["console"]
        }
    }
})

def logger():
    # Get the name from the caller of this function
    return logging.getLogger(sys._getframe(1).f_globals['__name__'])

sys._getframe(1)suggestion comes from here

sys._getframe(1)建议来自这里

Then to use your logger in any other file:

然后在任何其他文件中使用您的记录器：

from [your module name here] import logger

logger().debug("FOOOOOOOOO!!!")

Caveats:

注意事项：

1. You must run your files as modules, otherwise import [your module]won't work:
   • python -m [your module name].[your filename without .py]
2. The name of the logger for the entry point of your program will be __main__, but any solution using __name__will have that issue.

1. 您必须将文件作为模块运行，否则import [your module]将无法工作：
   • python -m [your module name].[your filename without .py]
2. 程序入口点的记录器名称将是__main__，但任何使用的解决方案__name__都会有这个问题。