Python 如何导入所有子模块？

How to import all submodules?

提问by linkmaster03

I have a directory structure as follows:

我有一个目录结构如下：

| main.py
| scripts
|--| __init__.py
   | script1.py
   | script2.py
   | script3.py

From main.py, the module scriptsis imported. I tried using pkgutils.walk_packagesin combination with __all__, but using that, I can only import all the submodules directly under mainusing from scripts import *. I would like to get them all under scripts. What would be the cleanest way to import all the submodules of scriptsso that I could access scripts.script1from main?

从main.py，scripts导入模块。我尝试pkgutils.walk_packages结合使用 with __all__，但是使用它，我只能直接在mainusing下导入所有子模块from scripts import *。我想把它们都放在下面scripts。什么是导入所有的子模块的干净的方式scripts，这样我可以访问scripts.script1从main？

EDIT: I am sorry that I was a bit vague. I would like to import the submodules on run-time without specifying them explicitly in __init__.py. I can use pkgutils.walk_packagesto get the submodule names (unless someone knows of a better way), but I am not sure of the cleanest way to use these names (or maybe the ImpImporters that walk_packagesreturns?) to import them.

编辑：对不起，我有点含糊。我想在运行时导入子模块而不在__init__.py. 我可以pkgutils.walk_packages用来获取子模块名称（除非有人知道更好的方法），但我不确定使用这些名称（或者可能是walk_packages返回的 ImpImporters 的最简洁方法？）来导入它们。

采纳答案by Joe Kington

Edit:Here's one way to recursively import everything at runtime...

编辑：这是在运行时递归导入所有内容的一种方法......

(Contents of __init__.pyin top package directory)

（__init__.py顶级包目录中的内容）

import pkgutil

__all__ = []
for loader, module_name, is_pkg in  pkgutil.walk_packages(__path__):
    __all__.append(module_name)
    _module = loader.find_module(module_name).load_module(module_name)
    globals()[module_name] = _module

I'm not using __import__(__path__+'.'+module_name)here, as it's difficult to properly recursively import packages using it. If you don't have nested sub-packages, and wanted to avoid using globals()[module_name], though, it's one way to do it.

我没有__import__(__path__+'.'+module_name)在这里使用，因为使用它很难正确递归地导入包。如果您没有嵌套的子包，并且想避免使用globals()[module_name]，那么这是一种方法。

There's probably a better way, but this is the best I can do, anyway.

可能有更好的方法，但无论如何，这是我能做的最好的方法。

Original Answer(For context, ignore othwerwise. I misunderstood the question initially):

原始答案（对于上下文，忽略其他。我最初误解了这个问题）：

What does your scripts/__init__.pylook like? It should be something like:

你scripts/__init__.py看起来像什么？它应该是这样的：

import script1
import script2
import script3
__all__ = ['script1', 'script2', 'script3']

You could even do without defining __all__, but things (pydoc, if nothing else) will work more cleanly if you define it, even if it's just a list of what you imported.

你甚至可以不定义__all__，但是如果你定义它，事情（pydoc，如果没有别的）会更干净地工作，即使它只是你导入的列表。

回答by physicsmichael

I was writing a small personal library and adding new modules all the time so I wrote a shell script to look for scripts and create the __init__.py's. The script is executed just outside of the main directory for my package, pylux.

我一直在编写一个小型个人库并添加新模块，因此我编写了一个 shell 脚本来查找脚本并创建__init__.py's. 该脚本在我的包 pylux 的主目录之外执行。

I know it probably isn't the answer you're looking for, but it servered its purpose for me and it might be useful to someone else, too.

我知道这可能不是您正在寻找的答案，但它为我服务了它的目的，它也可能对其他人有用。

#!/bin/bash

echo 'Traversing folder hierarchy...'

CWD=`pwd`


for directory in `find pylux -type d -exec echo {} \;`;
do
    cd $directory
    #echo Entering $directory
    echo -n "" > __init__.py

    for subdirectory in `find . -type d -maxdepth 1 -mindepth 1`;
    do
        subdirectory=`echo $subdirectory | cut -b 3-`
        #echo -n '    ' ...$subdirectory
        #echo -e '\t->\t' import $subdirectory
        echo import $subdirectory >> __init__.py
    done

    for pyfile in *.py ;
    do
        if [ $pyfile = $(echo __init__.py) ]; then
            continue
        fi
        #echo -n '    ' ...$pyfile
        #echo -e '\t->\t' import `echo $pyfile | cut -d . -f 1`
        echo import `echo $pyfile | cut -d . -f 1` >> __init__.py
    done
    cd $CWD

done


for directory in `find pylux -type d -exec echo {} \;`;
do
    echo $directory/__init__.py:
    cat $directory/__init__.py | awk '{ print "\t"
__all__ = ["I will get rewritten"]
# Don't modify the line above, or this line!
import automodinit
automodinit.automodinit(__name__, __file__, globals())
del automodinit
# Anything else you want can go after here, it won't get modified.
 }'
done

回答by Niall Douglas

I got tired of this problem myself, so I wrote a package called automodinit to fix it. You can get it from http://pypi.python.org/pypi/automodinit/. Usage is like this:

我自己也厌倦了这个问题，所以我写了一个名为 automodinit 的包来修复它。您可以从http://pypi.python.org/pypi/automodinit/获取它。用法是这样的：

1. Include the automodinit package into your setup.pydependencies.
2. Add the following to the beginning of the __init__.pyfile:

1. 将 automodinit 包包含到您的setup.py依赖项中。
2. 将以下内容添加到__init__.py文件的开头：

for x in __all__: import x

That's it! From now on importing a module will set __all__to a list of .py[co] files in the module and will also import each of those files as though you had typed:

就是这样！从现在开始，导入模块将设置__all__为模块中的 .py[co] 文件列表，并且还将导入每个文件，就像您输入的一样：

from pkgutil import walk_packages
from os import path

__all__ = []
__pkg_prefix = "%s." % __name__
__pkg_path = path.abspath(__path__[0]).rsplit("/", 1)[0] #parent directory

for loader, modname, _ in walk_packages([__pkg_path]):
    if modname.startswith(__pkg_prefix):
        #load the module / package
        module = loader.find_module(modname).load_module(modname)
        modname = modname[len(__pkg_prefix):] #strip package prefix from name
        #append all toplevel modules and packages to __all__
        if not "." in modname:
            __all__.append(modname)
            globals()[modname] = module
        #set everything else as an attribute of their parent package
        else:
            #get the toplevel package from globals()
            pkg_name, rest = modname.split(".", 1)
            pkg = globals()[pkg_name]
            #recursively get the modules parent package via getattr
            while "." in rest:
                subpkg, rest = rest.split(".", 1)
                pkg = getattr(pkg, subpkg)
            #set the module (or package) as an attribute of its parent package
            setattr(pkg, rest, module)

Therefore the effect of from M import *matches exactly import M.

因此效果from M import *完全匹配import M。

automodinit is happy running from inside ZIP archives and is therefore ZIP safe.

automodinit 很高兴从 ZIP 档案中运行，因此是 ZIP 安全的。

回答by l4mpi

I've played around with Joe Kington's Answerand have built a solution that uses globalsand get/setattrand thus doesn't need eval. A slight modification is that instead of directly using the packages __path__for walk_packages, I use the packages parent directory and then only import modules starting with __name__ + ".". This was done to reliably get all subpackages from walk_packages- in my use case I had a subpackage named testwhich caused pkgutil to iterate over the testpackage from python's library; furthermore, using __path__would not recurse into the packages subdirectories. All these issues were observed using jython and python2.5, the code below is only tested in jython thus far.

我打得四处乔金顿的回答，并建立了一个解决方案，它的用途globals，并get/setattr因而不需要EVAL。一个稍微的改变是，而不是直接使用包__path__的walk_packages，我用的是包父目录，然后开始只导入模块__name__ + "."。这样做是为了可靠地从中获取所有子包walk_packages- 在我的用例中，我有一个名为的子包test，它导致 pkgutiltest从 python 库中迭代包；此外， using__path__不会递归到包子目录中。所有这些问题都是使用 jython 和 python2.5 观察到的，以下代码目前仅在 jython 中进行了测试。

Also note that OPs question only talks about importing all modulesfrom a package, this code recursively imports all packages too.

另请注意，OP 问题仅讨论从包中导入所有模块，此代码也递归导入所有包。

from glob import iglob
from os.path import basename, relpath, sep, splitext

def import_submodules(__path__to_here):
    """Imports all submodules.
    Import this function in __init__.py and put this line to it:
    __all__ = import_submodules(__path__)"""
    result = []
    for smfile in iglob(relpath(__path__to_here[0]) + "/*.py"):
        submodule = splitext(basename(smfile))[0]
        importstr = ".".join(smfile.split(sep)[:-1])
        if not submodule.startswith("_"):
            __import__(importstr + "." + submodule)
            result.append(submodule)
    return result

As a future improvement I'll try to make this dynamic with a __getattr__hook on the package, so the actual modules are only imported when they are accessed...

作为未来的改进，我将尝试使用__getattr__包上的挂钩使其动态化，因此实际模块仅在访问时才导入...

回答by SzieberthAdam

This works nicely for me in Python 3.3. Note that this works only for submodules which are in files in the same directory as the __init__.py. With some work however it can be enhanced for supporting submodules in directories too.

这在 Python 3.3 中对我来说很好用。请注意，这仅适用于与__init__.py. 但是，通过一些工作，它也可以增强以支持目录中的子模块。

def import_submodules(package_name):
    """ Import all submodules of a module, recursively

    :param package_name: Package name
    :type package_name: str
    :rtype: dict[types.ModuleType]
    """
    package = sys.modules[package_name]
    return {
        name: importlib.import_module(package_name + '.' + name)
        for loader, name, is_pkg in pkgutil.walk_packages(package.__path__)
    }

回答by kolypto

Simply works, and allows relative import inside packages:

简单有效，并允许在包内进行相对导入：

__all__ = import_submodules(__name__).keys()

Usage:

用法：

import importlib
import pkgutil


def import_submodules(package, recursive=True):
    """ Import all submodules of a module, recursively, including subpackages

    :param package: package (name or actual module)
    :type package: str | module
    :rtype: dict[str, types.ModuleType]
    """
    if isinstance(package, str):
        package = importlib.import_module(package)
    results = {}
    for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
        full_name = package.__name__ + '.' + name
        results[full_name] = importlib.import_module(full_name)
        if recursive and is_pkg:
            results.update(import_submodules(full_name))
    return results

回答by Mr. B

This is based on the answer that kolypto provided, but his answer does not perform recursive import of packages, whereas this does. Although not required by the main question, I believe recursive import applies and can be very useful in many similar situations. I, for one, found this question when searching on the topic.

这是基于kolypto 提供的答案，但他的答案不执行包的递归导入，而这样做。尽管主要问题不需要，但我相信递归导入适用并且在许多类似情况下非常有用。首先，我在搜索该主题时发现了这个问题。

This is a nice, clean way of performing the import of the subpackage's modules, and should be portable as well, and it uses the standard lib for python 2.7+ / 3.x.

这是执行子包模块导入的一种很好、干净的方式，并且应该是可移植的，并且它使用 python 2.7+ / 3.x 的标准库。

# from main.py, as per the OP's project structure
import scripts
import_submodules(scripts)

# Alternatively, from scripts.__init__.py
import_submodules(__name__)

Usage:

用法：

| pkg
|--| __init__.py
   | main.py
   | scripts
   |--| __init__.py
      | script1.py
      | script2.py
      | script3.py

回答by user2561747

Not nearly as clean as I would like, but none of the cleaner methods worked for me. This achieves the specified behaviour:

不像我想要的那么干净，但没有一种更清洁的方法对我有用。这实现了指定的行为：

Directory structure:

目录结构：

import importlib as _importlib
import pkgutil as _pkgutil
__all__ = [_mod[1].split(".")[-1] for _mod in
           filter(lambda _mod: _mod[1].count(".") == 1 and not 
                               _mod[2] and __name__ in _mod[1],
                  [_mod for _mod in _pkgutil.walk_packages("." + __name__)])]
__sub_mods__ = [".".join(_mod[1].split(".")[1:]) for _mod in
                filter(lambda _mod: _mod[1].count(".") > 1 and not 
                                    _mod[2] and __name__ in _mod[1],
                       [_mod for _mod in 
                        _pkgutil.walk_packages("." + __name__)])]
from . import *
for _module in __sub_mods__:
    _importlib.import_module("." + _module, package=__name__)

Where pkg/scripts/__init__.pyis empty, and pkg/__init__.pycontains:

wherepkg/scripts/__init__.py为空，pkg/__init__.py包含：

##代码##

Although it's messy, it should be portable. I've used this code for several different packages.

虽然很乱，但应该是便携的。我已经将此代码用于几个不同的包。