使用 Python 3 在 Jupyter Notebook 中使用相对导入从位于另一个目录中的模块导入本地函数

Import local function from a module housed in another directory with relative imports in Jupyter Notebook using Python 3

提问by mpacer

I have a directory structure similar to the following

我有一个类似于以下的目录结构

meta_project
    project1
        __init__.py
        lib
            module.py
            __init__.py
    notebook_folder
        notebook.jpynb

When working in notebook.jpynbif I try to use a relative import to access a function function()in module.pywith:

当在工作notebook.jpynb，如果我尝试使用相对导入来访问函数function()在module.py有：

from ..project1.lib.module import function

I get the following error:

我收到以下错误：

SystemError                               Traceback (most recent call last)
<ipython-input-7-6393744d93ab> in <module>()
----> 1 from ..project1.lib.module import function

SystemError: Parent module '' not loaded, cannot perform relative import

Is there any way to get this to work using relative imports?

有什么办法可以使用相对导入来让它工作吗？

Note, the notebook server is instantiated at the level of the meta_projectdirectory, so it should have access to the information in those files.

请注意，笔记本服务器是在meta_project目录级别实例化的，因此它应该可以访问这些文件中的信息。

Note, also, that at least as originally intended project1wasn't thought of as a module and therefore does not have an __init__.pyfile, it was just meant as a file-system directory. If the solution to the problem requires treating it as a module and including an __init__.pyfile (even a blank one) that is fine, but doing so is not enough to solve the problem.

另请注意，至少在最初的预期project1中不被视为模块，因此没有__init__.py文件，它只是作为文件系统目录。如果问题的解决方案需要将其视为一个模块并包含一个__init__.py文件（甚至是一个空白文件），那很好，但这样做还不足以解决问题。

I share this directory between machines and relative imports allow me to use the same code everywhere, & I often use notebooks for quick prototyping, so suggestions that involve hacking together absolute paths are unlikely to be helpful.

我在机器之间共享这个目录，相对导入允许我在任何地方使用相同的代码，而且我经常使用笔记本进行快速原型设计，因此涉及将绝对路径组合在一起的建议不太可能有帮助。

---

---

Edit: This is unlike Relative imports in Python 3, which talks about relative imports in Python 3 in general and – in particular – running a script from within a package directory. This has to do with working within a jupyter notebook trying to call a function in a local module in another directory which has both different general and particular aspects.

编辑：这与Python 3 中的相对导入不同，后者一般讨论 Python 3 中的相对导入，特别是从包目录中运行脚本。这与在 jupyter notebook 中尝试调用另一个目录中本地模块中的函数有关，该目录具有不同的一般和特定方面。

采纳答案by metakermit

I had almost the same example as you in this notebookwhere I wanted to illustrate the usage of an adjacent module's function in a DRY manner.

在本笔记本中，我有与您几乎相同的示例，我想以 DRY 方式说明相邻模块功能的用法。

My solution was to tell Python of that additional module import path by adding a snippet like this one to the notebook:

我的解决方案是通过在笔记本中添加这样的代码段来告诉 Python 附加模块导入路径：

import os
import sys
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
    sys.path.append(module_path)

This allows you to import the desired function from the module hierarchy:

这允许您从模块层次结构中导入所需的功能：

from project1.lib.module import function
# use the function normally
function(...)

Note that it is necessary to add empty __init__.pyfiles to project1/and lib/folders if you don't have them already.

请注意，如果您还没有空__init__.py文件，则需要将它们添加到project1/和lib/文件夹中。

回答by Joshua Cook

Came here searching for best practices in abstracting code to submodules when working in Notebooks. I'm not sure that there is a best practice. I have been proposing this.

来到这里寻找在 Notebooks 中工作时将代码抽象为子模块的最佳实践。我不确定是否有最佳实践。我一直在提议这个。

A project hierarchy as such:

项目层次结构如下：

├── ipynb
│?? ├── 20170609-Examine_Database_Requirements.ipynb
│?? └── 20170609-Initial_Database_Connection.ipynb
└── lib
    ├── __init__.py
    └── postgres.py

And from 20170609-Initial_Database_Connection.ipynb:

来自20170609-Initial_Database_Connection.ipynb：

    In [1]: cd ..

    In [2]: from lib.postgres import database_connection

This works because by default the Jupyter Notebook can parse the cdcommand. Note that this does not make use of Python Notebook magic. It simply works without prepending %bash.

这是有效的，因为默认情况下 Jupyter Notebook 可以解析cd命令。请注意，这不会使用 Python Notebook 魔法。它只是在没有预先设置的情况下工作%bash。

Considering that 99 times out of a 100 I am working in Docker using one of the Project Jupyter Docker images, the following modification isidempotent

考虑到 100 次中有 99 次我使用Project Jupyter Docker 映像之一在 Docker 中工作，以下修改是幂等的

    In [1]: cd /home/jovyan

    In [2]: from lib.postgres import database_connection

回答by Gerges

So far, the accepted answer has worked best for me. However, my concern has always been that there is a likely scenario where I might refactor the notebooksdirectory into subdirectories, requiring to change the module_pathin every notebook. I decided to add a python file within each notebook directory to import the required modules.

到目前为止，公认的答案对我来说效果最好。但是，我一直担心有一种可能的情况，我可能会将notebooks目录重构为子目录，需要更改module_path每个笔记本中的 。我决定在每个 notebook 目录中添加一个 python 文件来导入所需的模块。

Thus, having the following project structure:

因此，具有以下项目结构：

project
|__notebooks
   |__explore
      |__ notebook1.ipynb
      |__ notebook2.ipynb
      |__ project_path.py
   |__ explain
       |__notebook1.ipynb
       |__project_path.py
|__lib
   |__ __init__.py
   |__ module.py

I added the file project_path.pyin each notebook subdirectory (notebooks/exploreand notebooks/explain). This file contains the code for relative imports (from @metakermit):

我project_path.py在每个笔记本子目录（notebooks/explore和notebooks/explain）中添加了该文件。此文件包含相对导入的代码（来自 @metakermit）：

import sys
import os

module_path = os.path.abspath(os.path.join(os.pardir, os.pardir))
if module_path not in sys.path:
    sys.path.append(module_path)

This way, I just need to do relative imports within the project_path.pyfile, and not in the notebooks. The notebooks files would then just need to import project_pathbefore importing lib. For example in 0.0-notebook.ipynb:

这样，我只需要在project_path.py文件中进行相对导入，而不是在笔记本中。笔记本文件只需要在导入project_path之前导入lib。例如在0.0-notebook.ipynb：

import project_path
import lib

The caveat here is that reversing the imports would not work. THIS DOES NOT WORK:

这里需要注意的是，反向导入是行不通的。这不起作用：

import lib
import project_path

Thus care must be taken during imports.

因此，在进口过程中必须小心。

回答by marr75

Researching this topic myself and having read the answers I recommend using the path.py librarysince it provides a context manager for changing the current working directory.

自己研究这个主题并阅读了我推荐使用path.py 库的答案，因为它提供了一个用于更改当前工作目录的上下文管理器。

You then have something like

然后你有类似的东西

import path
if path.Path('../lib').isdir():
    with path.Path('..'):
        import lib

Although, you might just omit the isdirstatement.

虽然，您可能只是省略了该isdir语句。

Here I'll add print statements to make it easy to follow what's happening

在这里，我将添加打印语句，以便轻松了解正在发生的事情

import path
import pandas

print(path.Path.getcwd())
print(path.Path('../lib').isdir())
if path.Path('../lib').isdir():
    with path.Path('..'):
        print(path.Path.getcwd())
        import lib
        print('Success!')
print(path.Path.getcwd())

which outputs in this example (where lib is at /home/jovyan/shared/notebooks/by-team/data-vis/demos/lib):

本示例中的输出（其中 lib 位于/home/jovyan/shared/notebooks/by-team/data-vis/demos/lib）：

/home/jovyan/shared/notebooks/by-team/data-vis/demos/custom-chart
/home/jovyan/shared/notebooks/by-team/data-vis/demos
/home/jovyan/shared/notebooks/by-team/data-vis/demos/custom-chart

Since the solution uses a context manager, you are guaranteed to go back to your previous working directory, no matter what state your kernel was in before the cell and no matter what exceptions are thrown by importing your library code.

由于该解决方案使用上下文管理器，因此无论您的内核在单元格之前处于什么状态，也无论导入库代码引发什么异常，您都可以保证返回到之前的工作目录。

回答by Victor Callejas

I have just found this pretty solution:

我刚刚找到了这个漂亮的解决方案：

import sys; sys.path.insert(0, '..') # add parent folder path where lib folder is
import lib.store_load # store_load is a file on my library folder

You just want some functions of that file

你只想要那个文件的一些功能

from lib.store_load import your_function_name

If python version >= 3.3 you do not need init.py file in the folder

如果 python 版本 >= 3.3 你不需要文件夹中的 init.py 文件

回答by Bobby

Here's my 2 cents:

这是我的 2 美分：

import sys

导入系统

map the path where the module file is located. In my case it was the desktop

映射模块文件所在的路径。就我而言，它是桌面

sys.path.append('/Users/John/Desktop')

sys.path.append('/Users/John/Desktop')

Either import the whole mapping module BUT then you have to use the .notation to map the classes like mapping.Shipping()

要么导入整个映射模块，然后你必须使用 .notation 来映射类，比如 mapping.Shipping()

import mapping #mapping.py is the name of my module file

导入映射 #mapping.py 是我的模块文件的名称

shipit = mapping.Shipment() #Shipment is the name of the class I need to use in the mapping module

shipit = mapping.Shipment() #Shipment 是我需要在映射模块中使用的类的名称

Or import the specific class from the mapping module

或者从映射模块导入特定的类

from mapping import Mapping

从映射导入映射

shipit = Shipment() #Now you don't have to use the .notation

shipit = Shipment() #现在你不必使用.notation

回答by t.perk

I have found that python-dotenvhelps solve this issue pretty effectively. Your project structure ends up changing slightly, but the code in your notebook is a bit simpler and consistent across notebooks.

我发现python-dotenv可以非常有效地解决这个问题。您的项目结构最终会略有变化，但笔记本中的代码在笔记本之间更简单且一致。

For your project, do a little install.

对于您的项目，请进行一些安装。

pipenv install python-dotenv

Then, project changes to:

然后，项目更改为：

├── .env (this can be empty)
├── ipynb
│   ├── 20170609-Examine_Database_Requirements.ipynb
│   └── 20170609-Initial_Database_Connection.ipynb
└── lib
    ├── __init__.py
    └── postgres.py

And finally, your import changes to:

最后，您的导入更改为：

import os
import sys

from dotenv import find_dotenv


sys.path.append(os.path.dirname(find_dotenv()))

A +1 for this package is that your notebooks can be several directories deep. python-dotenv will find the closest one in a parent directory and use it. A +2 for this approach is that jupyter will load environment variables from the .env file on startup. Double whammy.

这个包的 +1 是你的笔记本可以有几个目录深。python-dotenv 将在父目录中找到最接近的一个并使用它。这种方法的 +2 是 jupyter 将在启动时从 .env 文件加载环境变量。双重打击。