Python 为缺少依赖声明的包（如 scipy）构建轮子

Build wheel for a package (like scipy) lacking dependency declaration

提问by Midnighter

I think it doesn't make a difference here but I'm using Python 2.7.

我认为这里没有什么区别，但我使用的是 Python 2.7。

So the general part of my question is the following: I use a separate virtualenvfor each of my projects. I don't have administrator access and I don't want to mess with system-installed packages anyway. Naturally, I want to use wheels to speed up package upgrades and installations across the virtualenvs. How can I build a wheel whose dependencies are only met within a specific virtualenv?

所以我的问题的一般部分如下：我virtualenv为我的每个项目使用一个单独的。我没有管理员访问权限，无论如何我不想弄乱系统安装的软件包。自然，我想使用轮子来加速跨virtualenvs 的包升级和安装。如何构建仅在特定范围内满足依赖关系的轮子virtualenv？

Specifically, issuing

具体来说，发行

pip wheel -w $WHEELHOUSE scipy

fails with

失败

Building wheels for collected packages: scipy
  Running setup.py bdist_wheel for scipy
  Destination directory: /home/moritz/.pip/wheelhouse
  Complete output from command /home/moritz/.virtualenvs/base/bin/python -c "import setuptools;__file__='/home/moritz/.virtualenvs/base/build/scipy/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /home/moritz/.pip/wheelhouse:
  Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/home/moritz/.virtualenvs/base/build/scipy/setup.py", line 237, in <module>

    setup_package()

  File "/home/moritz/.virtualenvs/base/build/scipy/setup.py", line 225, in setup_package

    from numpy.distutils.core import setup

ImportError: No module named numpy.distutils.core

----------------------------------------
  Failed building wheel for scipy
Failed to build scipy
Cleaning up...

because numpyis not globally present and while building the wheel works when a virtualenvwith numpyinstalled is active, it seems like a terrible idea to have the wheel depend on a specific virtualenv's version of numpy.

因为numpy它不是全局存在的，并且当 a virtualenvwith numpyinstalled 处于活动状态时构建轮子可以工作，让轮子依赖于特定virtualenv版本的numpy.

pandaswhich also depends on numpyappears to install its own components of numpybut I'm not sure that's the best solution.

pandas这也取决于numpy似乎安装自己的组件，numpy但我不确定这是最好的解决方案。

I could install numpywith --userand use that to build the scipywheel. Are there better options?

我可以安装numpy使用--user，并用它来建立scipy轮。有更好的选择吗？

采纳答案by Jan Vlcinsky

Problem description

问题描述

• Have a python package (like scipy), which is dependent on other packages (like numpy) but setup.pyis not declaring that requirement/dependency.
• Building a wheel for such a package will succeed in case, current environment provides the package(s) which are needed.
• In case, required packages are not available, building a wheel will fail.

• 有一个 python 包（如scipy），它依赖于其他包（如numpy）但setup.py没有声明该要求/依赖项。
• 如果当前环境提供了所需的包，则为此类包构建轮子将成功。
• 如果所需的包不可用，构建轮子将失败。

Note: Ideal solution is to correct the broken setup.pyby adding there required package declaration. But this is mostly not feasible and we have to go another way around.

注意：理想的解决方案是setup.py通过添加所需的包声明来纠正损坏。但这大多是不可行的，我们必须另辟蹊径。

Solution: Install required packages first

解决方法：先安装需要的包

The procedure (for installing scipywhich requires numpy) has two steps

该过程（用于安装scipy需要numpy）有两个步骤

1. build the wheels
2. use the wheels to install the package you need

1. 造轮子
2. 使用轮子安装您需要的包

Populate wheelhouse with wheels you need

用您需要的轮子填充驾驶室

This has to be done only once and can be then reused many times.

这必须只做一次，然后可以重复使用多次。

1. have properly configured pip configuration so that installation from wheels is allowed, wheelhouse directory is set up and overlaps with download-cacheand find-linksas in following example of pip.conf:

   [global]
   download-cache = /home/javl/.pip/cache
   find-links = /home/javl/.pip/packages
   
   [install]
   use-wheel = yes
   
   [wheel]
   wheel-dir = /home/javl/.pip/packages
   

2. install all required system libraries for all the packages, which have to be compiled

3. build a wheel for required package (numpy)

   $ pip wheel numpy
   

4. set up virtualenv (needed only once), activate it and install there numpy:

   $ pip install numpy
   

   As a wheel is ready, it shall be quick.

5. build a wheel for scipy(still being in the virtualenv)

   $ pip wheel scipy
   

   By now, you will have your wheelhouse populated with wheels you need.

6. You can remove the temporary virtualenv, it is not needed any more.

1. 有正确配置PIP配置，以便从车轮该安装是允许的，操舵室目录被设置和使用重叠download-cache和find-links如在下面的例子pip.conf：

   [global]
   download-cache = /home/javl/.pip/cache
   find-links = /home/javl/.pip/packages
   
   [install]
   use-wheel = yes
   
   [wheel]
   wheel-dir = /home/javl/.pip/packages
   

2. 为所有必须编译的包安装所有必需的系统库

3. 为所需的包构建一个轮子 ( numpy)

   $ pip wheel numpy
   

4. 设置 virtualenv（只需要一次），激活它并安装在那里numpy：

   $ pip install numpy
   

   当车轮准备好时，它应该很快。

5. 为scipy（仍然在 virtualenv 中）构建一个轮子

   $ pip wheel scipy
   

   现在，您的驾驶室将安装您需要的轮子。

6. 您可以删除临时 virtualenv，不再需要它。

Installing into fresh virtualenv

安装到新的 virtualenv

I am assuming, you have created fresh virtualenv, activated it and wish to have scipyinstalled there.

我假设您已经创建了新的 virtualenv，激活了它并希望scipy安装在那里。

Installing scipyfrom new scipywheel directly would still fail on missing numpy. This we overcome by installing numpyfirst.

直接scipy从新scipy轮子安装仍然会在缺少numpy. 我们通过numpy首先安装来克服这一点。

$ pip install numpy

And then finish with scipy

然后以 scipy 结束

$ pip install scipy

I guess, this could be done in one call (but I did not test it)

我想，这可以在一个电话中完成（但我没有测试）

$ pip install numpy scipy

Repeatedly installing scipyof proven version

反复安装scipy经过验证的版本

It is likely, that at one moment in future, new release of scipyor numpywill be released and pip will attempt to install the latest version for which there is no wheel in your wheelhouse.

很可能在未来某个时刻，scipy或numpy将发布新版本，并且 pip 将尝试安装最新版本，而您的驾驶室中没有轮子。

If you can live with the versions you have used so far, you shall create requirements.txtstating the versions of numpyand scipyyou like and install from it.

如果你可以用到目前为止你已经使用了版本的生活，你必须创建requirements.txt陈述的版本，numpy和scipy你喜欢的，并从它安装。

This shall ensure needed package to be present before it is really used.

这将确保需要的包在真正使用之前存在。