PyInstaller with Pandas 创建超过 500 MB exe

PyInstaller with Pandas creates over 500 MB exe

提问by dylan_fan

I try to create an exe file using PyInstaller 3.2.1, for test purpose I tried to make an exe for following code:

我尝试使用 PyInstaller 3.2.1 创建一个 exe 文件，出于测试目的，我尝试为以下代码制作一个 exe：

import pandas as pd
print('hello world')

After considerable amount of time (15mins +) I finished with dist folder as big as 620 MB and build - 150 MB. I work on Windows using Python 3.5.2 |Anaconda custom (64-bit). Might be worth noting that in dist folder mkl files are responsible for almost 300 MB. I run pyinstaller using 'pyinstaller.exe foo.py'. I tried using --exclude-module to exclude some dependencies, still ended up with huge files. Whether I use onefile or onedir doesn't make any difference.

经过相当长的时间（15 分钟 +）后，我完成了 620 MB 的 dist 文件夹和 build - 150 MB。我使用 Python 3.5.2 | Anaconda custom（64 位）在 Windows 上工作。可能值得注意的是，在 dist 文件夹中，mkl 文件占近 300 MB。我使用'pyinstaller.exe foo.py'运行pyinstaller。我尝试使用 --exclude-module 来排除一些依赖项，但最终还是得到了巨大的文件。无论我使用 onefile 还是 onedir 都没有任何区别。

I am aware that exe must contain some important files but is it normal to be as big as almost 1 GB? I can provide warning log if necessary or anything that could be helpful to solve the matter.

我知道 exe 必须包含一些重要文件，但是几乎 1 GB 大是否正常？如有必要，我可以提供警告日志或任何有助于解决问题的信息。

P.S. In parallel my coworker created an exe from same sample script and ended up with less than 100 MB, difference is he is not using anaconda. Could that be the matter?

PS 同时，我的同事从相同的示例脚本创建了一个 exe，结果不到 100 MB，不同之处在于他没有使用 anaconda。这可能是问题吗？

Any help will be appreciated.

任何帮助将不胜感激。

回答by abicorios

PyInstaller create the big executable from the conda packages and the small executable from the pip packages. From this simple python code:

PyInstaller 从 conda 包创建大可执行文件，从 pip 包创建小可执行文件。从这个简单的python代码：

from pandas import DataFrame as df
print('h')

I obtain the 203MB executable by the conda packages and the 30MB executable by the pip packages. But conda is the nice replacement of the pure virtualenv. I can develop with conda and Jupyter, create some mycode.py (I can download jupyter notebook as py-file in myfolder). But my final solution is next: If you do not have it, install Minicondaand from the Windows Start Menu open Anaconda Prompt;

我通过 conda 包获得了 203MB 的可执行文件，通过 pip 包获得了 30MB 的可执行文件。但是 conda 是纯 virtualenv 的不错替代品。我可以使用 conda 和 Jupyter 进行开发，创建一些 mycode.py（我可以将 jupyter notebook 作为 py 文件下载到 myfolder 中）。但我的最终解决方案是下一个：如果您没有它，请安装Miniconda并从 Windows 开始菜单中打开 Anaconda Prompt；

    cd myfolder
    conda create -n exe python=3
    activate exe
    pip install pandas pyinstaller pypiwin32
    echo hiddenimports = ['pandas._libs.tslibs.timedeltas'] > %CONDA_PREFIX%\Lib\site-packages\PyInstaller\hooks\hook-pandas.py
    pyinstaller -F mycode.py

Where I create new environment 'exe', pypiwin32 need for pyinstaller but is not installed automaticaly, hook-pandas.py need for compile with pandas. Also, import submodules do not help me optimize the size of the executable file. So I do not need this thing:

在我创建新环境“exe”的地方，pypiwin32 需要 pyinstaller 但不会自动安装，hook-pandas.py 需要使用 Pandas 进行编译。此外，导入子模块并不能帮助我优化可执行文件的大小。所以我不需要这个东西：

from pandas import DataFrame as df

but I can just use the usual code:

但我可以只使用通常的代码：

import pandas as pd

Also, some errors are possible along using the national letters in paths, so it is nice the english user account for development tools.

此外，在路径中使用国家字母时可能会出现一些错误，因此开发工具的英文用户帐户很好。

回答by Maarten Fabré

This is probably because the Anaconda version of numpyis built using mkl.

这可能是因为 Anaconda 版本numpy是使用mkl 构建的。

If you want to reduce the size of the distributable, you could work with a seperate building virtual environmentwith the packages installed through pipinstead of conda

如果您想减小可分发文件的大小，您可以使用单独的构建虚拟环境，并通过安装的软件包pip而不是conda

回答by Nikhil VJ

Here's a way to still be using condaand avoid mkl. Install numpy before installing pandas with this alternate command:
conda install -c conda-forge numpy

这是一种仍然使用conda和避免的方法mkl。在使用此备用命令安装 Pandas 之前安装 numpy：
conda install -c conda-forge numpy

Avoids mkl, uses an OpenBLASpackage in its place. Full explanation in this issue at conda/conda-forge/numpy-feedstock github repo.

避免mkl，在其位置使用OpenBLAS包。这个问题的完整解释在conda/conda-forge/numpy-feedstock github repo。

回答by JSBY

A simple solution while working with Anaconda:

使用 Anaconda 时的简单解决方案：

-Make a new environment inside Anaconda Navigator. (The new environment is free from the large amounts of packages that are causing the problem.)

- 在 Anaconda Navigator 中创建一个新环境。（新环境没有导致问题的大量软件包。）

-Open a terminal and use pipinstall to include the packages you need. ( Make sure it is in the new environment)

- 打开终端并使用 pipinstall 来包含您需要的包。（确保它在新环境中）

-Run pyinstaller.

- 运行 pyinstaller。

I reduced my .exe from 300 MB to 30 MB.

我将我的 .exe 从 300 MB 减少到 30 MB。

回答by Mitchell_h

I have the Anaconda 3.5.5 build for Python on Windows 10 and was also getting excessively large executables using the Anaconda distribution.

我在 Windows 10 上安装了 Python 的 Anaconda 3.5.5 版本，并且还使用 Anaconda 发行版获得了过大的可执行文件。

I was able to correct this by doing the following:

我能够通过执行以下操作来纠正此问题：

1. First create a virtual environment (forums suggest virtualenv, but this gave me problems so instead I used venv)

   python -m venv C:/Python/NewEnv
   

1. 首先创建一个虚拟环境（论坛建议使用virtualenv，但这给我带来了问题，所以我改用了venv）

   python -m venv C:/Python/NewEnv
   

This creates a virtual environment inside C:/Python/NewEnv with base python, pip and setuptools

这将在 C:/Python/NewEnv 中创建一个带有基本 python、pip 和 setuptools 的虚拟环境

1. Next switch to the newly created environment

   C:/Python/NewEnv/Scripts/activate
   

2. 接下来切换到新创建的环境

   C:/Python/NewEnv/Scripts/activate
   

You'll know that the environment is different as your command prompt will be prefaced with your new environment name (NewEnv)

您会知道环境是不同的，因为您的命令提示符将以您的新环境名称 (NewEnv) 开头

1. Install numpy first, then scipy, then pandas

   pip install numpy==1.13.3
   pip install scipy==1.1.0
   pip install pandas==0.18.1
   pip install pypiwin32==223
   pip install pyinstaller==3.2
   

3. 先安装numpy，然后scipy，然后pandas

   pip install numpy==1.13.3
   pip install scipy==1.1.0
   pip install pandas==0.18.1
   pip install pypiwin32==223
   pip install pyinstaller==3.2
   

I had to use these versions as I've tried different ones, but any later version of pandas were giving me further issues.

我不得不使用这些版本，因为我尝试了不同的版本，但是任何更高版本的Pandas都给我带来了更多问题。

1. Once these have been installed you can compile your program

   C:/Python/NewEnv/Scripts/pyinstaller --onefile program.py
   

2. This will create a .spec file, which you'll need to modify with this version of pandas and pyinstaller to add hidden imports otherwise loading pandas from the executable will fail (Not sure if there's a pyinstaller command to just create the spec file, but if there is then rather do that - see ammendment#1)

4. 一旦安装了这些，你就可以编译你的程序了

   C:/Python/NewEnv/Scripts/pyinstaller --onefile program.py
   

5. 这将创建一个 .spec 文件，您需要使用此版本的 Pandas 和 pyinstaller 对其进行修改以添加隐藏的导入，否则从可执行文件加载 Pandas 将失败（不确定是否有 pyinstaller 命令来创建规范文件，但是如果有那么宁愿这样做 - 见修正案#1）

There will be a hidden imports line inside the newly created .spec file:

在新创建的 .spec 文件中会有一个隐藏的导入行：

    hiddenimports=[],

Change this to add pandas._libs.tslibs.timedeltas

更改此项以添加 pandas._libs.tslibs.timedeltas

    hiddenimports=['pandas._libs.tslibs.timedeltas'],

1. Then you can compile your program again against the .spec file

   C:/Python/NewEnv/Scripts/pyinstaller --onefile program.spec
   

6. 然后您可以针对 .spec 文件再次编译您的程序

   C:/Python/NewEnv/Scripts/pyinstaller --onefile program.spec
   

Note that this will install the program in whichever directory you are in so change directories before executing pyinstaller.

请注意，这会将程序安装在您所在的任何目录中，因此在执行 pyinstaller 之前更改目录。

Ammendmend#1: I see that it's possible to add the hook-pandas.py to the Pyinstaller hooks. So after you install pyinstaller in the new environment, run

修正#1：我发现可以将 hook-pandas.py 添加到 Pyinstaller 挂钩。所以在新环境中安装pyinstaller后，运行

    echo hiddenimports = ['pandas._libs.tslibs.timedeltas'] > C:\Python\NewEnv\Lib\site-packages\PyInstaller\hooks\hook-pandas.py

回答by captain

You need pure python environment, No Anaconda.

你需要纯 python 环境，没有 Anaconda。

Because, it has too many useless packages. Install new python environment on another PC with as few package as possible!

因为，它有太多无用的包。使用尽可能少的软件包在另一台 PC 上安装新的 python 环境！

Then try to use pyinstaller again. With this method, pyinstaller reduced the file from 200M to 8M.

然后再次尝试使用 pyinstaller。通过这种方法，pyinstaller 将文件从 200M 减少到 8M。

PS: If you lack of some packages, you can pip install ...

PS：如果你缺少一些包，你可以 pip install ...