在 Ubuntu 12.04 中的 Python 2.7 中导入 Tensorflow 时出错。'未找到 GLIBC_2.17'

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/33655731/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-19 13:43:04  来源:igfitidea点击:

Error while importing Tensorflow in Python 2.7 in Ubuntu 12.04. 'GLIBC_2.17 not found'

pythonubuntuglibctensorflow

提问by Tanvir

I have installed the Tensorflow bindings with python successfully. But when I try to import Tensorflow, I get the follwoing error.

我已经成功安装了 python 的 Tensorflow 绑定。但是当我尝试导入 Tensorflow 时,出现以下错误。

ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.17' not found (required by /usr/local/lib/python2.7/dist-packages/tensorflow/python/_pywrap_tensorflow.so)

I have tried to update GLIBC_2.15 to 2.17, but no luck.

我曾尝试将 GLIBC_2.15 更新到 2.17,但没有成功。

采纳答案by Fabiano

I had the same problem, so googling I made these steps:

我遇到了同样的问题,所以谷歌搜索我做了以下步骤:

$ sudo pip install --upgrade virtualenv
$ virtualenv --system-site-packages ~/tensorflow
$ cd ~/tensorflow
$ source bin/activate
$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
$ cd /tmp
$ wget http://launchpadlibrarian.net/137699828/libc6_2.17-0ubuntu5_amd64.deb
$ wget http://launchpadlibrarian.net/137699829/libc6-dev_2.17-0ubuntu5_amd64.deb
$ mkdir libc6_2.17
$ cd libc6_2.17
$ ar p ../libc6_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
$ ar p ../libc6-dev_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
$ cd -
$ LD_LIBRARY_PATH=/tmp/libc6_2.17/lib/x86_64-linux-gnu/ /tmp/libc6_2.17/lib/x86_64-linux-gnu/ld-2.17.so bin/python local/lib/python2.7/site-packages/tensorflow/models/image/mnist/convolutional.py

And to exit:

并退出:

$ deactivate 

That works for me.

这对我行得通。

回答by shanky_thebearer

That error mainly arises if your GNU C-Library is not up-to-date. You can check what version you're running using a simple

如果您的 GNU C-Library 不是最新的,则主要会出现该错误。您可以使用简单的方法检查您正在运行的版本

$ ldd --version

The output should be like this:

输出应该是这样的:

ldd (Ubuntu EGLIBC 2.19-0ubuntu6.6) 2.19
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

The 2.19 is your GLIBC version. To upgrade you can visit the GNU-C Library project website and download the latest version. The link to the latest glibc is here : GNU-C library DownloadAt the time of writing this answer the latest stable version was 2.22.

2.19 是您的 GLIBC 版本。要升级,您可以访问 GNU-C 库项目网站并下载最新版本。最新 glibc 的链接在这里:GNU-C 库下载在撰写此答案时,最新的稳定版本是 2.22

I tried running tensorflow on both Ubuntu 12.04 and 14.04. Ubuntu 14.04 doesn't throw this issue as it comes with glibc 2.19 installed by default.

我尝试在 Ubuntu 12.04 和 14.04 上运行 tensorflow。Ubuntu 14.04 不会抛出这个问题,因为它默认安装了 glibc 2.19。

Hope it helps.

希望能帮助到你。

回答by Théo T

I tried BR_User solutionand still had an annoying:

我尝试了BR_User 解决方案,但仍然很烦人:

ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.14' not found

I am on CentOS 6.7, it also lacks an updated c++ standard lib, so to build on BR_User solution I extracted the correct libstdc++ package, however I found no need for the virtual environment.

我在 CentOS 6.7 上,它也缺少更新的 c++ 标准库,因此为了构建 BR_User 解决方案,我提取了正确的 libstdc++ 包,但是我发现不需要虚拟环境。

Supposing you have already installed tensorflow, it gives:

假设您已经安装了 tensorflow,它会给出:

mkdir ~/my_libc_env
cd ~/my_libc_env
wget http://launchpadlibrarian.net/137699828/libc6_2.17-0ubuntu5_amd64.deb
wget http://launchpadlibrarian.net/137699829/libc6-dev_2.17-0ubuntu5_amd64.deb
wget ftp.riken.jp/Linux/scientific/7.0/x86_64/os/Packages/libstdc++-4.8.2-16.el7.x86_64.rpm
ar p libc6_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
ar p libc6-dev_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
rpm2cpio libstdc++-4.8.2-7mgc30.x86_64.rpm| cpio -idmv

and then run python with:

然后使用以下命令运行 python:

LD_LIBRARY_PATH="$HOME/my_libc_env/lib/x86_64-linux-gnu/:$HOME/my_libc_env/usr/lib64/" $HOME/my_libc_env/lib/x86_64-linux-gnu/ld-2.17.so `which python`

If it doesn't work, I have another solution, but you won't like it.

如果它不起作用,我有另一个解决方案,但你不会喜欢它。

回答by Théo T

Okay so here is the other solution I mentionned in my previous answer, it's more tricky, but should always work on systems with GLIBC>=2.12 and GLIBCXX>=3.4.13. In my case it was on a CentOS 6.7, but it's also fine for Ubuntu 12.04.

好的,这里是我在之前的回答中提到的另一个解决方案,它更棘手,但应该始终适用于 GLIBC>=2.12 和 GLIBCXX>=3.4.13 的系统。就我而言,它在 CentOS 6.7 上运行,但它也适用于 Ubuntu 12.04。

We're going to need a version of gcc that supports c++11, either on another machine or an isolated install; but not for the moment.

我们将需要一个支持 c++11 的 gcc 版本,无论是在另一台机器上还是独立安装;但暂时不是。

What we're gonna do here is edit the _pywrap_tensorflow.so binary in order to 'weakify' its libc and libstdc++ dependencies, so that ld accepts to link the stubs we're gonna make. Then we'll make those stubs for the missing symbols, and finally we're gonna pre-load all of this when running python.

我们在这里要做的是编辑 _pywrap_tensorflow.so 二进制文件以“弱化”其 libc 和 libstdc++ 依赖项,以便 ld 接受链接我们将要制作的存根。然后我们将为缺失的符号制作那些存根,最后我们将在运行 python 时预加载所有这些。

First of all, I want to thank James for his great article ( http://www.lightofdawn.org/wiki/wiki.cgi/NewAppsOnOldGlibc) and precious advices, I couldn't have made it without him.

首先,我要感谢 James 的精彩文章(http://www.lightofdawn.org/wiki/wiki.cgi/NewAppsOnOldGlibc)和宝贵的建议,没有他我不可能做到。

So let's start by weakifying the dependencies, it's just about replacing the right bytes in _pywrap_tensorflow.so. Please note that this step only works for the current version of tensorflow (0.6.0). So if its not done already create and activate your virtualenvif you have one (if you're not admin virtualenv is a solution, another is to add --userflag to pip command), and install tensorflow 0.6.0 (replace cpu by gpu in the url if you want the gpu version) :

所以让我们从弱化依赖开始,它只是替换 _pywrap_tensorflow.so 中正确的字节。请注意,此步骤仅适用于当前版本的 tensorflow (0.6.0)。因此,如果它还没有完成,则已经创建并激活您的virtualenv(如果您不是管理员,virtualenv 是一种解决方案,另一个是--user向 pip 命令添加标志),并安装 tensorflow 0.6.0(在url 如果你想要 gpu 版本):

pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.6.0-cp27-none-linux_x86_64.whl

And let's weakify all the annoying dependencies, here is the command for the cpu version of tensorflow:

让我们弱化所有烦人的依赖项,这里是 tensorflow 的 cpu 版本的命令:

TENSORFLOW_DIR=`python -c "import imp; print(imp.find_module('tensorflow')[1])"`
for addr in 0xC6A93C 0xC6A99C 0xC6A9EC 0xC6AA0C 0xC6AA1C 0xC6AA3C; do printf '\x02' | dd conv=notrunc of=${TENSORFLOW_DIR}/python/_pywrap_tensorflow.so bs=1 seek=$((addr)) ; done

And here is the gpu one (run only the correct one or you'll corrupt the binary):

这是 gpu 之一(只运行正确的一个,否则你会破坏二进制文件):

TENSORFLOW_DIR=`python -c "import imp; print(imp.find_module('tensorflow')[1])"`
for addr in 0xDC5EA4 0xDC5F04 0xDC5F54 0xDC5F74 0xDC5F84 0xDC5FA4; do printf '\x02' | dd conv=notrunc of=${TENSORFLOW_DIR}/python/_pywrap_tensorflow.so bs=1 seek=$((addr)) ; done

You can check it with:

您可以通过以下方式检查:

readelf -V ${TENSORFLOW_DIR}/python/_pywrap_tensorflow.so

Have a look at the article if you want to understand what's going on here.

如果你想了解这里发生了什么,请查看这篇文章。

Now we're gonna make the stubs for the missing libc symbols:

现在我们要为缺少的 libc 符号制作存根:

mkdir ~/my_stubs
cd ~/my_stubs
MYSTUBS=~/my_stubs
printf "#include <time.h>\n#include <string.h>\nvoid* memcpy(void *dest, const void *src, size_t n) {\nreturn memmove(dest, src, n);\n}\nint clock_gettime(clockid_t clk_id, struct timespec *tp) {\nreturn clock_gettime(clk_id, tp);\n}" > mylibc.c
gcc -s -shared -o mylibc.so -fPIC -fno-builtin mylibc.c

You needto perform that step on the machine with the missing dependencies (or machine with similar versions of standard libraries (in a cluster for example)).

需要在缺少依赖项的机器(或具有类似标准库版本的机器(例如在集群中))上执行该步骤。

Now we're gonna probably change of machine since we need a gcc that supports c++11, and it is probably not on the machine that lacks all the dependencies (or you can use an isolated install of a recent gcc). In the following I assume we're still in ~/my_stubsand somehow you share your home accross the machines, otherwise you'll just have to copy the .so files we're gonna generate when it's done.

现在我们可能要更换机器,因为我们需要一个支持 c++11 的 gcc,而且它可能不在缺少所有依赖项的机器上(或者您可以使用最近 gcc 的独立安装)。在下面,我假设我们仍然在,~/my_stubs并且您以某种方式在机器上共享您的家,否则您只需要复制我们将在完成后生成的 .so 文件。

So, one stub that we can do for libstdc++, and for the remaining missing ones we're going to compile them from gcc source (it might take some time to clone the repository):

所以,我们可以为 libstdc++ 做一个存根,对于剩下的缺失的存根,我们将从 gcc 源代码编译它们(克隆存储库可能需要一些时间):

printf "#include <functional>\nvoid std::__throw_bad_function_call(void) {\nexit(1);\n}" > bad_function.cc
gcc -std=c++11 -s -shared -o bad_function.so -fPIC -fno-builtin bad_function.cc
git clone https://github.com/gcc-mirror/gcc.git
cd gcc
mkdir my_include
mkdir my_include/ext
cp libstdc++-v3/include/ext/aligned_buffer.h my_include/ext
gcc -I$PWD/my_include -std=c++11 -fpermissive -s -shared -o $MYSTUBS/hashtable.so -fPIC -fno-builtin libstdc++-v3/src/c++11/hashtable_c++0x.cc
gcc -std=c++11 -fpermissive -s -shared -o $MYSTUBS/chrono.so -fPIC -fno-builtin libstdc++-v3/src/c++11/chrono.cc
gcc -std=c++11 -fpermissive -s -shared -o $MYSTUBS/random.so -fPIC -fno-builtin libstdc++-v3/src/c++11/random.cc
gcc -std=c++11 -fpermissive -s -shared -o $MYSTUBS/hash_bytes.so -fPIC -fno-builtin ./libstdc++-v3/libsupc++/hash_bytes.cc

And that's it! You can now run a tensorflow python script by preloading all our shared libraries (and your local libstdc++):

就是这样!您现在可以通过预加载我们所有的共享库(以及您的本地 libstdc++)来运行 tensorflow python 脚本:

LIBSTDCPP=`ldconfig -p | grep libstdc++.so.6 | grep 64 | cut -d' ' -f4` #For 64bit machines
LD_PRELOAD="$MYSTUBS/mylibc.so:$MYSTUBS/random.so:$MYSTUBS/hash_bytes.so:$MYSTUBS/chrono.so:$MYSTUBS/hashtable.so:$MYSTUBS/bad_function.so:$LIBSTDCPP" python ${TENSORFLOW_DIR}/models/image/mnist/convolutional.py

:)

:)

回答by Sam

My solution is similar to that of Theo T; although fine tuned for Xubuntu 12.04 (CAELinux 2013)

我的解决方案类似于 Theo T 的解决方案;尽管针对 Xubuntu 12.04 (CAELinux 2013) 进行了微调

sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.6.0-cp27-none-linux_x86_64.whl
sudo su
# prompt for password
TENSORFLOW_DIR=`python -c "import imp; print(imp.find_module('tensorflow')[1])"`
for addr in 0xC6A93C 0xC6A99C 0xC6A9EC 0xC6AA0C 0xC6AA1C 0xC6AA3C; do printf '\x02' | dd conv=notrunc of=${TENSORFLOW_DIR}/python/_pywrap_tensorflow.so bs=1 seek=$((addr)) ; done

readelf -V ${TENSORFLOW_DIR}/python/_pywrap_tensorflow.so
exit

mkdir ~/.my_stubs
cd ~/.my_stubs
MYSTUBS=~/.my_stubs


printf "#include <time.h>\n#include <string.h>\nvoid* memcpy(void *dest, const void *src, size_t n) {\nreturn memmove(dest, src, n);\n}\nint clock_gettime(clockid_t clk_id, struct timespec *tp) {\nreturn clock_gettime(clk_id, tp);\n}" > mylibc.c
gcc -s -shared -o mylibc.so -fPIC -fno-builtin mylibc.c

printf "#include <functional>\nvoid std::__throw_bad_function_call(void) {\nexit(1);\n}" > bad_function.cc
gcc -std=c++11 -s -shared -o bad_function.so -fPIC -fno-builtin bad_function.cc

git clone https://github.com/gcc-mirror/gcc.git

cd gcc
mkdir my_include
mkdir my_include/ext
cp libstdc++-v3/include/ext/aligned_buffer.h my_include/ext
gcc -I$PWD/my_include -std=c++11 -fpermissive -s -shared -o $MYSTUBS/hashtable.so -fPIC -fno-builtin libstdc++-v3/src/c++11/hashtable_c++0x.cc
gcc -std=c++11 -fpermissive -s -shared -o $MYSTUBS/chrono.so -fPIC -fno-builtin libstdc++-v3/src/c++11/chrono.cc
gcc -std=c++11 -fpermissive -s -shared -o $MYSTUBS/random.so -fPIC -fno-builtin libstdc++-v3/src/c++11/random.cc
gcc -std=c++11 -fpermissive -s -shared -o $MYSTUBS/hash_bytes.so -fPIC -fno-builtin ./libstdc++-v3/libsupc++/hash_bytes.cc

The following two steps would run the convolution.py script located in the tensorflow/models/image/mnist directory:

以下两个步骤将运行位于 tensorflow/models/image/mnist 目录中的 convolution.py 脚本:

LIBSTDCPP=`ldconfig -p | grep libstdc++.so.6 | grep 64 | cut -d' ' -f4` #For 64bit machines
LD_PRELOAD="$MYSTUBS/mylibc.so:$MYSTUBS/random.so:$MYSTUBS/hash_bytes.so:$MYSTUBS/chrono.so:$MYSTUBS/hashtable.so:$MYSTUBS/bad_function.so:$LIBSTDCPP" python ${TENSORFLOW_DIR}/models/image/mnist/convolutional.py

If you want python to be always loaded with these reduced dependencies, simply add this to the .bashrc file with the below step:

如果您希望 python 始终加载这些减少的依赖项,只需使用以下步骤将其添加到 .bashrc 文件中:

echo "LIBSTDCPP=`ldconfig -p | grep libstdc++.so.6 | grep 64 | cut -d' ' -f4`" >> ~/.bashrc
echo alias python="'"LD_PRELOAD='"''$'MYSTUBS/mylibc.so:'$'MYSTUBS/random.so:'$'MYSTUBS/hash_bytes.so:'$'MYSTUBS/chrono.so:'$'MYSTUBS/hashtable.so:'$'MYSTUBS/bad_function.so:'$'LIBSTDCPP'"' python"'" >> ~/.bashrc

And if you have ipython:

如果你有 ipython:

echo alias ipython="'"LD_PRELOAD='"''$'MYSTUBS/mylibc.so:'$'MYSTUBS/random.so:'$'MYSTUBS/hash_bytes.so:'$'MYSTUBS/chrono.so:'$'MYSTUBS/hashtable.so:'$'MYSTUBS/bad_function.so:'$'LIBSTDCPP'"' ipython"'" >> ~/.bashrc

Basically, I updated it to the python installation being done in dist_packages instead of site-packages. Also, I clone from the gcc-mirrorinstead of the native git page for gcc. Rest all is the same.

基本上,我将它更新为在 dist_packages 而不是 site-packages 中完成的 python 安装。另外,我从gcc-mirror克隆而不是gcc的本机 git 页面。其余的都是一样的。

The last few steps ensure that every time you run python or ipython, it would be loaded with the reduced dependencies.

最后几个步骤确保每次运行 python 或 ipython 时,它都会加载减少的依赖项。

回答by Yajun Huang

I have tried https://stackoverflow.com/a/34897674/5929065solution, and its worked perfectly. But libstdc++-4.8.2-7 download link cannot be accessed any more, try to pick up a new one from this page: http://rpm.pbone.net/index.php3/stat/4/idpl/31981489/dir/centos_7/com/libstdc++-4.8.5-4.el7.x86_64.rpm.html

我已经尝试过https://stackoverflow.com/a/34897674/5929065解决方案,并且效果很好。但是libstdc++-4.8.2-7下载链接已经无法访问了,尝试从这个页面获取一个新的:http://rpm.pbone.net/index.php3/stat/4/idpl/31981489/dir /centos_7/com/libstdc++-4.8.5-4.el7.x86_64.rpm.html

For me, I use this:

对我来说,我用这个:

wget ftp://mirror.switch.ch/pool/4/mirror/centos/7.2.1511/os/x86_64/Packages/libstdc++-4.8.5-4.el7.x86_64.rpm

wget ftp://mirror.switch.ch/pool/4/mirror/centos/7.2.1511/os/x86_64/Packages/libstdc++-4.8.5-4.el7.x86_64.rpm

回答by Honglei Liu

The link in @Théo T's answer is no longer working, so I had to try several alternatives to finally find one that works.

@Théo T 的答案中的链接不再有效,所以我不得不尝试几种替代方法,最终找到一种有效的方法。

mkdir ~/my_libc_env
cd ~/my_libc_env
wget http://launchpadlibrarian.net/137699828/libc6_2.17-0ubuntu5_amd64.deb
wget http://launchpadlibrarian.net/137699829/libc6-dev_2.17-0ubuntu5_amd64.deb
wget ftp://ftp.icm.edu.pl/vol/rzm5/linux-slc/centos/7.0.1406/cr/x86_64/Packages/libstdc++-4.8.3-9.el7.x86_64.rpm
ar p libc6_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
ar p libc6-dev_2.17-0ubuntu5_amd64.deb data.tar.gz | tar zx
rpm2cpio libstdc++-4.8.3-9.el7.x86_64.rpm| cpio -idmv

Run python code with tensorflow

使用 tensorflow 运行 python 代码

LD_LIBRARY_PATH="$HOME/my_libc_env/lib/x86_64-linux-gnu/:$HOME/my_libc_env/usr/lib64/" $HOME/my_libc_env/lib/x86_64-linux-gnu/ld-2.17.so `which python` your_program.py

回答by Mithril

Notice: you should download deb package by your platform:

注意:您应该根据您的平台下载 deb 包:

x86:

x86:

https://launchpad.net/ubuntu/raring/i386/libc6/2.17-0ubuntu5.1

https://launchpad.net/ubuntu/raring/i386/libc6/2.17-0ubuntu5.1

wget http://launchpadlibrarian.net/151932048/libc6_2.17-0ubuntu5.1_i386.deb

sudo dpkg -i libc6_2.17-0ubuntu5.1_i386.deb

x64:

x64:

https://launchpad.net/ubuntu/raring/amd64/libc6/2.17-0ubuntu5.1

https://launchpad.net/ubuntu/raring/amd64/libc6/2.17-0ubuntu5.1

wget http://launchpadlibrarian.net/151925896/libc6_2.17-0ubuntu5.1_amd64.deb

sudo dpkg -i libc6_2.17-0ubuntu5.1_amd64.deb

Test on my 12.04 Ubuntu x64.

在我的 12.04 Ubuntu x64 上测试。

回答by Tay Cho

Note to 'weakifying' solution of Theo Trouillon. It only applies for Tensorflow 0.6.0. If you want to apply it for Tensorflow 0.9.0, it gets trickier. My case was CPU mode, Centos 6.7 where GLIBC 2.12 GLIBCXX 3.4.13.

注意 Theo Trouillon 的“弱化”解决方案。它仅适用于 Tensorflow 0.6.0。如果你想将它应用于 Tensorflow 0.9.0,它会变得更加棘手。我的情况是 CPU 模式,Centos 6.7,其中 GLIBC 2.12 GLIBCXX 3.4.13。

Installing tensorflow :

安装张量流:

pip uninstall protobuf 
pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl

'weakifying' code should be CHANGED :

“弱化”代码应该改变:

TENSORFLOW_DIR=`python -c "import imp; print(imp.find_module('tensorflow')[1])"`
for addr in 0x376e34 0x376e94 0x376ef4 0x376f14 0x376f24 0x376f44 0x376f54 ; do printf '\x02' | dd conv=notrunc of=${TENSORFLOW_DIR}/python/_pywrap_tensorflow.so bs=1 seek=$((addr)) ; done
for addr in 0x93aa4 0x93b04 ; do printf '\x02' | dd conv=notrunc of=${TENSORFLOW_DIR}/contrib/layers/python/ops/_bucketization_op.so bs=1 seek=$((addr)) ; done
for addr in 0x95bec 0x95c4c ; do printf '\x02' | dd conv=notrunc of=${TENSORFLOW_DIR}/contrib/layers/python/ops/_sparse_feature_cross_op.so bs=1 seek=$((addr)) ; done
for addr in 0x9ffec 0x9fffc 0xa005c ; do printf '\x02' | dd conv=notrunc of=${TENSORFLOW_DIR}/contrib/metrics/python/ops/_set_ops.so bs=1 seek=$((addr)) ; done
for addr in 0x96aa4 0x96b04 0x96b24; do printf '\x02' | dd conv=notrunc of=${TENSORFLOW_DIR}/contrib/linear_optimizer/python/ops/_sdca_ops.so bs=1 seek=$((addr)) ; done

And following code should be ADDED to the stub generating code at newer machine.

以下代码应添加到较新机器上的存根生成代码中。

mkdir my_include/bits
cp libstdc++-v3/include/bits/shared_ptr_atomic.h my_include/bits/
cp libstdc++-v3/include/std/memory my_include/
gcc -I$PWD/my_include -std=c++11 -fpermissive -s -shared -o ${MYSTUBS}/shared_ptr.so -fPIC -fno-builtin ./libstdc++-v3/src/c++11/shared_ptr.cc
gcc -I$PWD/my_include -std=c++11 -fpermissive -s -shared -o ${MYSTUBS}/list.so -fPIC -fno-builtin ./libstdc++-v3/src/c++98/list.cc
echo "
#include <unistd.h>
#include <stdlib.h>
char * secure_getenv (const char *name) {
          if ((getuid () == geteuid ()) && (getgid () == getegid ())) return getenv (name); else  return NULL;
}" > getenv.cc
gcc -I$PWD/my_include  -std=c++11 -fpermissive -s -shared -o    ${MYSTUBS}/getenv.so -fPIC -fno-builtin getenv.cc

Of course the final running code should be changed to include additional stubs.

当然,最终运行的代码应该更改为包含额外的存根。

LD_PRELOAD="$MYSTUBS/list.so:$MYSTUBS/mylibc.so:$MYSTUBS/shared_ptr.so:$MYSTUBS/getenv.so:$MYSTUBS/random.so:$MYSTUBS/hash_bytes.so:$MYSTUBS/chrono.so:$MYSTUBS/hashtable.so:$MYSTUBS/bad_function.so:$LIBSTDCPP" python  ${TENSORFLOW_DIR}/models/image/mnist/convolutional.py

回答by Igor

I've just managed to install tensorflow 0.12rc0 on CentOS 6.5 with glibc 2.12, without having root privileges. Simply installing tensorflow binary via pip was giving me an error, related to GLIBC version as well.

我刚刚设法在 CentOS 6.5 上使用 glibc 2.12 安装了 tensorflow 0.12rc0,而没有 root 权限。简单地通过 pip 安装 tensorflow 二进制文件给了我一个错误,也与 GLIBC 版本有关。

Basically, you have 4 options how to deal with this (each with some advantages and disadvantages):

基本上,您有 4 个选项来处理这个问题(每个选项都有一些优点和缺点):

Option 1 - Upgrade your system GLIBC globally.

选项 1 - 全局升级您的系统 GLIBC。

This is, probably, the best option, ifyour system supports this, you have root privileges, and you are confident that this upgrade won't break anything for some weird reason. Ultimately, this goes up to upgrading the whole Linux distribution. Here's a nice short list of default GLIBC versions on popular distributions.

可能是最好的选择,如果您的系统支持此功能,您拥有 root 权限,并且您确信这次升级不会因为某些奇怪的原因而破坏任何东西。最终,这需要升级整个 Linux 发行版。是流行发行版上默认 GLIBC 版本的一个不错的简短列表。

Option 2 - Add second GLIBC to your system

选项 2 - 将第二个 GLIBC 添加到您的系统

Compile or download binary. The most simple&straightforward option. Especially if you only need to run few simple scripts.

编译或下载二进制文件。最简单直接的选择。特别是如果您只需要运行几个简单的脚本

  • It is possibleto have multiple versions of glibc on the same system, but one should do this with a great care.
  • You won't destroy your system, if all your changes would be limited to a virtual environment.
  • Many programs, installed/compiled before might be relying on old GLIBC, would just crashin your new environment (e.g. your python IDE). Including most basic bash commands, like "lc", "cd", etc.
  • Other side-effects like significant memory leaksare also possible.
  • Thus, it's a very bad idea to add new GLIBC to your normal environment, e.g. via .bashrc.
  • On the other hand, if you need some specific tool in your new virtual environment, you may recompile it, linking against new GLIBC. So, that it would work OK in your new enviroment.
  • However, personally, I quickly gave up recompiling everything I need in a new environment (without root and a package manager).
  • A slightly different approachis officially offered by GLIBC developers, for testing new GLIBC builds.
  • 可能具有相同的系统上的glibc的多个版本,但应该有一个非常小心做到这一点。
  • 如果您的所有更改都仅限于虚拟环境,您就不会破坏您的系统。
  • 许多之前安装/编译的程序可能依赖于旧的 GLIBC,只会在您的新环境(例如您的 Python IDE)中崩溃。包括最基本的 bash 命令,如“lc”、“cd”等。
  • 其他副作用,如显着的内存泄漏也是可能的。
  • 因此,将新的 GLIBC 添加到您的正常环境中是一个非常糟糕的主意,例如通过.bashrc.
  • 另一方面,如果您在新的虚拟环境中需要一些特定的工具,您可以重新编译它,链接到新的 GLIBC。因此,它可以在您的新环境中正常工作。
  • 但是,就我个人而言,我很快就放弃了在新环境(没有 root 和包管理器)中重新编译我需要的一切。
  • GLIBC 开发人员正式提供了一种稍微不同的方法,用于测试新的 GLIBC 构建。

Option 3 - Patch tensorflow

选项 3 - 修补 tensorflow

Thismay work for TF 0.6.0, but you would probably have to start again from scratch, when each new tensorflow version is released. E.g. here's a fix for 0.9.0.

可能适用于 TF 0.6.0,但是当每个新的 tensorflow 版本发布时,您可能必须从头开始。例如,这里是 0.9.0 的修复程序。

Option 4 - Compile tensorflow from source

选项 4 - 从源代码编译 tensorflow

If you re-compile it from source and link against your existing GLIBC, newer GLIBC would be no longer needed. Somehow, this option was not mentioned in any answer here yet. Imho, this is the best option, both "in general", and "specifically for tensorflow".

如果您从源代码重新编译它并链接到您现有的 GLIBC,将不再需要更新的 GLIBC。不知何故,这里的任何答案中都没有提到这个选项。恕我直言,这是“一般”和“专门用于张量流”的最佳选择。

  • This works OK with r0.11 and would probably work for years, but theoretically, it might break in some newer tensorflow version, if they would decide to actually use some new GLIBC functionality, not present in older versions.
  • To be honest, building tensorflow from source is not straightforward, especially on outdated systems.
  • 这适用于 r0.11,并且可能会工作多年,但从理论上讲,如果他们决定实际使用一些新的 GLIBC 功能,而不是旧版本中存在的某些新的 tensorflow 版本,它可能会中断。
  • 老实说,从源代码构建 tensorflow 并不简单,尤其是在过时的系统上。

A quick summary of "building tensorflow on outdated system":

“在过时的系统上构建 tensorflow”的快速总结:

Although the official guide provides a "installing from sources" section, there are few tricks you need to do to build it on an outdated system. Here I assume, that you do not have root privileges (if you do - you probably would be able to install the same pre-requestities with a package manager, rather them manually building them from source).

虽然官方指南提供了“从源代码安装”部分,但在过时的系统上构建它需要做的技巧很少。在这里,我假设您没有 root 权限(如果您有 - 您可能能够使用包管理器安装相同的预请求,而不是从源代码手动构建它们)。

I found two well-documented success stories: #1, #2and a number of useful posts on the official github (mostly about a set of libraries to link inside the binary): #1, #2, #3, #4. I had to combine tricks, described there to successfully compile TF in my case.

我发现了两个有据可查的成功故事:#1#2和官方 github 上的一些有用的帖子(主要是关于二进制文件中链接的一组库):#1#2#3#4。在我的案例中,我必须结合在那里描述的技巧才能成功编译 TF。

  1. First of all, check your gcc --version, and verifythat it supports c++11. Mine was 4.4.7, so it won't work. I've downloadedgcc-4.9.4 source code, and compiled it. This step is pretty straightforward, but the compilation itself may take few hours. As a workaround for an issue in bazel, I've compiled gcc with hardcoded pathsto as,ldand nm. However, you may try another workarounds: (1, 2).

    #!/bin/sh
    
    unset LIBRARY_PATH CPATH C_INCLUDE_PATH 
    unset PKG_CONFIG_PATH CPLUS_INCLUDE_PATH INCLUDE LD_LIBRARY_PATH
    
    cd gcc-4.9.4
    ./contrib/download_prerequisites
    
    mkdir objdir
    cd objdir
    
    
    # I've added --disable-multilib to fix the following error:
    # /usr/bin/ld: crt1.o: No such file: No such file or directory
    # collect2: ld returned 1 exit status
    # configure: error: I suspect your system does not have 32-bit 
    # developement libraries (libc and headers). If you have them,
    # rerun configure with --enable-multilib. If you do not have them, 
    # and want to build a 64-bit-only compiler, rerun configure 
    # with --disable-multilib.           
    
    ../configure --prefix=$HOME/opt/gcc-4.9.4 \
                 --disable-multilib \
                 --disable-nls \
                 --enable-languages=c,c++ \
                 --with-ld=/usr/bin/ld \
                 --with-nm=/usr/bin/nm \
                 --with-as=/usr/bin/as
    
    make        
    make install
    
  2. Check your java --version. Bazel requires JDK 8, install it if necessary. (They still providesome jdk7 related downloads, for bazel-0.4.1 but it looks like they consider it deprecated)

  3. I've created a separate use_gcc_4.9.4.shfile, with necessary environment variables. I use source ./use_gcc_4.9.4.shwhen I need to so something related to this newer compiler.

    #!/bin/sh
    this=$HOME/opt/gcc-4.9.4
    export PATH=$this/bin:$PATH
    export CPATH=$this/include:$CPATH
    export LIBRARY_PATH=$this/lib:$LIBRARY_PATH
    export LIBRARY_PATH=$this/lib64:$LIBRARY_PATH
    export LD_LIBRARY_PATH=$this/lib:$LD_LIBRARY_PATH
    export LD_LIBRARY_PATH=$this/lib64:$LD_LIBRARY_PATH
    
  4. The current bazel binary (0.4.1) requires GLIBC 2.14, so we have to compile bazel from sourceas well (with our new gcc). Works OK, unless you are only allowed to run a very limited number of threadson the target machine. (Thispost describes some additional workarounds, but in my case they were not needed, maybe due to recent updates in bazel code.)

  5. Obtain tensorflow source code git clone https://github.com/tensorflow/tensorflow, and install prerequisites you need (CUDA,cuDNN,python, etc). See official guide.

  6. If you're not using default system gcc (e.g. if you had to compile newer gcc, like discussed above), add the following linker flagsto tensorflow/third_party/gpus/crosstool/CROSSTOOL.tpl, line 59:

    linker_flag: "-L/home/username/localinst/opt/gcc-4.9.4/lib64"
    linker_flag: "-Wl,-rpath,/home/username/localinst/opt/gcc-4.9.4/lib64"
    

    Without this step, you would likely run into error messages like this:

    # ERROR: /home/username/localdistr/src/tensorflow/tensorflow/tensorflow/core/debug/BUILD:33:1: null failed: protoc failed: error executing command bazel-out/host/bin/external/protobuf/protoc '--cpp_out=bazel-out/local_linux-py3-opt/genfiles/' '--plugin=protoc-gen-grpc=bazel-out/host/bin/external/grpc/grpc_cpp_plugin' ... (remaining 8 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
    # bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by bazel-out/host/bin/external/protobuf/protoc)
    # bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by bazel-out/host/bin/external/protobuf/protoc)
    # bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by bazel-out/host/bin/external/protobuf/protoc)
    
  7. Finally, to avoid GLIBC dependencies, we have to statically link some libraries, by adding the -lrtlinker flag (maybe-lmas well). I found multiple posts, suggesting to add this in a different manner:

    Without -lrtI ran into GLIBC-version-specific error again, trying to import tensorflow:

    # ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/username/anaconda3/envs/myenvname/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow.so)
    

    Without -lmyou may run into this(for me, it turned out to be not necessary).

  8. Run the build process.

  1. 首先,检查您的gcc --version,并验证它是否支持 c++11。我的是 4.4.7,所以它不会工作。我已经下载了gcc-4.9.4 源代码,并编译了它。这一步非常简单,但编译本身可能需要几个小时。作为 bazel 中问题的解决方法,我编译了 gcc,其硬编码路径as,ldnm. 但是,您可以尝试其他解决方法:( 1, 2)。

    #!/bin/sh
    
    unset LIBRARY_PATH CPATH C_INCLUDE_PATH 
    unset PKG_CONFIG_PATH CPLUS_INCLUDE_PATH INCLUDE LD_LIBRARY_PATH
    
    cd gcc-4.9.4
    ./contrib/download_prerequisites
    
    mkdir objdir
    cd objdir
    
    
    # I've added --disable-multilib to fix the following error:
    # /usr/bin/ld: crt1.o: No such file: No such file or directory
    # collect2: ld returned 1 exit status
    # configure: error: I suspect your system does not have 32-bit 
    # developement libraries (libc and headers). If you have them,
    # rerun configure with --enable-multilib. If you do not have them, 
    # and want to build a 64-bit-only compiler, rerun configure 
    # with --disable-multilib.           
    
    ../configure --prefix=$HOME/opt/gcc-4.9.4 \
                 --disable-multilib \
                 --disable-nls \
                 --enable-languages=c,c++ \
                 --with-ld=/usr/bin/ld \
                 --with-nm=/usr/bin/nm \
                 --with-as=/usr/bin/as
    
    make        
    make install
    
  2. 检查您的java --version. Bazel 需要 JDK 8,如有必要,请安装它。(他们仍然提供一些与 jdk7 相关的下载,用于 bazel-0.4.1 但看起来他们认为它已被弃用)

  3. 我创建了一个单独的use_gcc_4.9.4.sh文件,其中包含必要的环境变量。source ./use_gcc_4.9.4.sh当我需要与这个较新的编译器相关的东西时,我会使用它。

    #!/bin/sh
    this=$HOME/opt/gcc-4.9.4
    export PATH=$this/bin:$PATH
    export CPATH=$this/include:$CPATH
    export LIBRARY_PATH=$this/lib:$LIBRARY_PATH
    export LIBRARY_PATH=$this/lib64:$LIBRARY_PATH
    export LD_LIBRARY_PATH=$this/lib:$LD_LIBRARY_PATH
    export LD_LIBRARY_PATH=$this/lib64:$LD_LIBRARY_PATH
    
  4. 当前的 bazel 二进制文件 (0.4.1)需要 GLIBC 2.14,因此我们也必须从源代码编译 bazel(使用我们的新 gcc)。工作正常,除非您只被允许在目标机器上运行非常有限数量的线程。(这篇文章描述了一些额外的解决方法,但在我的情况下不需要它们,可能是由于最近更新了 bazel 代码。)

  5. 获取 tensorflow 源代码git clone https://github.com/tensorflow/tensorflow,并安装您需要的先决条件(CUDA、cuDNN、python 等)。见官方指南

  6. 如果你不使用默认的系统GCC(例如,如果你有编译新的gcc,像上面所讨论的),添加下面的链接器标志tensorflow/third_party/gpus/crosstool/CROSSTOOL.tpl第59行

    linker_flag: "-L/home/username/localinst/opt/gcc-4.9.4/lib64"
    linker_flag: "-Wl,-rpath,/home/username/localinst/opt/gcc-4.9.4/lib64"
    

    如果没有这一步,你可能会遇到这样的错误消息这样

    # ERROR: /home/username/localdistr/src/tensorflow/tensorflow/tensorflow/core/debug/BUILD:33:1: null failed: protoc failed: error executing command bazel-out/host/bin/external/protobuf/protoc '--cpp_out=bazel-out/local_linux-py3-opt/genfiles/' '--plugin=protoc-gen-grpc=bazel-out/host/bin/external/grpc/grpc_cpp_plugin' ... (remaining 8 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
    # bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by bazel-out/host/bin/external/protobuf/protoc)
    # bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by bazel-out/host/bin/external/protobuf/protoc)
    # bazel-out/host/bin/external/protobuf/protoc: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by bazel-out/host/bin/external/protobuf/protoc)
    
  7. 最后,为了避免 GLIBC 依赖,我们必须静态链接一些库,通过添加-lrt链接器标志(也许-lm也是如此)。我发现了多个帖子,建议以不同的方式添加:

    没有-lrt我再次遇到 GLIBC 版本特定的错误,尝试import tensorflow

    # ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/username/anaconda3/envs/myenvname/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow.so)
    

    没有-lm你可能会遇到这个(对我来说,事实证明这是没有必要的)。

  8. 运行构建过程。

    source ./use_gcc_4.9.4.sh
    ./configure
    bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
    bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
    pip install --upgrade /tmp/tensorflow_pkg/tensorflow-0.12.0rc0-cp35-cp35m-linux_x86_64.whl
  1. Try to run the following simple python script to test if the most basic stuff is functioning:

    import tensorflow as tf
    hello = tf.constant('Hello, TensorFlow!')
    sess = tf.Session()
    print(sess.run(hello))
    
    a = tf.constant(10)
    b = tf.constant(32)
    print(sess.run(a + b))
    
  1. 尝试运行以下简单的 python 脚本来测试最基本的东西是否正常运行:

    import tensorflow as tf
    hello = tf.constant('Hello, TensorFlow!')
    sess = tf.Session()
    print(sess.run(hello))
    
    a = tf.constant(10)
    b = tf.constant(32)
    print(sess.run(a + b))