CentOS/RedHat install Tensorflow in HPC

Problem

Tensorflow(a .so library file in the python library path) need the Glibc2.15 + , but the Glibc version in RedHat in HPC is 2.12. This make the tensorflow install fail.

Analysis(Principle)

The .so dynamic library file is linked/loaded to the python executable file by the Dynamic linker/loader in runtime. We can specify the new Glibc’s library file(libc.so.6) path to the LD_LIBRARY_PATHenvironment, but the Glibc and the Dynamic loader/ld.so.2 is corresponding. Thus we also need to specify the new Glibc’s corresponding ld.so.2 file.

Refer to:
http://man7.org/linux/man-pages/man8/ld.so.8.html
http://man7.org/linux/man-pages/man1/ldd.1.html
https://www.cs.virginia.edu/~dww4s/articles/ld_linux.html
https://unix.stackexchange.com/questions/122670/using-alternate-libc-with-ld-linux-so-hacks-cleaner-method

Answer

We explicit specify the ld.so.2’s path in command line instead of the default ld.so.2 file path which specified in the binary file1 2. And use the --library-path parameter of ld.so.2 instead of LD_LIBRARY_PATH environment.3 For example:

${HOME}/local/lib64/ld-linux-x86-64.so.2 --library-path ${HOME}/local/lib64:${HOME}/local/usr/lib64 ${HOME}/anaconda/tensorflow/bin/python

Refer to:
http://xr0038.hatenadiary.jp/entry/2016/08/31/165011
https://stackoverflow.com/questions/33655731/error-while-importing-tensorflow-in-python2-7-in-ubuntu-12-04-glibc-2-17-not-f/34897674#34897674

Set up the environment

modified from4

For the previous steps, please refer to the original blog.

# activate the tensorflow pyenv in the anaconda
source activate tensorflow
LDPATH="${HOME}/local/lib64/ld-linux-x86-64.so.2"
LDLIBS="${HOME}/local/lib64"
LDLIBS="${LDLIBS}:${HOME}/local/usr/lib64"
LDLIBS="${LDLIBS}:/opt/cuda-7.5/lib64"
PYPATH="${HOME}/anaconda/tensorflow/bin/python"

# ${LDPATH} --library-path ${LDLIBS} ${PYPATH}

# ipython は以下のように起動する．
# IPYPATH="${HOME}/.pyenv/versions/TensorFlow/bin/ipython"

# ${LDPATH} --library-path ${LDLIBS} ${PYPATH} ${IPYPATH}

alias python-tensorflow="${LDPATH} --library-path ${LDLIBS} ${PYPATH}"
# alias ipython-tensoflow="${LDPATH} --library-path ${LDLIBS} ${PYPATH} ${IPYPATH}"

Use

~ python-tensorflow
>>> import tensorflow as tf
>>> sess = tf.Session()
>>> exit()

~ python-tensorflow xxx.py

---

1. http://man7.org/linux/man-pages/man8/ld.so.8.html ↩
2. patchELF ↩
3. http://man7.org/linux/man-pages/man8/ld.so.8.html#OPTIONS ↩
4. http://xr0038.hatenadiary.jp/entry/2016/08/31/165011 ↩